cdh5 mapreduce 例子
所属分类 bigdata
浏览量 1268
/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/jars
ls -a|grep example
hadoop-examples-2.6.0-mr1-cdh5.14.2.jar
hadoop-examples.jar
hadoop-mapreduce-examples-2.6.0-cdh5.14.2.jar
hbase-examples-1.2.0-cdh5.14.2.jar
mahout-examples-0.9-cdh5.14.2.jar
mahout-examples-0.9-cdh5.14.2-job.jar
oozie-examples.jar
spark-examples-1.6.0-cdh5.14.2-hadoop2.6.0-cdh5.14.2.jar
PI计算
hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.14.2.jar pi 5 5
单词统计 wordcount
test.txt
hello hadoop
apache hadoop is a framework for running applications on large cluster built of commodity hardware
the hadoop framework transparently provides applications both reliability and data motion
hadoop implements a computational paradigm named Map/Reduce
where the application is divided into many small fragments of work
each of which may be executed or re-executed on any node in the cluster
hadoop fs -ls /
hadoop fs -mkdir /tmp/dugang/
hadoop fs -ls /tmp/dugang
hadoop fs -put test.txt /tmp/dugang
hadoop fs -cat /tmp/dugang/test.txt
hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.14.2.jar wordcount /tmp/dugang/test.txt /tmp/dugang/out
输出到 /tmp/dugang/out 目录中
hadoop fs -ls /tmp/dugang/out
Found 33 items
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/_SUCCESS
-rw-r--r-- 3 root supergroup 16 2019-10-09 12:55 /tmp/dugang/out/part-r-00000
-rw-r--r-- 3 root supergroup 7 2019-10-09 12:55 /tmp/dugang/out/part-r-00001
-rw-r--r-- 3 root supergroup 26 2019-10-09 12:55 /tmp/dugang/out/part-r-00002
-rw-r--r-- 3 root supergroup 7 2019-10-09 12:55 /tmp/dugang/out/part-r-00003
-rw-r--r-- 3 root supergroup 31 2019-10-09 12:55 /tmp/dugang/out/part-r-00004
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00005
-rw-r--r-- 3 root supergroup 30 2019-10-09 12:55 /tmp/dugang/out/part-r-00006
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00007
-rw-r--r-- 3 root supergroup 20 2019-10-09 12:55 /tmp/dugang/out/part-r-00008
-rw-r--r-- 3 root supergroup 11 2019-10-09 12:55 /tmp/dugang/out/part-r-00009
-rw-r--r-- 3 root supergroup 10 2019-10-09 12:55 /tmp/dugang/out/part-r-00010
-rw-r--r-- 3 root supergroup 18 2019-10-09 12:55 /tmp/dugang/out/part-r-00011
-rw-r--r-- 3 root supergroup 23 2019-10-09 12:55 /tmp/dugang/out/part-r-00012
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00013
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00014
-rw-r--r-- 3 root supergroup 23 2019-10-09 12:55 /tmp/dugang/out/part-r-00015
-rw-r--r-- 3 root supergroup 17 2019-10-09 12:55 /tmp/dugang/out/part-r-00016
-rw-r--r-- 3 root supergroup 37 2019-10-09 12:55 /tmp/dugang/out/part-r-00017
-rw-r--r-- 3 root supergroup 7 2019-10-09 12:55 /tmp/dugang/out/part-r-00018
-rw-r--r-- 3 root supergroup 11 2019-10-09 12:55 /tmp/dugang/out/part-r-00019
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00020
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00021
-rw-r--r-- 3 root supergroup 6 2019-10-09 12:55 /tmp/dugang/out/part-r-00022
-rw-r--r-- 3 root supergroup 9 2019-10-09 12:55 /tmp/dugang/out/part-r-00023
-rw-r--r-- 3 root supergroup 13 2019-10-09 12:55 /tmp/dugang/out/part-r-00024
-rw-r--r-- 3 root supergroup 24 2019-10-09 12:55 /tmp/dugang/out/part-r-00025
-rw-r--r-- 3 root supergroup 21 2019-10-09 12:55 /tmp/dugang/out/part-r-00026
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00027
-rw-r--r-- 3 root supergroup 8 2019-10-09 12:55 /tmp/dugang/out/part-r-00028
-rw-r--r-- 3 root supergroup 20 2019-10-09 12:55 /tmp/dugang/out/part-r-00029
-rw-r--r-- 3 root supergroup 26 2019-10-09 12:55 /tmp/dugang/out/part-r-00030
-rw-r--r-- 3 root supergroup 0 2019-10-09 12:55 /tmp/dugang/out/part-r-00031
hadoop fs -cat /tmp/dugang/out/part-r-00030
running 1
transparently 1
hadoop fs -cat /tmp/dugang/out/part-r-00000
a 2
many 1
on 2
上一篇
下一篇
hdfs namenode HA 高可用
HDFS Federation 机制
CDH 5.14.2 各种角色
HDFS特点及优缺点
hdfs安全模式
文件内容对比技巧