提交example案例到YARN上运行

0、使用官方例子,将MapReduce提交到YARN上运行

1、进入文件夹

[hadoop@hadoop000 mapreduce]$ pwd
/home/hadoop/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/mapreduce
[hadoop@hadoop000 mapreduce]$ ls
hadoop-mapreduce-client-app-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-common-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-core-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-hs-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-jobclient-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-jobclient-2.6.0-cdh5.15.1-tests.jar
hadoop-mapreduce-client-nativetask-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-shuffle-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar

2、运行jar包里的类

(1)提示:计算pi值

pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.

(2)提示:使用方法

Usage: org.apache.hadoop.examples.QuasiMonteCarlo <nMaps> <nSamples>

(3)运行运行jar包里的类

[hadoop@hadoop000 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar pi 2 3

(4)刷新浏览器界面All Applications

3、wordcount

(1)找到wordcount

[hadoop@hadoop000 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar

(2)使用vi重新创建文件

[hadoop@hadoop000 data]$ pwd
/home/hadoop/data
[hadoop@hadoop000 data]$ vi data.txt

hello   world   welcome
Hello   hello   welcome

(3)在hadoop中多层创建input文件,并传入文件

[hadoop@hadoop000 data]$ hadoop fs -mkdir -p /wc/input

[hadoop@hadoop000 data]$ hadoop fs -put data.txt /wc/input

(4)执行

[hadoop@hadoop000 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar wordcount /wc/input/data.txt /wc/output

(5)查看结果

没有区分大小写,也不能自动删除重复性文档。

[hadoop@hadoop000 mapreduce]$ hadoop fs -text /wc/output/part-r-00000
上一篇:Spark的Monitoring


下一篇:Hadoop停止集群以及如何单个进程启动