前言
由于以前已经写过一篇Centos 7环境下安装配置2.8的随笔,因此这篇写得精简些,只挑选一些重要环节记录一下。
安装环境为:两台主机均为Centos 7.*操作系统,两台机器配置分别为:
主机centos-hadBeta1:IP为:10.10.11.225
从机hadoopnode1:IP为:10.10.11.254
1.配置etc/hadoop文件夹中的文件:
配置Hadoop-env.sh文件中的Java_Home变量(/etc/profile文件配置与此相同)。
export JAVA_HOME=jdk安装路径
core-size.xml文件配置项为:
<property>
<name>fs.defaultFS</name>
<value>hdfs://10.10.11.225:9000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/hadoop/temp/hadoopdata</value>
</property>
hdfs-site.xml文件配置项为:
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/usr/hadoop/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/usr/hadoop/hdfs/data</value>
</property>
yarn-site.xml文件配置项为:
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>10.10.11.225:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>10.10.11.225:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>10.10.11.225:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>10.10.11.225:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>10.10.11.225:8088</value>
</property>
mapred-site.xml文件配置项为:
<property>
<name>mapred.job.tracker</name>
<value>http://10.10.11.225:9001</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>
/usr/hadoop/etc/hadoop,
/usr/hadoop/share/hadoop/common/*,
/usr/hadoop/share/hadoop/common/lib/*,
/usr/hadoop/share/hadoop/hdfs/*,
/usr/hadoop/share/hadoop/hdfs/lib/*,
/usr/hadoop/share/hadoop/mapreduce/*,
/usr/hadoop/share/hadoop/mapreduce/lib/*,
/usr/hadoop/share/hadoop/yarn/*,
/usr/hadoop/share/hadoop/yarn/lib/*
</value>
</property>
works文件配置如下(在hadoop 3.0 alpha4版本以前为slaves文件):
10.10.11.254
2.账户配置
在sbin/start-yarn.sh、stop-yarn.sh、start-dfs.sh、stop-dfs.sh文件中分别添加hadoop 3.0版本所需账户,配置如下:
YARN_RESOURCEMANAGER_USER=root
YARN_NODEMANAGER_USER=root
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root
HDFS_DATANODE_USER=root
3.初始化
将从机(hadoopnode1)也如配置完成后,初始化Hadoop,命令如下:
hdfs namenode -format
4.启动结果
配置完成后,运行start-all命令启动hadoop,即可见到datanode启动后结果,如下图所示: