安装Hadoop-单机模式(1)
一、在Ubuntu下创建hadoop组和hadoop用户
1)创建hadoop用户组
sudo addgroup hadoop //添加用户组
2)创建hadoop用户
sudo adduser -ingroup hadoop hadoop // 创建hadoop用户,需要输入新的密码
3、为hadoop用户添加权限,打开/etc/sudoers文件,添加:
root ALL=(ALL:ALL) ALL
hadoop ALL=(ALL:ALL) ALL //添加这行, 给hadoop用户赋予和root用户同样的权限
二、用新增加的hadoop用户登录Ubuntu系统
su hadoop //切换到hadoop用户,或者直接从设置->切换用户
三、安装ssh
1)安装ssh
sudo apt-get install openssl-server //安装ssh
2)重启,并检查服务是否启动
sudo /etc/init.d/ssh start //重启服务
ps -e | grep ssh //查看服务是否正确启动
3)设置免密码登录,生成私钥和公钥,在/home/hadoop/.ssh下生成两个文件:id_rsa和id_rsa.pub
ssh-keygen -t rsa -P "" //一路回车就行
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys //公钥追加到authorized_keys
ssh localhost //登录测试是否成功,之后exit退出
四、安装Java环境
sudo apt-get install openjdk--jdk //安装java
java -version //检测是否安装成功
如果出错,按这种方式(测试成功):
)下载jdk:http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html )解压到指定目录:sudo tar xvf ./jdk-8u5-linux-x64.tar.gz -C /usr/lib/jvm )改名字: sudo mv jdk1..0_05/ java-8u5-sun
)在 ~/.bashrc文件加入,并source一下
export JAVA_HOME=/usr/lib/jvm/java-8u5-sun
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
)检查是否成功:java -version
五、安装hadoop2.4.0
1)官网下载 :http://mirror.bit.edu.cn/apache/hadoop/common/
2)安装,并修改权限
sudo tar xzf hadoop-2.4..tar.gz //解压
sudo mv hadoop-2.4. /usr/local/hadoop //移动到指定目录
sudo chmod /usr/local/hadoop //修改权限
3)配置,执行update-alternatives --config java,查找java安装路径 :/usr/lib/jvm/java-8u5-sun,在.bashrc文件添加:
#HADOOP VARIABLES START
export JAVA_HOME=/usr/lib/jvm/java--openjdk-amd64
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
#HADOOP VARIABLES END
4)修改JAVA_HOME变量,打开:/usr/local/hadoop/etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr/lib/jvm/java-8u5-sun //修改环境变量
六、WordCount测试
1)在/usr/local/hadoop路径下创建input文件夹 :mkdir input
2)拷贝README.txt到input : cp README.txt input
3)执行WordCount:
sudo bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.2-sources.jar org.apache.hadoop.examples.WordCount input output //红色换成自己路径下的jar包
4)执行 cat output/*,查看字符统计结果
ref:http://www.cnblogs.com/kinglau/p/3794433.html