hadoop-1.2.1 伪分布配置

首先JDK安装及相关环境变量配置

# Java environment setting
JAVA_HOME=/usr/java/default
CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
PATH=$JAVA_HOME/bin:$PATH
JRE_HOME=$JAVA_HOME/jre
export JAVA_HOME CLASSPATH PATH JRE_HOME # Maven configuration
M2_HOME=/usr/local/apache-maven/default
PATH=$PATH:$M2_HOME/bin # hadoop setting
HADOOP_HOME=/home/asn/opt/hadoop-1.2.
PATH=$PATH:$HADOOP_HOME/bin

1. 修改主机名

[asn@vm-centos6-asn opt]$ cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=vm-centos6-asn

在/etc/hosts文件中,添加主机名映射

[root@vm-centos6-asn ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 vm-centos6-asn
:: localhost localhost.localdomain localhost6 localhost6.localdomain6

2. hadoop配置

1)core-site.xml

 <?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.56.101:9000</value>
</property> <property>
<name>hadoop.tmp.dir</name>
<value>/home/asn/opt/hadoop-tmp</value>
</property>
</configuration>

2) hdfs-site.xml

 <?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>dfs.data.dir</name>
<value>/home/asn/opt/hadoop-data</value>
</property> <property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

3) mapred-site.xml

 <?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property>
<name>mapred.job.tracker</name>
<value>192.168.56.101:9001</value>
</property>
</configuration>

4)  设置hadoop-env.sh文件中的JAVA_HOME环境变量

[asn@vm-centos6-asn conf]$ cat hadoop-env.sh
# Set Hadoop-specific environment variables here.

JAVA_HOME环境变量必须配置,其他环境变量可选。 当运行一个分布式配置时,最好配置该文件中的JAVA_HOME环境变量,以便于在远程节点上被正确定义。
# The only required environment variable is JAVA_HOME. All others are optional.
# When running a distributed configuration it is best to
# set JAVA_HOME in this file, so that it is correctly defined on remote nodes. # The java implementation to use. Required.
export JAVA_HOME=/usr/java/default # Extra Java CLASSPATH elements. Optional.
# export HADOOP_CLASSPATH= # The maximum amount of heap to use, in MB. Default is .
# export HADOOP_HEAPSIZE= # Extra Java runtime options. Empty by default.
# export HADOOP_OPTS=-server

3. 免密码登陆ssh配置

[asn@vm-centos6-asn ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/asn/.ssh/id_rsa): 私钥存放文件
Created directory '/home/asn/.ssh'.
Enter passphrase (empty for no passphrase): 直接回车
Enter same passphrase again: 直接回车
Your identification has been saved in /home/asn/.ssh/id_rsa.
Your public key has been saved in /home/asn/.ssh/id_rsa.pub. 公钥存放文件
The key fingerprint is:
6e::e0:::b5:6b:fb:::::1e:b2:8f: asn@vm-centos6-asn
The key's randomart image is:
+--[ RSA ]----+
| ... ..oo= |
| . . . = . |
| . . . . . E|
| . . . o . |
| = o S . o |
| . = = . |
| . + + |
| . + |
| . |
+-----------------+
[asn@vm-centos6-asn ~]$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys 把公钥拷贝到认证文件中 [asn@vm-centos6-asn ~]$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is bd:c8:f0:::be:ee:2f::6d::::d7:c0:.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts. 首次连接提示是否把localhost添加到已知的主机列表中
Last login: Tue Dec :: from localhost
[asn@vm-centos6-asn ~]$ exit
logout
Connection to localhost closed. [asn@vm-centos6-asn ~]$ ssh localhost 直接免密码登录成功
Last login: Tue Dec :: from localhost
[asn@vm-centos6-asn ~]$

4. hadoop  namenode -format  格式化名字节点

5. start-all.sh 启动

[asn@vm-centos6-asn ~]$ start-all.sh
starting namenode, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-namenode-vm-centos6-asn.out
localhost: starting datanode, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-datanode-vm-centos6-asn.out
localhost: starting secondarynamenode, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-secondarynamenode-vm-centos6-asn.out
starting jobtracker, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-jobtracker-vm-centos6-asn.out
localhost: starting tasktracker, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-tasktracker-vm-centos6-asn.out

[asn@vm-centos6-asn ~]$ jps
Jps
NameNode
SecondaryNameNode
JobTracker
TaskTracker
DataNode
上一篇:host_network_interfaces_slow_mode_thresholds


下一篇:第26章 java进制操作