datanode无法连接到master

初次在VM上配置Hadoop,开了三台虚拟机,一个作namenode,jobtracker

另外两台机子作datanode,tasktracker

配置好后,启动集群

通过http://localhost:50700查看cluster状况

Hadoop配置datanode无法连接到master

发现没有datanode

检查结点,发现datanode 进程已经启动,查看datanode机器上的日志

2014-03-01 22:11:17,473 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

2014-03-01 22:11:18,477 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2014-03-01 22:11:19,481 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2014-03-01 22:11:20,485 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-03-01 22:11:21,489 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

发现datanode 无法连接到master ,但是经过尝试,可以ping通,到结点查看,9000端口也处于监听状态,百思不得其解

最终发现core-site.xml 中

<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>

才意识到监听127.0.0.1端口,并不能被外机访问

改为主机名,一切正常

<property>
<name>fs.default.name</name>
<value>hdfs://Master:9000</value>
</property>

hosts:

10.207.160.34 Master

不要以为这样就好了, 其实这是个坑来的..不信你试试:

root@VM_160_34_centos:~>  netstat -an | grep
tcp 127.0.0.1: 0.0.0.0:* LISTEN
tcp 127.0.0.1: 127.0.0.1: ESTABLISHED
tcp 127.0.0.1: 127.0.0.1: ESTABLISHED

这里要直接写IP, 反正我是被坑了.

<value>hdfs://10.207.160.34:9000</value> 

看现在就好了

hadoop@VM_160_34_centos:/usr/local/hadoop-2.4.> netstat -nlpa|grep
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 10.207.160.34: 0.0.0.0:* LISTEN /java
tcp 10.207.160.34: 10.207.160.34: TIME_WAIT -
tcp 10.207.160.34: 10.221.32.234: ESTABLISHED /java

我们再 report 一下:

hadoop@VM_160_34_centos:/usr/local/hadoop-2.4.> hadoop dfsadmin -report
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it. Configured Capacity: (7.87 GB)
Present Capacity: (4.50 GB)
DFS Remaining: (4.50 GB)
DFS Used: ( KB)
DFS Used%: 0.00%
Under replicated blocks:
Blocks with corrupt replicas:
Missing blocks: -------------------------------------------------
Datanodes available: ( total, dead) Live datanodes:
Name: 10.221.32.234: (Node1)
Hostname: TENCENT64.site
Decommission Status : Normal
Configured Capacity: (7.87 GB)
DFS Used: ( KB)
Non DFS Used: (3.37 GB)
DFS Remaining: (4.50 GB)
DFS Used%: 0.00%
DFS Remaining%: 57.18%
Configured Cache Capacity: ( B)
Cache Used: ( B)
Cache Remaining: ( B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Last contact: Fri Aug :: CST
上一篇:微信公众平台通用接口API指南


下一篇:java输入最大10位数,倒数输出(很鸡肋)