hdfs client access the hdfs cluster not in one domain

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html#Clients_use_Hostnames_when_connecting_to_DataNodes

sometimes when you access your hadoop cluster outside the internal cluster, you maybe encounter some route problem.

1, how to expose you hdfs service on internet?

the hdfs client communicate the master( if the master can access) ,but the master return datanodes's internal ip or hostname,

the client can not do the correct name resolution for the host name or can not access the internal ip.

for the name resolution, if the data node ip can access, you should set the parameter

dfs.client.use.datanode.hostname false Whether clients should use datanode hostnames when connecting to datanodes.

2.if you use python to access hdfs, you should pay attention to another configure item.

  dfs.webhdfs.enabled  -> true

because python hdfs module use webhdfs to access hadoop by default.

上一篇:java Ajax的应用


下一篇:[位运算] [搜索] [递推优化] [计算几何] TEST 2016.7.15