当我们安装好并正常运行hdfs后输入http://xxxxxxxxx:50070会进入下图所示的页面。 其中Browse the filesystem 是查看文件系统的入口。
但是在发现这个链接一直无效。通过Chrome的开发工具可以看这个链接访问地址是：nn_browsedfscontent.jsp
下面是nn_browsedfscontent.jsp的代码
<%@ page
contentType="text/html; charset=UTF-8"
import="java.io.*"
import="java.security.PrivilegedExceptionAction"
import="java.util.*"
import="javax.servlet.*"
import="javax.servlet.http.*"
import="org.apache.hadoop.conf.Configuration"
import="org.apache.hadoop.hdfs.*"
import="org.apache.hadoop.hdfs.server.namenode.*"
import="org.apache.hadoop.hdfs.server.datanode.*"
import="org.apache.hadoop.hdfs.protocol.*"
import="org.apache.hadoop.hdfs.security.token.delegation.*"
import="org.apache.hadoop.io.Text"
import="org.apache.hadoop.security.UserGroupInformation"
import="org.apache.hadoop.security.token.Token"
import="org.apache.hadoop.util.*"
import="java.text.DateFormat"
import="java.net.InetAddress"
import="java.net.URLEncoder"
%>
<%!
static String getDelegationToken(final NameNode nn,
HttpServletRequest request, Configuration conf)
throws IOException, InterruptedException {
final UserGroupInformation ugi = JspHelper.getUGI(request, conf);
Token<DelegationTokenIdentifier> token =
ugi.doAs(
new PrivilegedExceptionAction<Token<DelegationTokenIdentifier>>()
{
public Token<DelegationTokenIdentifier> run() throws IOException {
return nn.getDelegationToken(new Text(ugi.getUserName()));
}
});
return token.encodeToUrlString();
} public void redirectToRandomDataNode(
NameNode nn,
HttpServletRequest request,
HttpServletResponse resp,
Configuration conf
) throws IOException, InterruptedException {
String tokenString = null;
if (UserGroupInformation.isSecurityEnabled()) {
tokenString = getDelegationToken(nn, request, conf);
}
FSNamesystem fsn = nn.getNamesystem();
String datanode = fsn.randomDataNode();
String redirectLocation;
String nodeToRedirect;
int redirectPort;
if (datanode != null) {
redirectPort = Integer.parseInt(datanode.substring(datanode.indexOf(':')
+ 1));
nodeToRedirect = datanode.substring(0, datanode.indexOf(':'));
}
else {
nodeToRedirect = nn.getHttpAddress().getHostName();
redirectPort = nn.getHttpAddress().getPort();
}
String fqdn = InetAddress.getByName(nodeToRedirect).getCanonicalHostName();
redirectLocation = "http://" + fqdn + ":" + redirectPort +
"/browseDirectory.jsp?namenodeInfoPort=" +
nn.getHttpAddress().getPort() +
"&dir=/" +
(tokenString == null ? "" :
JspHelper.getDelegationTokenUrlParam(tokenString));
resp.sendRedirect(redirectLocation);
}
%> <html> <title></title> <body>
<%
NameNode nn = (NameNode)application.getAttribute("name.node");
Configuration conf = (Configuration) application.getAttribute(JspHelper.CURRENT_CONF);
redirectToRandomDataNode(nn, request, response, conf);
%>
<hr> <h2>Local logs</h2>
<a href="/logs/">Log</a> directory <%
out.println(ServletUtil.htmlFooter());
%>
从代码中可以看出实际是跳转到一台datanode的browseDirectory.jsp，如：
http://xxxxxxx:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=/
而xxxxxxx是集群中一台datanode的主机名。
那么就可以从以下两个点出发去看为什么无效：
1.50075端口是否正常
2.使用的机器是否能通过主机面访问datanode
检查之后发现自己这两个都没有配置......
最后通过在hdfs-site.xml中添加如下配置
<property>
<name>dfs.datanode.http.address</name>
<value>10.0.0.234:50075</value>
</property>
在hosts中添加datanode的主机命和ip修复了这个问题
记录、共勉。