在jar包中不能直接获取File文件,应当使用InputStream接收文件
原代码:
public static FileSystem getFileSystem(String hadoopConfPath) throws IOException {
Configuration conf = new Configuration();
File coreSiteFile= ResourceUtils.getFile("classpath:"+hadoopConfPath + "/core-site.xml");
File hdfsFile = ResourceUtils.getFile("classpath:"+hadoopConfPath + "/hdfs-site.xml");
if(!coreSiteFile.exists()){
throw new RuntimeException("core-site.xml is not in " + hadoopConfPath);
}
if(!hdfsFile.exists()){
throw new RuntimeException("core-site.xml is not in " + hadoopConfPath);
}
conf.addResource(corSiteFileStream);
conf.addResource(hdfsFileStream);
FileSystem fs = FileSystem.get(conf);
return fs;
}
打成jar包放入Linux,路径后报错,如下:
"class path resource [config/hadoop-conf.test/hadoop-conf/core-site.xml] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/data/***/***.jar!/BOOT-INF/classes!/config/hadoop-conf.test/hadoop-conf/core-site.xml"
更正后:
public static FileSystem getFileSystem(String hadoopConfPath) throws IOException {
Configuration conf = new Configuration();
InputStream corSiteFileStream = HDFSUtil.class.getClassLoader().getResourceAsStream(hadoopConfPath + "/core-site.xml");
InputStream hdfsFileStream=HDFSUtil.class.getClassLoader().getResourceAsStream(hadoopConfPath + "/hdfs-site.xml");
conf.addResource(corSiteFileStream);
conf.addResource(hdfsFileStream);
FileSystem fs = FileSystem.get(conf);
return fs;
}