一、HDFS 文件上传
@Test public void testCopyFromLocal() throws URISyntaxException, IOException, InterruptedException { // 1. 获取 fs 对象 Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(new URI("hdfs://wt1:9000"), conf, "tom"); // 2. 执行上传 fs.copyFromLocalFile(new Path("D:\\test.txt"), new Path("/")); // 3. 关闭资源 fs.close(); }
优先级 由高到底
1、代码
Configuration conf = new Configuration(); // 设置备份 conf.set("dfs.replication", "2");
2、项目配置文件
src\main\resources\hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration>
3、节点配置文件
二、文件下载
@Test public void testCopyToLocal() throws URISyntaxException, IOException, InterruptedException { // 1. 获取 fs 对象 Configuration conf = new Configuration(); // 设置备份 FileSystem fs = FileSystem.get(new URI("hdfs://wt1:9000"), conf, "tom"); // 2. 执行下载 fs.copyToLocalFile(new Path("/wt/li/test.txt"), new Path("E:\\abc")); // 3. 关闭资源 fs.close(); }
三、文件删除
@Test public void testDeleteFile() throws URISyntaxException, IOException, InterruptedException { // 获取 fs 对象 Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(new URI("hdfs://wt1:9000"), conf, "tom"); // 执行删除 fs.delete(new Path("/wt/li/test.txt"), true); // 关闭 fs fs.close(); }