HDFS API 操作实例(一) HDFS读写操作

1. 读取HDFS文件

1.1 字符读取HDFS上的文件

   Configuration conf = new Configuration();
   Path path = new Path(pathstr); 
   FileSystem fs = FileSystem.get(conf);
   FSDataInputStream fsin= fs.open(path); 
   BufferedReader br =null;
   String line ;
   try{
    br = new BufferedReader(new InputStreamReader(fsin));
       while ((line = br.readLine()) != null) {
         System.out.println(line);
        } 
   }finally{
    br.close();
}

2. 写入HDFS文件

2.1 字节写入HDFS文件

  Configuration conf = new Configuration();
  Path path = new Path(path); 
  FileSystem fs = FileSystem.get(conf); 
  FSDataOutputStream out = fs.create(src);
  out.write(sb.toString().getBytes());
  out.close();

2.2 HDFS 文件中追加(append)数据

public static boolean appendRTData(String src, String drc) { // src源文件 drc 追加文件
    boolean flag = false;
 
    Configuration conf = new Configuration();
    FileSystem fs = null;
    try {
      fs = FileSystem.get(URI.create(src), conf);
      InputStream in = new BufferedInputStream(new FileInputStream(drc));
      OutputStream out = fs.append(new Path(src));
      IOUtils.copyBytes(in, out, 4096, true);
    } catch (IOException e) {
      e.printStackTrace();
    }
 
    return flag;
  }

猜你喜欢

转载自www.cnblogs.com/yyy-blog/p/10525760.html