在做一个hadoop项目中,利用HDFS存储日志文件。为了将未关闭的文件进行恢复,我们需要判断一个文件是否为正在打开的状态。fs.getClient().namenode.getBlockLocations(path.toUri().getPath(), 0,
fs.getFileStatus(path).getLen()).isUnderConstruction();通过这个调用可以获取是否在打开。再调用fs.recoverLease(path)去强制关闭该文件。由于我们在getBlockLocations中最初用path.toString方法去调,结果导致以下NullPointException
org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException at org.apache.hadoop.ipc.Client.call(Client.java:1107) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226) at $Proxy0.getBlockLocations(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at $Proxy0.getBlockLocations(Unknown Source) at com.webex.da.hdc.web.DFSServlet.isOpen(DFSServlet.java:272) at com.webex.da.hdc.web.DFSServlet.rename(DFSServlet.java:255) at com.webex.da.hdc.web.DFSServlet.handleRename(DFSServlet.java:182) at com.webex.da.hdc.web.DFSServlet.doPost(DFSServlet.java:93) at javax.servlet.http.HttpServlet.service(HttpServlet.java:641) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
经过仔细调查,发现是path的问题,path.toString将会有hdfs://namenode前缀,而这个接口不应该包含这个前缀。因此我们在调用HDFS接口时一定要消息,如果要传递path为字符串,不应该包含前缀