在eclipse中编写pyspark代码,需要指定winuntil路径,用以解决 java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. 故障
将winuntils-master.zip 直接释放到pyspark项目的根目录下。
E:\eclipse-workspace\pyspark-project 的目录
2018/10/16 08:03 <DIR> .
2018/10/16 08:03 <DIR> ..
2018/10/14 08:01 386 .project
2018/10/14 08:01 443 .pydevproject
2018/10/15 10:53 <DIR> .settings
2018/10/15 11:23 <DIR> basic
2018/10/16 07:55 <DIR> core
2018/10/14 15:45 <DIR> spark
2018/10/14 15:21 <DIR> src
2018/10/15 11:04 <DIR> winuntil
代码表示为
os.environ['HADOOP_HOME'] = 'E:\\eclipse-workspace\\pyspark-project\\winuntil'
可解决 java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. 报错故障