Failed to submit local jar to spark cluster: java.nio.file.NoSuchFileException

The standalone mode cluster wants to pass jar files to hdfs because the driver is on any node in the cluster.

hdfs dfs -put xxx.jar /user/
spark-submit --master spark://xxx:7077
–deploy-mode cluster
–supervise
–driver-memory 512m
–total-executor-cores 1
–executor-memory 512m
–executor-cores 1
–class com.xiyou.bi.streaming.game.common.DmMoGameviewOnlineLogic
hdfs://xxx:8020/user/hutao/xxx.jar

猜你喜欢

转载自blog.csdn.net/weixin_43654136/article/details/84586952