spark.debug.maxToStringFields

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/qq_35744460/article/details/83650603

spark运行报错

 Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

需要设置.config("spark.debug.maxToStringFields", "100")

SparkSession
        .builder()
        .master("local[4]")
        .appName("report")
        .config("spark.debug.maxToStringFields", "100")
        .getOrCreate();

猜你喜欢

转载自blog.csdn.net/qq_35744460/article/details/83650603