20/09/27 18:32:57 WARN ObjectStore: Failed to get database spark, returning NoSuchObjectException
Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘spark’ not found;
项目场景:
spark整合Hive,idea本地用spark读取Hive中的数据
问题描述:
在hive客户端查看数据库spark是存在的但是用spark连接Hive查不到数据库spark
异常如下
20/09/27 18:32:57 WARN ObjectStore: Failed to get database spark, returning NoSuchObjectException
Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘spark’ not found;
hive客户端查看数据库
hive> show databases;
OK
default
spark
Time taken: 0.11 seconds, Fetched: 3 row(s)
原因分析:
spark没找到Hive的元数据
解决方案:
加上元数据配置,hive.metastore.uris
scala中整合Hive
hdfs-site.xml中加入配置
<property>
<name>hive.metastore.uris</name>
<value>thrift://node003:9083</value>
</property>
java中spark整合Hive
spark加入配置属性config(“hive.metastore.uris”,“thrift://node003:9083”)
SparkSession spark = SparkSession.builder().appName("随便写").config("hive.metastore.uris","thrift://node003:9083").enableHiveSupport().getOrCreate();