转载请注明出处:http://blog.csdn.net/dongdong9223/article/details/86030401
本文出自【我是干勾鱼的博客】
Ingredients:
-
Java:Java SE Development Kit 8u162(Oracle Java Archive),Linux下安装JDK修改环境变量
-
Hadoop:hadoop-2.9.1.tar.gz(Apache Hadoop Releases Downloads, All previous releases of Hadoop are available from the Apache release archive site)
-
Hive:hive-2.3.4(mirrors.tuna.tsinghua.edu.cn,Mirror site for Hive)
1 下载Hive
执行命令:
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz
将apache-hive-2.3.4-bin.tar.gz下载到目录:
/opt/hive/
中。
2 解压缩
解压缩:
tar -xzvf apache-hive-2.3.4-bin.tar.gz
3 配置
在文件:
/etc/profile
中加入内容:
# hive
export HIVE_HOME=/opt/hive/apache-hive-2.3.4-bin
export PATH=$HIVE_HOME/bin:$PATH
运行source命令使之生效:
source /etc/profile
4 初始化元数据
4.1 修改“metastore_db”名称为“metastore_db.tmp”
进入Hive的bin目录执行文件夹名称修改命令:
# cd /opt/hive/apache-hive-2.3.4-bin/bin
# mv metastore_db metastore_db.tmp
这里参考了Hive错误:Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000),做这个修改的原因是,在使用Derby存储元数据的时候,如果不进行这个修改,初始化的过程会出现错误:
# schematool -dbType derby -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.derby.sql
Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
4.2 初始化元数据
正如Hive官网Running HiveServer2 and Beeline中所说,从Hive 2.1开始,需要使用schematool命令对元数据进行初始化。元数据既可以保存在Hive自带的Derby数据库上,也可以保存在指定数据库(比如MySQL)上,简单起见这里就是用默认的Derby数据库,初始化命令如下:
schematool -dbType derby -initSchema
5 运行Hive
5.1 安装并启动Hadoop
安装Hadoop可以参考阿里云ECS上搭建Hadoop集群环境——使用两台ECS服务器搭建“Cluster mode”的Hadoop集群环境,启动Hadoop命令为:
sbin/start-all.sh
5.2 创建HDFS上的文件夹及权限设置
$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse
5.3 执行Hive命令
5.3.1 进入Hive
运行命令:
# hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>
5.3.2 查看所有数据库
hive> show databases;
OK
default
Time taken: 0.02 seconds, Fetched: 1 row(s)
5.3.3 查看所有数据表
hive> show tables;
OK
Time taken: 0.049 seconds
6 参考
阿里云ECS上搭建Hadoop集群环境——使用两台ECS服务器搭建“Cluster mode”的Hadoop集群环境
Hive错误:Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000)
关于hive异常:Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStor