目录
安装MySQL:
1. 检查当前环境是否安装mysql服务(命令:rpm -qa | grep -i mysql)
2. 卸载自带的mysql
3. 卸载软件:rpm -e --nodeps mysql-libs-5.1.71-1.el6.x86_64
4. 安装mysql客户端与服务器.
1 上传安装包
sftp> cd /opt/package/
master:
MySQL-client-5.5.45-1.linux2.6.x86_64.rpm
MySQL-server-5.5.45-1.linux2.6.x86_64.rpm
2 解压
MySQL:
[root@master package]# rpm -ivh MySQL-client-5.5.45-1.linux2.6.x86_64.rpm --nodeps –force
[root@master package]# rpm -ivh MySQL-server-5.5.45-1.linux2.6.x86_64.rpm --nodeps --force
3 配置MySQL(slave1中安装并配置)
[root@master package]# /usr/bin/mysql_install_db
[root@master package]# service mysql start
检查MySQL状态 service mysql status
启动MySQL服务 service mysql start
停止MySQL服务 service mysql stop
重启MySQL服务 service mysql restart
启动报错:error!The server quit without updating PID file
解决:将/etc/mysql下的my.cnf文件删除,再次启动MySQL服务
mysql登陆
mysql> use mysql;
mysql> select host,user,password from user;
mysql> update user set password=password(‘mysql123’) where user='root';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'mysql123' WITH GRANT OPTION;
mysql> flush privileges;
mysql> exit;
[root@master package]# chkconfig mysql on
[root@master package]# chkconfig --list | grep mysql
mysql 0:off 1:off 2:on 3:on 4:on 5:on 6:off |
使用新密码登陆:
[root@master package]# mysql -u root -p
Enter password:mysql123
可以登录进去就对了。如果不输入账号和密码直接进去,就错了。
Hive搭建:
安装前准备工作:
安装JDK,Hadoop,MySQL,MySQL的驱动包
安装分类:
内嵌、独立(本地)、远程三种安装模式。
内嵌模式(元数据保在内嵌的derby种,允许一个会话链接,尝试多个会话链接时会报错)
本地模式(本地安装mysql 替代derby存储元数据)
远程模式(远程安装mysql 替代derby存储元数据)
Hive远程安装:
1 上传安装包
sftp> cd /opt/package/
apache-hive-1.2.1-bin.tar.gz
mysql-connector-java-5.1.32.tar.gz
2 进入MySQL查看用户与权限:
$ mysql -uroot -proot
mysql> select host,user from user;
3 在MySQL中创建一个用户并授权给hive使用
mysql> CREATE USER 'hive'@'%' IDENTIFIED BY 'mysq123';
mysql> grant all privileges on *.* to hive@'%' identified by 'mysql123';
mysql> grant all on *.* to hive@'localhost' identified by 'mysql123';
mysql> grant all on *.* to hive@'master' identified by 'mysql123';
mysql> flush privileges;
mysql> create database hivedb;
查看用户添加成功没:
mysql> select host,user from user;
4 配置环境变量并更新
$ vi /etc/profile
export HIVE_HOME=/opt/software/hive-1.2.1
export PATH=$PATH:$HIVE_HOME/bin
刷新保存配置文件
$ source /etc/profile
5 修改文件名
[root@master opt]# cd /opt/software/hive-1.2.1/conf/
[root@master conf]# mv hive-default.xml.template hive-site.xml
[root@master conf]# mv hive-log4j.properties.template hive-log4j.properties
[root@master conf]# mv hive-exec-log4j.properties.template hive-exec-log4j.properties
[root@master conf]# mv hive-env.sh.template hive-env.sh
6 修改hive-env.sh文件
[root@masterconf]# vi hive-env.sh
export JAVA_HOME=/opt/software/jdk1.8.0_162
export HADOOP_HOME=/opt/software/hadoop-2.7.3
export HIVE_HOME=/opt/software/hive-1.2.1
export HIVE_CONF_DIR=/opt/software/hive-1.2.1/conf
7 编辑hive-log4j.properties文件
[root@master conf]# vi hive-log4j.properties
修改dir:
hive.log.dir=/opt/software/hive-1.2.1/logs
创建logs目录:
[root@master hive-1.2.1]# mkdir /opt/software/hive-1.2.1/logs
8 编辑hive-site.xml文件
cd /opt/software/hive-1.2.1/conf/
[root@master conf]# vi hive-site.xml
删除里面所有内容,添加如下内容:
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://ns1/hive/warehouse</value>
</property>
<!--配置hive元数据库存储,本地、远程的区别就在这里-->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://master:3306/hivedb?characterEncoding=UTF-8</value>
</property>
<!-- HDFS路径,用于存储不同 map/reduce 阶段的执行计划和这些阶段的中间输出结果 -->
<!--给hive添加mysql数据库的驱动包,将驱动包添加到lib目录下即可-->
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<!-- mysql登陆名-->
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<!-- mysql登陆密码-->
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>mysql123</value>
</property>
<!--不配置可能会产生错误的配置-->
<property>
<name>hive.exec.scratchdir</name>
<value>hdfs://ns1/hive/scratchdir</value>
</property>
<property>
<name>hive.querylog.location</name>
<value>/opt/software/hive-1.2.1/logs</value>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/opt/software/hive-1.2.1/resources</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
<property>
<name>hive.server2.logging.operation.log.location</name>
<value>/opt/software/hive-1.2.1/operation_logs</value>
<description>Top level directory where operation logs are stored if logging functionality is enabled</description>
</property>
9 解压MySQL的驱动包
将MySQL-connect-Java.bin.jar cp到hive下:
[root@master lib]# cp mysql-connector-java-5.1.32-bin.jar /opt/software/hive-1.2.1/lib/
将hive/lib下的 jline-2.12.jar包复制到hadoop下,不然会报错。hadoop原有jar包删除;
[root@master lib]# cp jline-2.12.jar /opt/software/hadoop-2.7.3/share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib
[root@master lib]# cp jline-2.12.jar /opt/software/hadoop-2.7.3/share/hadoop/yarn/lib
[root@master lib]# cp jline-2.12.jar /opt/software/hadoop-2.7.3/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib
[root@master lib]# rm -rf /opt/software/hadoop-2.7.3/share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/jline-0.9.94.jar
[root@master lib]# rm -rf /opt/software/hadoop-2.7.3/share/hadoop/yarn/lib/jline-0.9.94.jar
[root@master lib]# rm -rf /opt/software/hadoop-2.7.3/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/jline-0.9.94.jar
查看:
[root@master hadoop-2.7.3]# find / -name jline*
10 运行hive
启动hive前要先启动zookeeper和Hadoop集群!!!!
启动:
启动之前可以先观察一下端口 ss -nal 看看是否有9083这个端口
直接输入:hive
警告,是因为在0.10 0.11或者之后的HIVE版本 hive.metastore.local 属性不再使用。
把这个属性的配置删除就可以了。
然后上hdfs的web界面,看看根目录下是否有hive
注意:高可用集群要在所有namenode上面都安装hive,所以在slave上面安装MySQL,并将master上面的hive远程拷贝到slave1上。
多用户状态可参考一下链接:
https://www.cnblogs.com/pingzizhuanshu/p/9022270.html