CDH6.2安装手册
一、系统信息:
1.系统配置及软件环境:
cpu:16c
mem:47g+swap2g
disk:/ 17g xfs /data 500g xfs
net:10000M/s
CentOS Linux release 7.4.1708 (Core)
CDH6.2.0
CDSW1.6.0
mysql5.7
jdk1.8.0_221
mysql-connector-java-8.0.17
2.服务分配:
10.12.9.91:cm-server,cm-agent,mysql,jdk
10.12.9.92:cm-agent,jdk
10.12.9.93:cm-agent,jdk
---------------------------------------( 二、准备软件 )--------------------------------------------------------------
一、准备工作:
1.下载cdh组件安装包:
https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/
cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-server-db-2-6.2.0-968826.el7.x86_64.rpm
2.下载cdh parcel包:
https://archive.cloudera.com/cdh6/6.2.0/parcels/
CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1
https://archive.cloudera.com/cdsw1/1.6.0/parcels/
CDSW-1.6.0.p1.1294376-el7.parcel
CDSW-1.6.0.p1.1294376-el7.parcel.sha
3.下载mysql5.7(可跳过):
mysql-5.7.26.tar.gz
4.下载mysql-connect-java.jar:
mysql-connector-java-8.0.17.tar.gz
5.下载CentOS7镜像:
CentOS-7-x86_64-DVD-1810.iso
6.下载boost:
http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz
7.所有机器创建目录(可跳过):
mkdir -p /data/repos
mkdir -p /data/cloudera/
chmod -R 777 /data/repos
8.将所有下载内容上传至一台服务器临时目录中:
10.12.9.91:/data/repos
9.将agent有关的包拷贝至对应机器上
cd /data/repos/
scp cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
mysql-connector-java-8.0.17.tar.gz [email protected]:/data/repos/
scp cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
mysql-connector-java-8.0.17.tar.gz [email protected]:/data/repos/
---------------------------------------( 三、开始进行配置 )--------------------------------------------------------------
一、底层配置(所有机器):
1.修改每台机器的主机名:
hostnamectl set-hostname node-91
hostnamectl set-hostname node-92
hostnamectl set-hostname node-93
2.将集群所有机器的host信息更新至集群当中每台的/etc/hosts中:
vim /etc/hosts
3.关闭所有机器的selinux:
修改/etc/selinux/config 配置文件,
SELINUX=disabled,保存后重启系统
4.关闭所有机器的防火墙
service firewalld stop
chkconfig firewalld off
service firewalld status
5.挂载系统镜像(可跳过):
mount -o loop /data/repos/CentOS-7-x86_64-DVD-1810.iso /mnt
6. 将之前的yum repo文件迁移(可跳过):
mkdir –p /etc/yum.repos.d/bak
mv /etc/yum.repos.d/* /etc/yum.repos.d/bak/
7.编写yum repo文件(可跳过, 使用其他方式):
cd /etc/yum.repos.d/
vim centos7.repo
[centos7-iso]
name=centos7-iso
baseurl=file:///mnt/
enabled=1
gpgcheck=0
8.安装底层基础包:
yum install httpd mod_ssl python-psycopg2 MySQL-python redhat-lsb openssl-devel -y
9.配置mysql-connect-java:
tar zxf mysql-connector-java-8.0.17.tar.gz
cp mysql-connector-java-8.0.17.jar /usr/share/java/mysql-connector-java.jar
10.修改内核参数:
vim /etc/sysctl.conf
vim /proc/sys/vm/swappiness
默认是30,cdh6安装建议最大是10
修改命令:
临时解决:
通过 echo 10 > /proc/sys/vm/swappiness 即可解决。
永久解决:
sysctl -w vm.swappiness=10
echo vm.swappiness = 10 >> /etc/sysctl.conf
echo net.core.somaxconn=32768 >> /etc/sysctl.conf
echo sysctl –p >> /etc/sysctl.conf
验证:
cat /proc/sys/vm/swappiness
cat /etc/sysctl.conf
11.调整最大文件打开数:
vim /etc/security/limits.conf
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535
12.关闭透明大页:
手动执行:
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
修改:vim /etc/rc.local
第一种方式(建议此方式):
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo 'echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled' >> /etc/rc.local
第二种方式:
#永久修改hugepage
sudo chmod +x /etc/rc.d/rc.local
vim /etc/rc.d/rc.local
将下述内容添加⾄至⽂文件结尾部分
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never >
/sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never >
/sys/kernel/mm/transparent_hugepage/defrag
fi
13.ssh无密码登录
ssh-keygen -t rsa
cp id_rsa.pub authorized_keys 或者 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
并设置authorized_keys的访问权限
chmod 600 ~/.ssh/authorized_keys
ssh-copy-id -i cdh02
ssh-copy-id -i cdh03
尝试登陆验证:ssh cdh02、ssh cdh03、ssh cdh01
14. 检查主机时间同步
第一种方式:CentOS 7默认以chronycd服务作时间同步,而不以ntpd作为同步服务(建议此方式)
1>.所有服务器都以master01作为同步服务器,在所有服务器更改文件:
a) 文件为: /etc/chrony.conf
b) 以#注释其它所有默认时间服务器
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
c) 增加行server master01 iburst
server cdh01 iburst
2>.在master01上同时更改如下配置/etc/chrony.conf 添加一行:
# Allow NTP client access from local network.
allow 192.168.200.0/24
打开如下一行:
local stratum 10
3>.启动服务:
systemctl restart chronyd
systemctl enable chronyd
systemctl status chronyd
4>.检查服务状态:
chronyc sources
第二种方式:以ntpd作为同步服务(不建议此方式)
(1).所有节点配置NTP服务,集群中所有主机必须保持时间同步,如果时间相差较⼤大会引起各种问题。
Readhat 7 默认安装chrony ,首先要卸载,然后安装ntp服务
rpm -qa | grep chrony
(2).三台机器都执行:yum -y remove chrony
所有节点安装相关组件:yum install ntp
保持一致: rpm -qa | grep ntpd
启动服务: systemctl start ntpd
配置开机启动: systemctl enable ntpd
(3).master主节点配置:
在配置之前,先使⽤用ntpdate手动同步一下时间,免得本机与对时中心时间差距太⼤大,使得ntpd不不能正常同步。
这里选用 cdh6-xingye002 作为对时中心
ntpdate -u ntp1.aliyun.com
(4).备份配置: cp /etc/ntp.conf /etc/ntp.conf_back
(5).创建目录:
mkdir -p /var/lib/ntp/
mkdir -p /var/log/ntp/
(6).启动:
systemctl stop ntpd
systemctl start ntpd
systemctl status ntpd
(7).httpd服务配置:
vim /etc/httpd/conf/httpd.conf
修改:119 DocumentRoot "/var/www/"
添加 129行
<Directory "/var/www/html">
AllowOverride None
Require all granted
Options Indexes FollowSymLinks
</Directory>
(8).启动httpd: service httpd start
开机自启httpd:systemctl enable httpd
查看服务:service httpd status
注:如果服务器器⽆无httpd服务,且无法联网,可先通过第4步骤配置本地源( baseurl=file:///opt/third/yum/ ),然后从本地源安装httpd。
15.java安装
tar -zxf jdk-8u221-linux-x64.tar.gz -C /usr/jdk64/
vim /etc/profile
第一种老方式配置:
export JAVA_HOME=/usr/jdk64/jdk1.8.0_221
#export CLASSPATH=./:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export CLASSPATH=./:$JAVA_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH
第二种新方式配置(CDH6.2.1版本建议此方式):
注:一定在各节点单独安装(因为同步解压包会失去软连接)
scp /opt/jdk-8u221-linux-x64.rpm root@cdh02:/opt
scp /opt/jdk-8u221-linux-x64.rpm root@cdh03:/opt
安装:
cd /opt
rpm -ivh jdk-8u221-linux-x64.rpm
安装后
vim /etc/profile
export JAVA_HOME=/usr/java/default
export CLASSPATH=./:$JAVA_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH
source /etc/profile
java -version
16.重启服务器
reboot
---------------------------------------( 四、开始部署 )--------------------------------------------------------------
一、server端部署:
---------------------------------------------mysql
1.mysql准备:
第一种安装mysql方式(可跳过, 使用其他方式进行安装 )
(1)安装底层依赖包:
yum install cmake make gcc gcc-c++ ncurses-* –y
(2)编译安装:
cd /data/repos
tar zxf mysql-5.7.26.tar.gz
cd mysql-5.7.26
cmake -DCMAKE_INSTALL_PREFIX=/data/mysql5.7 -DWITH_BOOST=/data/repos/boost_1_59_0.tar.gz
make && make install
(3)自定义配置:
rm –f /etc/my.cnf
vim /data/mysql5.7/my.cnf
[mysqld]
server_id = 1
skip-grant-tables=1
datadir=/data/mysql5.7/data
port=3306
socket=/data/mysql5.7/data/mysql.sock
user=mysql
character-set-server=utf8
back_log = 500
max_connections = 1000
max_connect_errors = 1000
open_files_limit = 10000
table_open_cache = 2000
table_definition_cache=2000
innodb_open_files=5000
log_error = /data/mysql5.7/logs/mysql-error.log
slow_query_log = 1
long_query_time = 0.05
slow_query_log_file = /data/mysql5.7/logs/mysql-slow.log
default_storage_engine = InnoDB
query_cache_size = 64M
innodb_buffer_pool_size=256M
innodb_flush_log_at_trx_commit=2
innodb_file_per_table=1
innodb_read_io_threads=4
innodb_write_io_threads=4
innodb_io_capacity=500
innodb_io_capacity_max=2000
innodb_lru_scan_depth=500
innodb_autoinc_lock_mode= 2
innodb_buffer_pool_instances = 4
innodb_use_native_aio=on
innodb_flush_neighbors=0
innodb_support_xa=off
innodb_log_file_size=128M
innodb_log_file_size=1G
innodb_log_files_in_group=2
innodb_doublewrite=0
log-bin=mysqld-bin
sync_binlog=60
expire_logs_days=3
key_buffer_size=32M
symbolic-links=0
[mysqld_safe]
log-error=/data/mysql5.7/logs/mysqld.log
pid-file=/data/mysql5.7/mysqld.pid
[mysql]
default-character-set=utf8
(4)目录权限配置:
useradd -d /home/mysql -s /sbin/nologin mysql
mkdir –p /data/mysql5.7/data
mkdir –p /data/mysql5.7/logs
touch /data/mysql5.7/logs/mysqld.log
chown –R mysql.mysql /data/mysql5.7
(5)初始化mysql库:
/data/mysql5.7/bin/mysqld --initialize --basedir=/export/mysql5.7/ --datadir=/data/mysql5.7/data --user=mysql
(6)系统底层配置:
cp /data/mysql5.7/support-files/mysql.server /etc/init.d/mysqld
chkconfig --add mysqld
chkconfig --level 35 mysqld on
vim /etc/profile
PATH=$PATH:/data/mysql5.7/bin
source /etc/profile
(7)启动mysql:
service mysqld start
(8)登录root用户修改密码:
mysql -u root -h 127.0.0.1
update mysql.user set authentication_string=password('123456') where user='root';
flush privileges;
exit;
(9)重启mysql:
将my.cnf中的skip-grant-tables删除,重启mysql
service mysqld stop
service mysqld start
(7):设置⾃自动启动
systemctl enable mysqld
systemctl daemon-reload
第二种安装mysql方式(建议使用此方式)
(1):去官⽹网查看最新安装包
https://dev.mysql.com/downloads/repo/yum/
(2):下载MySQL源安装包
wget http://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
安装MySql源
yum -y install mysql57-community-release-el7-11.noarch.rpm
查看一下安装效果
yum repolist enabled | grep mysql.*
(3):安装MySQL服务器
yum install mysql-community-server
中间会弹出是与否的选择,选择y即可,然后耐⼼心等待吧。。。。。。。
(4):启动MySQL服务
systemctl start mysqld.service
运行一下命令查看一下运行状态
systemctl status mysqld.service
(5):初始化数据库密码
查看一下初始密码: grep "password" /var/log/mysqld.log
登录 mysql -uroot -p
# 不修改密码无法执⾏行
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITHGRANT OPTION;
# 修改密码 mysql默认安装了了密码安全检查插件(validate_password),默认密码检查策略略要求密码必须包含:大小# 写字母、数字和特殊符号,并且长度不能少于8位。
否则会提示ERROR 1819(HY000): Your password does not # satisfy the current policy requirements错误
ALTER USER 'root'@'localhost' IDENTIFIED BY 'Root@1298';
# 远程连接数据库的时候需要输⼊入⽤用户名和密码
# 用户名:root
# 密码:123456
# 指点ip:%代表所有Ip,此处也可以输⼊入Ip来指定Ip
# 输入后使修改⽣生效还需要下面的语句
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'Root@1298' WITH GRANT OPTION;
# 刷新
FLUSH PRIVILEGES;
(6):数据库授权
数据库没有授权,只⽀支持localhost本地访问,也可以通过修改表来实现远程:
use mysql;
update user set host = '%' where user = 'root';
select host, user from user;
(7):设置⾃自动启动
systemctl enable mysqld
systemctl daemon-reload
--------------------------------------------
2.配置cm相关内容:
(1).配置权限
mysql -u root -h 127.0.0.1 –p
set global binlog_format='ROW';
create database cm;
grant all on cm.* to 'scm'@'%' identified by 'hadoop123';
grant all on cm.* to 'scm'@'localhost' identified by 'hadoop123';
flush privileges;
(2).待cm server包安装好后初始化cm库( 此步:等待 下main3-(1)安装完后操作此步 ):
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql cm scm hadoop123
3.cm准备:
(1)安装cm server包:
cd /data/repos
rpm -ivh cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
rpm –ivh cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm
(2)创建parcel目录(CDH默认 安装 Parcel 目录 是: /opt/cloudera/parcels ):
mkdir -p /opt/cloudera/parcel-repo
mkdir -p /opt/cloudera/parcels
chown -R cloudera-scm.cloudera-scm /opt/cloudera
如下是改用其他目录:
mkdir -p /data/cloudera/parcel-repo
mkdir -p /data/cloudera/parcels
chown -R cloudera-scm.cloudera-scm /data/repos
chown -R cloudera-scm.cloudera-scm /data/cloudera
(3)parcel包准备(CDH默认本地 Parcel 软件存储库源路径 是: /opt/cloudera/parcel-repo/ ):
mv CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel* /opt/cloudera/parcel-repo/
cd /opt/cloudera/parcel-repo/
mv CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1 CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha
(4)启动cm server:
systemctl start cloudera-scm-server
systemctl stop cloudera-scm-server
systemctl status cloudera-scm-server
观察 tail -200f /var/log/cloudera-scm-server/cloudera-scm-server.log 有无报错信息
(5)访问cm平台:
浏览器输入 http://10.12.9.91:7180 / http://cdh01:7180/cmf/login 验证server是否启动成功
二、agent端部署:
1.安装cm agent包:
cd /data/repos
rpm –ivh cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
rpm -ivh cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
安装rpm -ivh cloudera-manager-agent 遇到缺少包:
yum install openssl-devel -y
2.修改agent端配置:
vim /etc/cloudera-scm-agent/config.ini
server_host=10.12.9.91
parcel_dir=/data/cloudera/parcels
注意:agent节点都要创建目录
mkdir -p /data/cloudera/parcels
chown -R cloudera-scm.cloudera-scm /data/cloudera
3.启动cm agent:
systemctl start cloudera-scm-agent
systemctl stop cloudera-scm-agent
systemctl status cloudera-scm-agent
观察 tail -200f /var/log/cloudera-scm-agent/cloudera-scm-agent.log 当中有无报错
重启server端 (没有异常就跳过此步):
systemctl start cloudera-scm-server
systemctl status cloudera-scm-server
重启各节点 agent (没有异常就跳过此步)
systemctl restart cloudera-scm-agent
systemctl status cloudera-scm-agent
注意事项:
以上部署过程为纯手动部署,一些操作步骤可以用更灵活的方式执行,比如以自建yum源的方式替代挂载镜像的方式进行安装,一些文件可从server端拷贝至agent端,以避免重复上传或配置
三、自动安装阶段
如果页面检查不到对应版本:
重新启动:
systemctl restart cloudera-scm-server
systemctl restart cloudera-scm-agent
1、Inspect Cluster
You have created a new empty cluster. Cloudera recommends that you run the following inspections. For accurate measurements, Cloudera recommends that they are performed sequentially.
检查集群
您已经创建了一个新的空集群。Cloudera建议您运行以下检查。对于精确的测量,Cloudera建议按顺序进行。
2、Inspect Network Performance
Once the inspection is complete, review the inspector results before proceeding.
检查网络性能
一旦检查完成,在进行之前要回顾检查结果。
√ Fix the issues and run the inspection tools again.
修复问题并重新运行检查工具。
Quit the wizard and Cloudera Manager will delete the temporarily created cluster.
退出向导,Cloudera管理器将删除临时创建的集群。
I understand the risks, let me continue with cluster creation.
我了解风险,让我继续集群创建。
3、Inspect Hosts
Once the inspection is complete, review the inspector results before proceeding.
检查主机
一旦检查完成,在进行之前要回顾检查结果。
图1
图2
⚠️警告1 的处理:
Cloudera 建议将 /proc/sys/vm/swappiness 设置为最大值 10。当前设置为 30。使用 sysctl 命令在运行时更改该设置并编辑 /etc/sysctl.conf,以在重启后保存该设置。您可以继续进行安装,但 Cloudera Manager 可能会报告您的主机由于交换而运行状况不良。以下主机将受到影响:
vim /proc/sys/vm/swappiness
默认是30,cdh6安装建议最大是10
修改命令:
解决:
临时解决
通过echo 10 > /proc/sys/vm/swappiness 即可解决。
永久解决
sysctl -w vm.swappiness=10
echo vm.swappiness = 10 >> /etc/sysctl.conf
重新检查并查看结果:
⚠️警告2 的处理:
已启用透明大页面压缩,可能会导致重大性能问题。请运行“echo never > /sys/kernel/mm/transparent_hugepage/defrag”和“echo never > /sys/kernel/mm/transparent_hugepage/enabled”以禁用此设置,然后将同一命令添加到 /etc/rc.local 等初始化脚本中,以便在系统重启时予以设置。以下主机将受到影响:
处理:
大内存页禁用:手动执行
echo never>/sys/kernel/mm/transparent_hugepage/defrag
echo never>/sys/kernel/mm/transparent_hugepage/enabled
⚠️警告3:
该群集的主机上未安装 Java。
第二种新方式配置(CDH6.2.1版本建议此方式):
注:一定在各节点单独安装(因为同步解压包会失去软连接)
scp /opt/jdk-8u221-linux-x64.rpm root@cdh02:/opt
scp /opt/jdk-8u221-linux-x64.rpm root@cdh03:/opt
安装:
cd /opt
rpm -ivh jdk-8u221-linux-x64.rpm
vim /etc/profile
export JAVA_HOME=/usr/java/default
export CLASSPATH=./:$JAVA_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH
source /etc/profile
java -version
重新检查并查看结果:
检验通过!
Quit the wizard and Cloudera Manager will delete the temporarily created cluster.
I understand the risks, let me continue with cluster creation.
退出向导,Cloudera管理器将删除临时创建的集群。
我了解风险,让我继续集群创建。
在主节点 hadoop01节点上(mysql节点):
mysql -u root -p hadoop123
hive:
create database hivedb default character set utf8 default collate utf8_general_ci;
grant all on hivedb.* to 'hivedb'@'%' identified by 'hadoop123';
grant all on hivedb.* to 'hivedb'@'localhost' identified by 'hadoop123';
Activity Monitor:
create database rman default character set utf8 default collate utf8_general_ci;
grant all on rman.* to 'rman'@'%' identified by 'hadoop123';
grant all on rman.* to 'rman'@'localhost' identified by 'hadoop123';
Reports Manager:
create database amon default character set utf8 default collate utf8_general_ci;
grant all on amon.* to 'amon'@'%' identified by 'hadoop123';
grant all on amon.* to 'amon'@'localhost' identified by 'hadoop123';
Oozie Server:
create database oozie default character set utf8 default collate utf8_general_ci;
grant all on oozie.* to 'oozie'@'%' identified by 'hadoop123';
grant all on oozie.* to 'oozie'@'localhost' identified by 'hadoop123';
Hue:
create database hue default character set utf8 default collate utf8_general_ci;
grant all on hue.* to 'hue'@'%' identified by 'hadoop123';
grant all on hue.* to 'hue'@'localhost' identified by 'hadoop123';
show databases;
flush privileges;
use mysql
select user,host,password from mysql.user;
组件启动顺序
大功告成!
50070:
http://192.168.200.154:9870/dfshealth.html#tab-overviewHue:
http://192.168.200.154:8889/hue/editor/?type=hive
设置hive
set hive.cli.print.header=true;
useradd testuser 创建用户testuser
passwd testuser 给已创建的用户testuser设置密码
说明:新创建的用户会在/home下创建一个用户目录testuser
usermod --help 修改用户这个命令的相关参数
userdel testuser 删除用户testuser
rm -rf testuser 删除用户testuser所在目录
解决desc命令注释中文乱码:修改hive存储在mysql里的元数据相关信息
use hivedb;
1).修改字段注释字符集
alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
2).修改表注释字符集
alter table TABLE_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;
3).修改分区表参数,以支持分区键能够用中文表示
alter table PARTITION_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;
alter table PARTITION_KEYS modify column PKEY_COMMENT varchar(4000) character set utf8;
4).修改索引注解
alter table INDEX_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8