官方提供的几种Kubernetes部署方式
minikube Minikube是一个工具,可以在本地快速运行一个单点的Kubernetes,尝试Kubernetes或日常开发的用户使用。不能用于生产环境。 kubeadm Kubeadm也是一个工具,提供kubeadm init和kubeadm join指令,用于快速部署Kubernetes集群。 二进制包 从官方下载发行版的二进制包,手动部署每个组件,组成Kubernetes集群。 小结:生产环境中部署Kubernetes集群,只有Kubeadm和二进制包可选,Kubeadm降低部署门槛,但屏蔽了很多细节,遇到问题很难排查。我们这里使用二进制包部署Kubernetes集群,我也是推荐大家使用这种方式,虽然手动部署麻烦点,但学习很多工作原理,更有利于后期维护。
软件环境:
软件 | 版本 |
操作系统 | CentOS7.5_x64 |
Docker | 18-ce |
Kubernetes | 1.12 |
服务器角色:
角色 |
IP |
组件 |
k8s-master01 | 192.168.125.122 | kube-apiserver,kube-controller-manager,kube-scheduler,etcd |
k8s-node01 | 192.168.125.123 | kubelet,kube-proxy,docker,flannel,etcd |
k8s-node02 | 192.168.125.124 | kubelet,kube-proxy,docker,flannel,etcd |
基础环境配置:
k8s-master01:
hostnamectl set-hostname k8s-master01 echo "192.168.125.122 k8s-master01" >> /etc/hosts systemctl stop firewalld;systemctl disable firewalld setenforce 0;sed -s '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config yum install ntpdate wget unzip lrzsz vim -y;ntpdate -u cn.ntp.org.cn swapoff -a cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system logout
k8s-node01:
hostnamectl set-hostname k8s-node01 echo "192.168.125.123 k8s-node01" >> /etc/hosts systemctl stop firewalld;systemctl disable firewalld setenforce 0;sed -s '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config yum install ntpdate wget unzip lrzsz vim -y;ntpdate -u cn.ntp.org.cn swapoff -a cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system logout
k8s-node02:
hostnamectl set-hostname k8s-node02 echo "192.168.125.124 k8s-node02" >> /etc/hosts systemctl stop firewalld;systemctl disable firewalld setenforce 0;sed -s '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config yum install ntpdate wget unzip lrzsz vim -y;ntpdate -u cn.ntp.org.cn swapoff -a cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system logout
1. 部署Etcd集群
使用cfssl来生成自签证书,先下载cfssl工具: wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo 注:可以上传脚本cfssl.sh并执行。
1.1 生成证书创建
以下三个文件: cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "www": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF cat > ca-csr.json <<EOF { "CN": "etcd CA", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF #----------------------- cat > server-csr.json <<EOF { "CN": "etcd", "hosts": [ "192.168.125.122", "192.168.125.123", "192.168.125.124" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] } EOF 生成证书: cfssl gencert -initca ca-csr.json | cfssljson -bare ca - cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server # ls *pemca-key.pem ca.pem server-key.pem server.pem 证书这块知道怎么生成、怎么用即可,建议暂时不必过多研究。 脚本配置: 创建目录/root/k8s/etcd-cert,并将文件etcd-cert.sh 、etcd.sh上传到该目录下,将文件etcd-cert.sh中的hosts做相应的更改,然后执行bash bash /root/k8s/etcd-cert/etcd-cert.sh即可完成证书的申请。
1.2 部署Etcd
二进制包下载地址:https://github.com/coreos/etcd/releases/tag/v3.3.10 以下部署步骤在规划的三个etcd节点操作一样,唯一不同的是etcd配置文件中的服务器IP要写当前的: 解压二进制包:: # mkdir /opt/etcd/{bin,cfg,ssl} -p # tar zxvf etcd-v3.3.10-linux-amd64.tar.gz # mv etcd-v3.3.10-linux-amd64/{etcd,etcdctl} /opt/etcd/bin 创建etcd配置文件: # cat /opt/etcd/cfg/etcd #[Member] ETCD_NAME="etcd01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://192.168.125.122:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.125.122:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.125.122:2380" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.125.122:2379" ETCD_INITIAL_CLUSTER="etcd01=https://192.168.125.122:2380,etcd02=https://192.168.125.123:2380,etcd03=https://192.168.125.124:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" ETCD_NAME 节点名称ETCD_DATA_DIR 数据目录 ETCD_LISTEN_PEER_URLS 集群通信监听地址 ETCD_LISTEN_CLIENT_URLS 客户端访问监听地址 ETCD_INITIAL_ADVERTISE_PEER_URLS 集群通告地址 ETCD_ADVERTISE_CLIENT_URLS 客户端通告地址 ETCD_INITIAL_CLUSTER 集群节点地址 ETCD_INITIAL_CLUSTER_TOKEN 集群 TokenETCD_INITIAL_CLUSTER_STATE 加入集群的当前状态,new是新集群,existing表示加入已有集群 systemd管理etcd: # cat /usr/lib/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/opt/etcd/cfg/etcd ExecStart=/opt/etcd/bin/etcd --name=${ETCD_NAME} --data-dir=${ETCD_DATA_DIR} --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} --initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} --initial-cluster=${ETCD_INITIAL_CLUSTER} --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} --initial-cluster-state=new --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target 把刚才生成的证书拷贝到配置文件中的位置: # cp ca*pem server*pem /opt/etcd/ssl 启动并设置开启启动: # systemctl start etcd # systemctl enable etcd 其它节点配置同上 配置k8s-master到k8s-node节点的ssh认证 ssh免秘钥: ssh-keygen ssh-copy-id [email protected] ssh-copy-id [email protected] scp -r /opt/etcd/ [email protected]:/opt/ scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/ scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/ scp -r /opt/etcd/ [email protected]:/opt/ scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/ scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/ 修改k8s-node节点的/opt/etcd/cfg/etcd 配置文件,监听IP更改为本机。 ssh [email protected] "systemctl start etcd;systemctl enable etcd" ssh [email protected] "systemctl start etcd;systemctl enable etcd" 都部署完成后,检查etcd集群状态: # /opt/etcd/bin/etcdctl \ --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \ --endpoints="https://192.168.125.122:2379,https://192.168.125.123:2379,https://192.168.125.124:2379" \ cluster-health 执行结果: # /opt/etcd/bin/etcdctl \ > --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \ > --endpoints="https://192.168.125.122:2379,https://192.168.125.123:2379,https://192.168.125.124:2379" \ > cluster-health member 1fd18cc9663e3095 is healthy: got healthy result from https://192.168.125.123:2379 member 3a7ee6dbc6ae9ddb is healthy: got healthy result from https://192.168.125.124:2379 member 79714bdc6157cf6d is healthy: got healthy result from https://192.168.125.122:2379 cluster is healthy 或者:给脚本etcd.sh 执行权限,然后运行指令 ./etcd.sh etcd01 192.168.125.122 etcd02=https://192.168.125.123:2380,etcd03=https://192.168.125.124:2380 注:第一个启动的节点,会有报错忽略,先去启动其它节点的etcd服务即可。 如果输出上面信息,就说明集群部署成功。如果有问题第一步先看日志:/var/log/message 或 journalctl -u etcd