网络策略与calico
1.网络策略
-
前置条件
网络策略通过网络插件 来实现。要使用网络策略,你必须使用支持 NetworkPolicy 的网络解决方案。 创建一个 NetworkPolicy 资源对象而没有控制器来使它生效的话,是没有任何作用的。(Flannel不支持 NetworkPolicy,所以使用Flannei网络插件是不会隔离pod的) -
隔离和非隔离的 Pod
默认情况下,Pod 是非隔离的,它们接受任何来源的流量。
Pod 在被某 NetworkPolicy 选中时进入被隔离状态。 一旦名字空间中有 NetworkPolicy 选择了特定的 Pod,该 Pod 会拒绝该 NetworkPolicy 所不允许的连接。 (名字空间下其他未被 NetworkPolicy 所选择的 Pod 会继续接受所有的流量)
网络策略不会冲突,它们是累积的。 如果任何一个或多个策略选择了一个 Pod, 则该 Pod 受限于这些策略的 入站(Ingress)/出站(Egress)规则的并集。因此评估的顺序并不会影响策略的结果。
为了允许两个 Pods 之间的网络数据流,源端 Pod 上的出站(Egress)规则和 目标端 Pod 上的入站(Ingress)规则都需要允许该流量。 如果源端的出站(Egress)规则或目标端的入站(Ingress)规则拒绝该流量, 则流量将被拒绝。
2. 环境的设置
2.1 移除原来的flannel网络插件,保证实验的准确性(不移除也可以)
[root@server2 ~]# kubectl delete -f kube-flannel.yml ##删除fannel网络插件配置
[root@server2 ~]# kubectl -n kube-system get pod ##查看是否删除
[root@server2 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
demo 1/1 Running 10 3d1h
deployment-6456d7c676-2s77n 1/1 Running 0 67m
deployment-6456d7c676-wx9tk 1/1 Running 0 67m
[root@server2 ~]# kubectl delete deployments.apps deployment
[root@server2 ~]# kubectl get svc
[root@server2 ~]# kubectl delete svc nginx-svc
[root@server2 ~]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/demo 1/1 Running 10 3d1h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d21h
##移除fannel生成的配置文件
[root@server2 ~]# cd /etc/cni/
[root@server2 cni]# cd net.d/
[root@server2 net.d]# ls
10-flannel.conflist
[root@server2 net.d]# mv 10-flannel.conflist /mnt/ ##每一个节点都需要移除
[root@server3 net.d]# mv 10-flannel.conflist /mnt/
[root@server4 net.d]# mv 10-flannel.conflist /mnt/
[root@server2 net.d]# ps ax | grep gannel
2.2 提前拉取所需镜像
##1. 拉取
[root@server1 harbor]# docker pull calico/kube-controllers:v3.16.1 ##拉取的镜像可以通过官网下载的calico.yaml文件来查看
[root@server1 harbor]# docker pull calico/cni:v3.16.1
[root@server1 harbor]# docker pull calico/pod2daemon-flexvol:v3.16.1
[root@server1 harbor]# docker pull calico/node:v3.16.1
## 2. 上传
[root@server1 harbor]# docker tag calico/node:v3.16.1 reg.westos.org/calico/node:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/node:v3.16.1
[root@server1 harbor]# docker tag calico/pod2daemon-flexvol:v3.16.1 reg.westos.org/calico/pod2daemon-flexvol:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/pod2daemon-flexvol:v3.16.1
[root@server1 harbor]# docker tag calico/cni:v3.16.1 reg.westos.org/calico/cni:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/cni:v3.16.1
[root@server1 harbor]# docker tag calico/kube-controllers:v3.16.1 reg.westos.org/calico/kube-controllers:v3.16.1
[root@server1 harbor]# docker push reg.westos.org/calico/kube-controllers:v3.16.1
3. 配置Calico网络插件
3.1 配置calico
- calico简介:
flannel实现的是网络通信,calico的特性是在pod之间的隔离(网络策略)。
通过BGP路由,但大规模端点的拓扑计算和收敛往往需要一定的时间和计算资源。
纯三层的转发,中间没有任何的NAT和overlay,转发效率最好。
Calico 仅依赖三层路由可达。Calico 较少的依赖性使它能适配所有 VM、Container、白盒或者混合环境场景。
- 安装calico:
# wget https://docs.projectcalico.org/manifests/calico.yaml ##通过官网可以查看(最新版本下载比较慢,所以我使用的是)
# vim calico.yml
- name: CALICO_IPV4POOL_IPIP ##IPIP表示隧道
value: "off"
# kubectl apply -f calico.yaml ##创建
[root@server2 ~]# mkdir calico
[root@server2 ~]# cd calico/
[root@server2 calico]# ls
calico.yaml deny-nginx.yaml
[root@server2 calico]# kubectl apply -f calico.yaml ##应用calico网络插件
[root@server2 calico]# kubectl -n kube-system get pod ##查看calico节点是否运行
[root@server2 calico]# cd /etc/cni/net.d/ ##查看calico插件配置,每个结点都有
[root@server2 net.d]# ls
10-calico.conflist calico-kubeconfig
3.2 测试
## 1.先生成两个标签为nginx的pod
[root@server2 calico]# cd ../ingress/
[root@server2 ingress]# ls
auth demo.yml deploy.yaml nginx-svc.yml nginx.yml tls.crt tls.key
[root@server2 ingress]# kubectl apply -f nginx-svc.yml
[root@server2 ingress]# kubectl get pod -L app
NAME READY STATUS RESTARTS AGE APP
demo 1/1 Running 10 3d1h
deployment-6456d7c676-plxjz 1/1 Running 0 17s nginx
deployment-6456d7c676-rfr85 1/1 Running 0 17s nginx
[root@server2 ingress]# kubectl get svc
[root@server2 ingress]# curl 10.111.89.37
## 2. 运行测试文件,查看是否隔离标签为nginx的pod(网络策略,隔离所有ngix标签的pod)
[root@server2 ~]# cd calico/
[root@server2 calico]# vim deny-nginx.yaml
[root@server2 calico]# cat deny-nginx.yaml # NetworkPolicy模式
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-nginx
spec:
podSelector:
matchLabels:
app: nginx
[root@server2 calico]# kubectl apply -f deny-nginx.yaml ##运行
networkpolicy.networking.k8s.io/deny-nginx created
[root@server2 calico]# kubectl get networkpolicies.networking.k8s.io ##查看限制
NAME POD-SELECTOR AGE
deny-nginx app=nginx 20s
[root@server2 calico]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d21h
nginx-svc ClusterIP 10.111.89.37 <none> 80/TCP 5m1s
[root@server2 calico]# curl 10.111.89.37
## 1.先生成两个标签为nginx的pod
2. 运行测试文件,查看是否隔离标签为nginx的pod
4. 网络策略示例
[root@server2 calico]# kubectl -n kube-system describe pod calico-node-6k72w ##出错可以查看详细信息
[root@server2 ~]# kubectl delete -f kube-flannel.yml ##或者出错查看日志
4.1 限制访问指定服务
- 3.2 中的测试就是这个示例
4.2 允许指定pod访问服务
[root@server2 calico]# kubectl run nginx --image=myapp:v2 -it
[root@server2 calico]# kubectl get pod
NAME READY STATUS RESTARTS AGE
demo 1/1 Running 10 3d3h
deployment-6456d7c676-plxjz 1/1 Running 0 90m
deployment-6456d7c676-rfr85 1/1 Running 0 90m
nginx 1/1 Running 1 28s
[root@server2 calico]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo 1/1 Running 10 3d3h 10.244.1.28 server3 <none> <none>
deployment-6456d7c676-plxjz 1/1 Running 0 90m 10.244.141.192 server3 <none> <none>
deployment-6456d7c676-rfr85 1/1 Running 0 90m 10.244.22.0 server4 <none> <none> ##标签为nginx的ip
nginx 1/1 Running 1 35s 10.244.141.193 server3 <none> <none>
[root@server2 calico]# vim access-demo.yaml
[root@server2 calico]# cat access-demo.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: access-nginx
spec:
podSelector:
matchLabels:
app: nginx
ingress:
- from:
- podSelector:
matchLabels:
app: demo
[root@server2 calico]# kubectl apply -f access-demo.yaml
networkpolicy.networking.k8s.io/access-nginx created
[root@server2 calico]# kubectl get networkpolicies.
NAME POD-SELECTOR AGE
access-nginx app=nginx 20s
deny-nginx app=nginx 87m
[root@server2 calico]# kubectl label pod demo app=demo ##设置标签
pod/demo labeled
[root@server2 calico]# kubectl get pod -L app
NAME READY STATUS RESTARTS AGE APP
demo 1/1 Running 11 3d3h demo
deployment-6456d7c676-plxjz 1/1 Running 0 93m nginx
deployment-6456d7c676-rfr85 1/1 Running 0 93m nginx
nginx 1/1 Running 1 3m37s
[root@server2 calico]# kubectl attach demo -it
Defaulting container name to demo.
Use 'kubectl describe pod/demo -n default' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # ping 10.244.22.0
PING 10.244.22.0 (10.244.22.0): 56 data bytes
64 bytes from 10.244.22.0: seq=0 ttl=62 time=0.812 ms
^C
--- 10.244.22.0 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.812/0.812/0.812 ms
/ # curl 10.244.22.0
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
4.3 禁止 namespace 中所有 Pod 之间的相互访问
[root@server2 calico]# kubectl create namespace demo ##创建新的namespace实验
[root@server2 calico]# kubectl get ns
[root@server2 calico]# kubectl run demo1 --image=busyboxplus -it -n demo ##创建demo1
[root@server2 calico]# kubectl run demo2 --image=busyboxplus -it -n demo ##创建demo2
[root@server2 calico]# kubectl -n demo get pod
[root@server2 calico]# kubectl -n demo get pod --show-labels
[root@server2 calico]# vim deny-pod.yaml
[root@server2 calico]# cat deny-pod.yaml
[root@server2 calico]# cat deny-pod.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: demo
spec:
podSelector: {
}
[root@server2 calico]# kubectl apply -f deny-pod.yaml
[root@server2 calico]# kubectl get networkpolicies. -n demo
## 测试
[root@server2 calico]# kubectl get pod -o wide -n demo ##查看ip
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo1 0/1 Error 1 9m36s 10.244.22.2 server4 <none> <none>
demo2 1/1 Running 1 8m32s 10.244.141.194 server3 <none> <none>
[root@server2 calico]# kubectl attach demo1 -it -n demo ##测试demo1和demo2之间是否可以相互访问
Defaulting container name to demo1.
Use 'kubectl describe pod/demo1 -n demo' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # curl 10.244.141.194
4.4 禁止其他 namespace 访问服务
[root@server2 calico]# vim deny-ns.yaml
[root@server2 calico]# cat deny-ns.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: deny-namespace
spec:
podSelector:
matchLabels:
ingress:
- from:
- podSelector: {
}
[root@server2 calico]# kubectl apply -f deny-ns.yaml
[root@server2 calico]# kubectl attach demo2 -it -n demo
Defaulting container name to demo2.
Use 'kubectl describe pod/demo2 -n demo' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # curl 10.244.22.0
4.5 只允许指定namespace访问服务
[root@server2 calico]# kubectl create namespace test #创建新的ns
[root@server2 calico]# kubectl run demo3 --image=busyboxplus -it -n test ##创建测试pod
[root@server2 calico]# vim access-ns.yaml
[root@server2 calico]# cat access-ns.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: access-namespace
spec:
podSelector:
matchLabels:
run: nginx ##指定标签
ingress:
- from:
- namespaceSelector:
matchLabels:
role: prod ##指定ns的角色名字
[root@server2 calico]# kubectl label ns test role=prod ##设置namespace标签
[root@server2 calico]# kubectl get ns --show-labels
[root@server2 calico]# kubectl apply -f access-ns.yaml ##访问指定标签的pod
[root@server2 calico]# kubectl attach demo3 -it -n test
/ # curl 10.244.141.193 ##标签为run=nginx的pod节点,且访问成功
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
4.6 允许外网访问服务
[root@server2 ingress]# pwd
/root/ingress
[root@server2 ingress]# cat demo.yml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
#tls:
# - hosts:
# - www1.westos.org
# secretName: tls-secret
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: nginx-svc
servicePort: 80
[root@server2 ingress]# kubectl apply -f demo.yml
[root@server2 ingress]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-demo <none> www1.westos.org 172.25.13.3,172.25.13.4 80 4h6m
[root@server2 calico]# vim access-ex.yaml
[root@server2 calico]# cat access-ex.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: web-allow-external
spec:
podSelector:
matchLabels:
app: nginx
ingress:
- ports:
- port: 80
from: []