Kubernetes部署（八）：Flannel網絡部署

相關內容：

Kubernetes部署（一）：架構及功能說明
Kubernetes部署（二）：系統環境初始化
Kubernetes部署（三）：CA證書制作
Kubernetes部署（四）：ETCD集群部署
Kubernetes部署（五）：Haproxy、Keppalived部署
Kubernetes部署（六）：Master節點部署
Kubernetes部署（七）：Node節點部署
Kubernetes部署（八）：Flannel網絡部署
Kubernetes部署（九）：CoreDNS、Dashboard、Ingress部署
Kubernetes部署（十）：儲存之glusterfs和heketi部署
Kubernetes部署（十一）：管理之Helm和Rancher部署
Kubernetes部署（十二）：helm部署harbor企業級鏡像倉庫

Flannel 需要在所有的master和node都部署
1.為Flannel生成證書

[root@node-01 ssl]# vim flanneld-csr.json
{
  "CN": "flanneld",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成證書

[root@node-01 ssl]# cfssl gencert -ca=/data/kubernetes/ssl/ca.pem \
   -ca-key=/data/kubernetes/ssl/ca-key.pem \
   -config=/data/kubernetes/ssl/ca-config.json \
   -profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld

3.分發證書

[root@node-01 ssl]# for n in `seq 201 206`; do scp flanneld*.pem 10.31.90.$n:/data/kubernetes/ssl/;done

4.下載Flannel軟件包

[root@node-01 k8s]#wget https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz
[root@node-01 k8s]# tar zxf flannel-v0.10.0-linux-amd64.tar.gz
[root@node-01 k8s]# for n in `seq 201 206`;do scp flanneld mk-docker-opts.sh 10.31.90.$n:/data/kubernetes/bin/;done
復制對應腳本到/data/kubernetes/bin目錄下。
[root@node-01 k8s]# for n in `seq 201 206`;do scp remove-docker0.sh 10.31.90.$n:/data/kubernetes/bin/;done    

5.配置Flannel

[root@node-04 ssl]# vim /data/kubernetes/cfg/flannel
FLANNEL_ETCD="-etcd-endpoints=https://10.31.90.201:2379,https://10.31.90.202:2379,https://10.31.90.203:2379"
FLANNEL_ETCD_KEY="-etcd-prefix=/kubernetes/network"
FLANNEL_ETCD_CAFILE="--etcd-cafile=/data/kubernetes/ssl/ca.pem"
FLANNEL_ETCD_CERTFILE="--etcd-certfile=/data/kubernetes/ssl/flanneld.pem"
FLANNEL_ETCD_KEYFILE="--etcd-keyfile=/data/kubernetes/ssl/flanneld-key.pem"
復制配置到其它節點上
[root@node-01 ~]# for n in `seq 201 206`;do scp /data/kubernetes/cfg/flannel 10.31.90.$n:/data/kubernetes/cfg/;done

6.設置Flannel系統服務

[root@node-01 ~]# vim /usr/lib/systemd/system/flannel.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
Before=docker.service

[Service]
EnvironmentFile=-/data/kubernetes/cfg/flannel
ExecStartPre=/data/kubernetes/bin/remove-docker0.sh
ExecStart=/data/kubernetes/bin/flanneld ${FLANNEL_ETCD} ${FLANNEL_ETCD_KEY} ${FLANNEL_ETCD_CAFILE} ${FLANNEL_ETCD_CERTFILE} ${FLANNEL_ETCD_KEYFILE}
ExecStartPost=/data/kubernetes/bin/mk-docker-opts.sh -d /run/flannel/docker

Type=notify

[Install]
WantedBy=multi-user.target
RequiredBy=docker.service

復制系統服務腳本到其它節點上
[root@node-01 k8s]# for n in `seq 201 206`;do scp /usr/lib/systemd/system/flannel.service 10.31.90.$n:/usr/lib/systemd/system/flannel.service;done

Flannel CNI集成

下載CNI插件

[root@node-01 ~]# wget https://github.com/containernetworking/plugins/releases/download/v0.7.1/cni-plugins-amd64-v0.7.1.tgz
[root@node-01 ~]# mkdir /data/kubernetes/bin/cni
[root@node-01 src]# tar zxf cni-plugins-amd64-v0.7.1.tgz -C /data/kubernetes/bin/cni
[root@node-01 k8s]# for n in `seq 201 206`;do scp /data/kubernetes/bin/cni/* 10.31.90.$n:/data/kubernetes/bin/cni/;done     

創建Etcd的key

[root@node-01 ~]# /data/kubernetes/bin/etcdctl --ca-file /data/kubernetes/ssl/ca.pem --cert-file /data/kubernetes/ssl/flanneld.pem --key-file /data/kubernetes/ssl/flanneld-key.pem \
      --no-sync -C https://10.31.90.201:2379,https://10.31.90.202:2379,https://10.31.90.203:2379 \
mk /kubernetes/network/config '{ "Network": "10.2.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}' >/dev/null 2>&1

啟動flannel

[root@node-01 ~]# systemctl daemon-reload
[root@node-01 ~]# systemctl enable flannel
[root@node-01 ~]# chmod +x /data/kubernetes/bin/*
[root@node-01 ~]# systemctl start flannel

查看服務狀態

[root@node-01 ~]# systemctl status flannel

配置Docker使用Flannel

[root@node-01 ~]# vim /usr/lib/systemd/system/docker.service
[Unit] #在Unit下面修改After和增加Requires
After=network-online.target firewalld.service flannel.service
Wants=network-online.target
Requires=flannel.service

[Service] #增加EnvironmentFile=-/run/flannel/docker
Type=notify
EnvironmentFile=-/run/flannel/docker
ExecStart=/usr/bin/dockerd $DOCKER_OPTS

將配置復制到其它所有的node

[root@node-01 k8s]# for n in `seq 201 206`;do scp /usr/lib/systemd/system/docker.service 10.31.90.$n:/usr/lib/systemd/system/docker.service;done

重啟Docker

[root@node-01 ~]# systemctl daemon-reload
[root@node-01 ~]# systemctl restart docker

再查看各個節點會發現docker0網卡和flannel網卡的ip地址都是我們上面配置的網段了。

[root@node-01 k8s]# ifconfig 
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.2.84.1  netmask 255.255.255.0  broadcast 10.2.84.255
        ether 02:42:5e:c6:0c:aa  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.2.84.0  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::8ccc:15ff:fedd:c00d  prefixlen 64  scopeid 0x20<link>
        ether 8e:cc:15:dd:c0:0d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 8 overruns 0  carrier 0  collisions 0

至此k8s的集群就已經全部部署完成，后續會繼續補充管理、監控、存儲等方面的文檔。

后續會陸續更新所有的安裝文檔，如果你覺得我寫的不錯，希望大家多多關注點贊，非常感謝！