kubernetes集群安裝指南：docker-ce部署

docker是POD基本運行環境，kubernetes默認以dockerd作為runtime運行引擎，在安裝docker-ce前必須先升級內核到最新lts版，這是因為為了讓docker-ce支持數據存儲類型overlay2，這里將會以二進制文件方式部署在所有的worker節點上部署docker-ce。

1. 準備工作

#################### Variable parameter setting ######################
DOCKER_INSTALL_PATH=/data/apps/k8s/docker
SOFTWARE=/root/software
VERSION=18.09.6
PACKAGE=docker-${VERSION}.tgz
DOWNLOAD_URL=https://download.docker.com/linux/static/stable/x86_64/$PACKAGE
MIRRORS1=https://docker.mirrors.ustc.edu.cn
MIRRORS2=https://registry-mirrors.mo9.com
USER=docker

2. 安裝docker

準備docker-ce安裝環境

# 1.Uninstall the original docker installation package
sudo yum -y remove docker docker-client \
  docker-client-latest  docker-common docker-latest \
  docker-latest-logrotate docker-selinux docker-engine-selinux docker-engine 
### 2 Setting firewall rules for docker
/sbin/iptables -P FORWARD ACCEPT
sudo sed -i '/iptables -P FORWARD ACCEPT/d' /etc/rc.local
echo -e "/sbin/iptables -P FORWARD ACCEPT"  >> /etc/rc.local

### 3.Install docker-ce package with yum.
sudo yum install -y yum-utils device-mapper-persistent-data lvm2 bridge-utils 
sudo yum-config-manager \
  --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

創建docker安裝相關安裝目錄以及dockers用戶組

### 4.Install docker-ce package with source.
# Check if the install directory exists and Check if the docker group exists .
if [ ! -d $DOCKER_INSTALL_PATH/bin ]; then
     mkdir -p $DOCKER_INSTALL_PATH/bin
fi

egrep "^$USER" /etc/group-
if [ $? -ne 0 ]; then
     groupadd $USER
fi

### 5.Download source package of docker-ce
if [ ! -f "$SOFTWARE/docker-${VERSION}.tgz" ]; then
     wget $DOWNLOAD_URL -P $SOFTWARE >>/dev/null 2>&1
fi
cd $SOFTWARE && tar -zxf $SOFTWARE/docker-${VERSION}.tgz -C ./
sudo cp -fp $SOFTWARE/docker/* $DOCKER_INSTALL_PATH/bin
cd $DOCKER_INSTALL_PATH/bin/
ln -sf docker,dockerd,containerd,containerd-shim,runc,ctr /usr/local/bin

### 6.Create daemon.json file for docker
# Create daemon.json file  path
if [ ! -d "/etc/docker" ]; then
     mkdir /etc/docker/
fi

• docker默認以root用戶運行，為了安全起見，方便其他用戶調用docker api，需要創建一個docker組用戶

創建docker配置文件

cat >/etc/docker/daemon.json <<EOF
{
    "authorization-plugins": [],
    "dns": ["223.5.5.5","223.4.4.4"],
    "dns-opts": [],
    "dns-search": [],
    "exec-opts": [],
    "data-root": "$DOCKER_INSTALL_PATH/data",
        "exec-root": "$DOCKER_INSTALL_PATH/exec",
    "experimental": false,
    "storage-driver": "overlay2",
        "storage-opts": ["overlay2.override_kernel_check=true"  ],
    "labels": [],
    "live-restore": true,
    "log-driver": "syslog",
    "log-opts": {},
    "pidfile": "/var/run/docker/docker.pid",
    "cluster-store": "",
    "cluster-store-opts": {},
    "cluster-advertise": "",
    "max-concurrent-downloads": 20,
    "max-concurrent-uploads": 5,
    "shutdown-timeout": 15,
    "debug": true,
    "default-ulimit": ["65535:65535"],
    "hosts": ["tcp://127.0.0.1:2376","unix:///var/run/docker.sock"],
    "log-level": "INFO",
    "swarm-default-advertise-addr": "",
    "api-cors-header": "",
    "selinux-enabled": false,
    "userns-remap": "",
    "group": "docker",
    "cgroup-parent": "",
    "init": false,
    "init-path": "/usr/libexec/docker-init",
    "ipv6": false,
    "iptables": true,
    "ip-forward": false,
    "userland-proxy": false,
    "userland-proxy-path": "/usr/libexec/docker-proxy",
    "ip": "0.0.0.0",
    "bridge": "",
    "fixed-cidr": "",
    "default-gateway": "",
    "icc": false,
    "raw-logs": false,
    "registry-mirrors": ["$MIRRORS1", "$MIRRORS2"],
    "seccomp-profile": "",
    "insecure-registries": [],
    "runtimes": {
        "cc-runtime": {
            "path": "/usr/bin/cc-runtime"
        },
        "custom": {
            "path": "/usr/local/bin/my-runc-replacement",
            "runtimeArgs": [
                "--debug"
            ]
        }
    }
}
EOF

• 這里使用overlay2作為docker數據存儲引擎
• 關于docker demon.json參數請參考docker官方文檔

創建docker服務啟動文件

cat >/usr/lib/systemd/system/docker.service <<"EOF"
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
EnvironmentFile=-/run/flannel/docker
ExecStart=/usr/local/bin/dockerd  $DOCKER_NETWORK_OPTIONS
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
EOF

• 因為使用flannel作為容器的網絡方案，所以需要指定flannel的參數配置文件
• flannel配置文件的參數不能和docker demon.json里的配置重復，否則docker服務啟動失敗；

啟動docker服務：

service dokcker start

如服務啟動失敗，使用journal查看日志

sudo journal -u docker

• 在啟動的時候，需要先啟動flannel網絡服務，然后再啟動容器服務；

docker-ce部署完成后，接下來安裝kubelet，請參考：kubernetes集群安裝指南：kubelet組件部署