kubernetes二進制集群部署 三 ——負載均衡調度器部

內容要點：

1、實驗環境

2、負載均衡調度器部署

一、實驗環境：

基于之前部署好的多 Master 集群架構的基礎上，部署兩臺調度器服務器（這邊我用的是 nginx），實現負載均衡：

kubernetes二進制集群部署一——etcd存儲組件、flannel網絡組件部署：

https://blog.51cto.com/14475876/2470049

kubernetes二進制集群部署一——單master集群部署+多master群及部署：

https://blog.51cto.com/14475876/2470063

服務器信息

需要兩個的腳本：

第一個：keepalived.conf
!?Configuration?File?for?keepalived?
?
global_defs?{?
???#?接收郵件地址?
???notification_email?{?
?????acassen@firewall.loc?
?????failover@firewall.loc?
?????sysadmin@firewall.loc?
???}?
???#?郵件發送地址?
???notification_email_from?Alexandre.Cassen@firewall.loc??
???smtp_server?127.0.0.1?
???smtp_connect_timeout?30?
???router_id?NGINX_MASTER?
}?
vrrp_script?check_nginx?{
????script?"/usr/local/nginx/sbin/check_nginx.sh"
}
vrrp_instance?VI_1?{?
????state?MASTER?
????interface?eth0
????virtual_router_id?51?#?VRRP?路由?ID實例，每個實例是唯一的?
????priority?100????#?優先級，備服務器設置?90?
????advert_int?1????#?指定VRRP?心跳包通告間隔時間，默認1秒?
????authentication?{?
????????auth_type?PASS??????
????????auth_pass?1111?
????}??
????virtual_ipaddress?{?
????????10.0.0.188/24?
????}?
????track_script?{
????????check_nginx
????}?
}
mkdir?/usr/local/nginx/sbin/?-p
vim?/usr/local/nginx/sbin/check_nginx.sh
count=$(ps?-ef?|grep?nginx?|egrep?-cv?"grep|$$")
if?[?"$count"?-eq?0?];then
????/etc/init.d/keepalived?stop
fi
chmod?+x?/usr/local/nginx/sbin/check_nginx.sh
第二個：nginx
cat?>?/etc/yum.repos.d/nginx.repo?<<?EOF
[nginx]
name=nginx?repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
EOF
stream?{
???log_format??main??'$remote_addr?$upstream_addr?-?[$time_local]?$status?$upstream_bytes_sent';
????access_log??/var/log/nginx/k8s-access.log??main;
????upstream?k8s-apiserver?{
????????server?10.0.0.3:6443;
????????server?10.0.0.8:6443;
????}
????server?{
????????????????listen?6443;
????????????????proxy_pass?k8s-apiserver;
????}
????}

二、負載均衡調度器部署

//首先關閉防火墻：
[root@localhost?~]#?systemctl?stop?firewalld.service?
[root@localhost?~]#?setenforce?0

//將這個腳本文件放進家目錄中：
[root@localhost?~]#?ls
anaconda-ks.cfg??initial-setup-ks.cfg??keepalived.conf??nginx.sh??公共??模板??視頻??圖片??文檔??下載??音樂??桌面

//建立本地yum倉庫：
[root@localhost?~]#?vim?/etc/yum.repos.d/nginx.repo
[nginx]
name=nginx?repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
[root@localhost?~]#?yum?list
[root@localhost?~]#?yum?install?nginx?-y???//下載nginx
//接下來是添加四層轉發：
[root@localhost?~]#?vim?/etc/nginx/nginx.conf
添加以下模塊：
stream?{
???log_format??main??'$remote_addr?$upstream_addr?-?[$time_local]?$status?$upstream_bytes_sent';
????access_log??/var/log/nginx/k8s-access.log??main;
????upstream?k8s-apiserver?{
????????server?192.168.109.138:6443;????//master01的IP地址
????????server?192.168.109.230:6443;????//master02的IP地址
????}
????server?{
????????????????listen?6443;
????????????????proxy_pass?k8s-apiserver;
????}
????}
[root@localhost?~]#?systemctl?start?nginx?????//開啟服務
//接下來是部署?keepalived服務：
[root@localhost?~]#?yum?install?keepalived?-y
//修改配置文件（nginx01是master）：
[root@localhost?~]#?cp?keepalived.conf?/etc/keepalived/keepalived.conf?
cp：是否覆蓋"/etc/keepalived/keepalived.conf"？?yes
[root@localhost?~]#?vim?/etc/keepalived/keepalived.conf
//做如下刪改：
!?Configuration?File?for?keepalived
global_defs?{
???#?接收郵件地址
???notification_email?{
?????acassen@firewall.loc
?????failover@firewall.loc
?????sysadmin@firewall.loc
???}
???#?郵件發送地址
???notification_email_from?Alexandre.Cassen@firewall.loc
???smtp_server?127.0.0.1
???smtp_connect_timeout?30
???router_id?NGINX_MASTER
}
vrrp_script?check_nginx?{
????script?"/etc/nginx/check_nginx.sh"???##檢測腳本的路徑，稍后會創建
}
vrrp_instance?VI_1?{
????state?MASTER
????interface?ens33
????virtual_router_id?51
????priority?100?????##優先級
????advert_int?1
????authentication?{
????????auth_type?PASS
????????auth_pass?1111
????}
????virtual_ipaddress?{
????????192.168.109.100/24??????##虛擬IP地址
????}
????track_script?{
????????check_nginx
????}
}
//nginx02（是backup），配置如下：
!?Configuration?File?for?keepalived
global_defs?{
???#?接收郵件地址
???notification_email?{
?????acassen@firewall.loc
?????failover@firewall.loc
?????sysadmin@firewall.loc
???}
???#?郵件發送地址
???notification_email_from?Alexandre.Cassen@firewall.loc
???smtp_server?127.0.0.1
???smtp_connect_timeout?30
???router_id?NGINX_MASTER
}
vrrp_script?check_nginx?{
????script?"/etc/nginx/check_nginx.sh"???##檢測腳本的路徑，稍后會創建
}
vrrp_instance?VI_1?{
????state?BACKUP
????interface?ens33
????virtual_router_id?51
????priority?90?????##優先級低于master
????advert_int?1
????authentication?{
????????auth_type?PASS
????????auth_pass?1111
????}
????virtual_ipaddress?{
????????192.168.109.100/24??????##虛擬IP地址
????}
????track_script?{
????????check_nginx
????}
}
//創建檢測腳本
[root@localhost?~]#?vim?/etc/nginx/check_nginx.sh
count=$(ps?-ef?|grep?nginx?|egrep?-cv?"grep|$$")
if?[?"$count"?-eq?0?];then
????systemctl?stop?keepalived
fi
[root@localhost?~]#?chmod?+x?/etc/nginx/check_nginx.sh???//授權
[root@localhost?~]#?systemctl?start?keepalived.service???//開啟服務
[root@localhost?~]#?ip?a???//查看ip地址

2、實驗結果驗證

驗證一：漂移地址是否起作用（高可用是否實現）

1、此時 虛擬ip在 nginx01 上，驗證地址漂移，可以在 lb01 中使用 pkill nginx 停止nginx服務，再在 lb02 上使用 ip a 命令查看地址是否進行了漂移。

2、恢復，此時，在 nginx02上，我們先啟動 nginx服務，再啟動 keepalived服務，再用 ip a命令查看，地址又漂移回來了，而 nginx02上沒有虛擬ip。

驗證二：驗證負載均衡是否實現<此時VIP在bl2上>

1、修改nginx01（master）的首頁內容：

[root@localhost?~]#?vim?/usr/share/nginx/html/index.html
<h2>Welcome?to?master?nginx!</h2>

2、修改nginx02（backup）的首頁內容：

[root@localhost?~]#?vim?/usr/share/nginx/html/index.html
<h2>Welcome?to?backup?nginx!</h2>

3、用瀏覽器訪問：http://192.168.109.100/

此時，負載均衡和高可用功能都已經完全實現了！！！

3、部署 node節點：

//開始修改?node節點配置文件統一的?VIP（bootstrap.kubeconfig,kubelet.kubeconfig）
修改內容：server:?https://192.168.109.100:6443（都改成vip）
[root@localhost?cfg]#?vim?/opt/kubernetes/cfg/bootstrap.kubeconfig
[root@localhost?cfg]#?vim?/opt/kubernetes/cfg/kubelet.kubeconfig
[root@localhost?cfg]#?vim?/opt/kubernetes/cfg/kube-proxy.kubeconfig
//重啟服務：
[root@localhost?cfg]#?systemctl?restart?kubelet.service?
[root@localhost?cfg]#?systemctl?restart?kube-proxy.service
//檢查修改內容：
[root@localhost?cfg]#??grep?100?*
bootstrap.kubeconfig:????server:?https://192.168.109.100:6443
kubelet.kubeconfig:????server:?https://192.168.109.100:6443
kube-proxy.kubeconfig:????server:?https://192.109.220.100:6443
//接下來在?調度器1?上查看?nginx的k8s日志：
[root@localhost?~]#?tail?/var/log/nginx/k8s-access.log
192.168.109.131?192.168.109.138:6443?-?[09/Feb/2020:13:14:45?+0800]?200?1122
192.168.109.131?192.168.109.230:6443?-?[09/Feb/2020:13:14:45?+0800]?200?1121
192.168.109.132?192.168.109.138:6443?-?[09/Feb/2020:13:18:14?+0800]?200?1120
192.168.109.132?192.168.109.230:6443?-?[09/Feb/2020:13:18:14?+0800]?200?1121
可以看出是以輪詢調度的算法，將請求流量分發給兩臺master
————?接下來是測試創建?Pod：
在?master01?上操作：
[root@localhost?kubeconfig]#?kubectl?run?nginx?--image=nginx
//查看狀態：
[root@localhost?kubeconfig]#??kubectl?get?pods
NAME????????????????????READY???STATUS????RESTARTS???AGE
nginx-dbddb74b8-zbhhr???1/1?????Running???0??????????47s
此時已經創建完成，正在運行中
***?注意日志問題?***：
[root@localhost?kubeconfig]#?kubectl?logs?nginx-dbddb74b8-zbhhr
Error?from?server?(Forbidden):?Forbidden?(user=system:anonymous,?verb=get,?resource=nodes,?subresource=proxy)?(?pods/log?nginx-dbddb74b8-zbhhr)
此時，由于權限問題查看日志，會出現報錯
解決辦法（提升權限）：
[root@localhost?kubeconfig]#?kubectl?create?clusterrolebinding?cluster-system-anonymous?--clusterrole=cluster-admin?--user=system:anonymous
clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous?created
此時，再次查看日志，就不會出現報錯：
//查看?Pod網絡：
[root@localhost?kubeconfig]#?kubectl?get?pods?-o?wide
NAME????????????????????READY???STATUS????RESTARTS???AGE?????IP????????????NODE??????????????NOMINATED?NODE
nginx-dbddb74b8-zbhhr???1/1?????Running???0??????????7m11s???172.17.93.2???192.168.109.131???<none>
可以看出，這個在master01上創建的pod被分配到了node01上了。
我們可以在對應網絡的?node節點上操作就可以直接訪問：
在node01上操作：
[root@localhost?cfg]#?curl?172.17.93.2

此時，由于 flannel網絡組件的作用下，都可以在node01和node02的瀏覽器上訪問這個地址：172.17.93.2

由于剛剛訪問了網頁，我們也可以在 master01上查看到日志信息：