Redis主從集群搭建及容災部署(哨兵sentinel)

Redis容災部署（哨兵Sentinel）

    1、sentinel介紹

   Redis Sentinel 

   Sentinel(哨兵)是用于監控redis集群中Master狀態的工具，其已經被集成在redis2.4+的版本中

    2、Sentinel作用： 

1)：Master狀態檢測 

2)：如果Master異常，則會進行Master-Slave切換，將其中一個Slave作為Master，將之前的Master作為Slave 

3)：Master-Slave切換后，master_redis.conf、slave_redis.conf和sentinel.conf的內容都會發生改變，即master_redis.conf中會多一行slaveof的配置，sentinel.conf的監控目標會隨之調換 

3、Sentinel工作方式： 

1)：每個Sentinel以每秒鐘一次的頻率向它所知的Master，Slave以及其他 Sentinel 實例發送一個 PING 命令 

2)：如果一個實例（instance）距離最后一次有效回復 PING 命令的時間超過 down-after-milliseconds 選項所指定的值， 則這個實例會被 Sentinel 標記為主觀下線。 

3)：如果一個Master被標記為主觀下線，則正在監視這個Master的所有 Sentinel 要以每秒一次的頻率確認Master的確進入了主觀下線狀態。 

4)：當有足夠數量的 Sentinel（大于等于配置文件指定的值）在指定的時間范圍內確認Master的確進入了主觀下線狀態， 則Master會被標記為客觀下線 

5)：在一般情況下， 每個 Sentinel 會以每 10 秒一次的頻率向它已知的所有Master，Slave發送 INFO 命令 

6)：當Master被 Sentinel 標記為客觀下線時，Sentinel 向下線的 Master 的所有 Slave 發送 INFO 命令的頻率會從 10 秒一次改為每秒一次 

7)：若沒有足夠數量的 Sentinel 同意 Master 已經下線， Master 的客觀下線狀態就會被移除。 

若 Master 重新向 Sentinel 的 PING 命令返回有效回復， Master 的主觀下線狀態就會被移除。

4、主觀下線和客觀下線 

主觀下線：Subjectively Down，簡稱 SDOWN，指的是當前 Sentinel 實例對某個redis服務器做出的下線判斷。 

客觀下線：Objectively Down， 簡稱 ODOWN，指的是多個 Sentinel 實例在對Master Server做出 SDOWN 判斷，并且通過 SENTINEL is-master-down-by-addr 命令互相交流之后，得出的Master Server下線判斷，然后開啟failover.

通俗來講就是： 

redis的sentinel系統用來管理多個redis服務器，可以實現一個功能上實現HA的集群。該系統主要執行三個任務： 

1、監控（ Monitoring ）： Redis Sentinel實時監控主服務器和從服務器運行狀態。 

2、提醒（notification）： 當被監控的某個 Redis 服務器出現問題時， Redis Sentinel 可以向系統管理員發送通知， 也可以通過 API 向其他程序發送通知 

3. 故障遷移（failover發）：自動主從切換

   4. 統一的配置管理：連接者詢問sentinel取得主從的地址

    5、Raft分布式算法

1. 主要用途：用于分布式系統，系統容錯，以及選出領頭羊

2. 作者：Diego Ongaro，畢業于哈佛

3. 目前用到這個算法的項目有：

a. CoreOS : 見下面

b. ectd : a distributed, consistent shared configuration 

c. LogCabin : 分布式存儲系統

d. redis sentinel : redis 的監控系統

   Sentinel使用的Raft算法核心: 原則

1. 所有sentinel都有選舉的領頭羊的權利

2. 每個sentinel都會要求其他sentinel選舉自己為領頭羊(主要由發現redis客觀下線的sentinel先發起選舉)

3. 每個sentinel只有一次選舉的機會

4. 采用先到先得的原則

5. 一旦加入到系統了，則不會自動清除(這一點很重要, why?)

6. 每個sentinel都有唯一的uid，不會因為重啟而變更

7. 達到領頭羊的條件是 N/2 + 1個sentinel選擇了自己

8. 采用配置紀元，如果一次選舉出現腦裂，則配置紀元會遞增，進入下一次選舉，所有sentinel都會處于統一配置紀元，以最新的為標準。

Raft算法核心: 可視圖

   Raft Visualization (算法演示)

   Raft分布式算法的應用

   coreos:云計算新星 Docker 正在以火箭般的速度發展，與它相關的生態圈也漸入佳境，CoreOS 就是其中之一。CoreOS 是一個全新的、面向數據中心設計的 Linux 操作系統，在2014年7月發布了首個穩定版本，目前已經完成了800萬美元的A輪融資。

    一個簡單的主從結構加sentinel集群的架構圖如下： 

              192.168.110.134（master redis，sentinel）----------->192.168.110.135（slave redis，sentinel）

                        |                   \          /            |

                        |                    \        /             |

                        |                     \      /              |

                        |                      \    /               |

                        |                       \  /                |

                        |                        \                  |

                        |                         \                 |

                        |                       /  \                |

                        |                      /    \               |

                        |                     /      \              |

                        |                    /        \             |           

                    192.168.110.132(sentinel)----------->192.168.110.133(slave redis，sentinel)

      上圖是一主一從節點，加上兩個部署了sentinel的集群，sentinel集群之間會互相通信，溝通交流redis節點的狀態，做出相應的判斷并進行處理，這里的主觀下線狀態和客觀下線狀態是比較重要的狀態，它們決定了是否進行故障轉移。 

      可以 通過訂閱指定的頻道信息，當服務器出現故障得時候通知管理員 客戶端可以將 Sentinel 看作是一個只提供了訂閱功能的 Redis 服務

      器，你不可以使用 PUBLISH 命令向這個服務器發送信息，但你可以用 SUBSCRIBE 命令或者 PSUBSCRIBE 命令， 通過訂閱給定的頻道來獲取

      相應的事件提醒。        

     6、配置哨兵

       1）、配置一個哨兵Sentinel的模式：

       把源碼中的sentinel.conf配置文件拷貝到指定目錄

       [root@web1 ~]# mkdir /usr/local/sentinel

       [root@web1 ~]# cd /usr/local/src/redis-3.2.6/

       [root@web1 redis-3.2.6]# cp sentinel.conf /usr/local/sentinel/

       [root@web1 redis-3.2.6]# cd /usr/local/sentinel/

       [root@web1 sentinel]# vim sentinel.conf 

[root@web1 sentinel]# grep -v "^#" sentinel.conf |grep -v "^$"

port 26379

dir /tmp

sentinel monitor mymaster 192.168.110.134 6379 1

sentinel down-after-milliseconds mymaster 30000

sentinel parallel-syncs mymaster 1

sentinel failover-timeout mymaster 180000

        例子：

        #sentinel端口

port 26380

#工作路徑

dir "/usr/local/redis-6380"

# 守護進程模式

daemonize yes

# 指明日志文件名

logfile "./sentinel.log"

#哨兵監控的master，主從配置一樣，在進行主從切換時6379會變成當前的master端口，

sentinel monitor mymaster 192.168.137.40 6379 1

# master或slave多長時間（默認30秒）不能使用后標記為s_down狀態。

sentinel down-after-milliseconds mymaster 5000

#若sentinel在該配置值內未能完成failover操作（即故障時master/slave自動切換），則認為本次failover失敗。

sentinel failover-timeout mymaster 18000

#設置master和slaves驗證密碼

sentinel auth-pass mymaster 123456

#哨兵程序自動添加的部分

# Generated by CONFIG REWRITE

sentinel config-epoch mymaster 0

sentinel leader-epoch mymaster 1

###指明了當前群集的從庫的ip和端口，在主從切換時該值會改變

sentinel known-slave mymaster 192.168.137.40 6380

###除了當前的哨兵還有哪些監控的哨兵

sentinel known-sentinel mymaster 192.168.137.40 26379 7a88891a6147e202a53601ca16a3d438e9d55c9d

sentinel current-epoch 1

主要是這一項：

sentinel monitor mymaster 192.168.110.134 6379 2

my mymaster 后跟的是master的ip和端口

最后一個’2’代表我要啟動只要有2個sentinel認為master下線，就認為該master客觀下線，啟動failover并選舉產生新的master。

通常最后一個參數不能多于啟動的sentinel實例數。

        [root@web2 ~]# mkdir /usr/local/sentinel

[root@web2 ~]# cp /usr/local/src/redis-3.2.6/sentinel.conf /usr/local/sentinel/

[root@web2 ~]# cd /usr/local/sentinel/

[root@web2 sentinel]# vim sentinel.conf 

[root@web2 sentinel]# grep -v "^#" sentinel.conf |grep -v "^$"

port 26379

dir /tmp

sentinel monitor mymaster 192.168.110.134 6379 2

sentinel down-after-milliseconds mymaster 30000

sentinel parallel-syncs mymaster 1

sentinel failover-timeout mymaster 180000

以后臺啟動模式啟動兩個sentinel（哨兵）:

[root@web1 sentinel]# redis-sentinel /usr/local/sentinel/sentinel.conf &

[1] 18378

[root@web1 sentinel]# 18378:X 16 Dec 15:46:16.732 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-``__ ''-._                                             

      _.-``    `.  `_.  ''-._           Redis 3.2.6 (00000000/0) 64 bit

  .-`` .-```.  ```\/    _.,_ ''-._                                   

 (    '      ,       .-`  | `,    )     Running in sentinel mode

 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379

 |    `-._   `._    /     _.-'    |     PID: 18378

  `-._    `-._  `-./  _.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |           http://redis.io        

  `-._    `-._`-.__.-'_.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |                                  

  `-._    `-._`-.__.-'_.-'    _.-'                                   

      `-._    `-.__.-'    _.-'                                       

          `-._        _.-'                                           

              `-.__.-'                                               

18378:X 16 Dec 15:46:16.734 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

18378:X 16 Dec 15:46:16.734 # Sentinel ID is 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

18378:X 16 Dec 15:46:16.734 # +monitor master mymaster 192.168.110.134 6379 quorum 1

由上圖可以看到：

1、 哨兵已經啟動，它的id為236f14b361fc5a0dc0621cf88823ed6e6252b2f3

2、 為master數據庫添加了一個監控

3、 發現了2個slave（由此可以看出，哨兵無需配置slave，只需要指定master，哨兵會自動發現slave）

   [root@web1 sentinel]# ps aux|grep redis

root     18357  0.1  1.0 135572  2440 ?        Ssl  15:40   0:00 redis-server *:6379                         

root     18378  0.1  1.0 133528  2572 pts/1    Sl   15:46   0:00 redis-sentinel *:26379 [sentinel]               

root     18400  0.0  0.3 103244   864 pts/1    S+   15:51   0:00 grep redis

    6、從Redis宕機測試

        把其中一個從redis 的進程 kill掉：

        [[root@web1 sentinel]# kill 18357

[root@web1 sentinel]# ps aux|grep redis

root     18378  0.1  1.0 133528  2572 pts/1    Sl   15:46   0:01 redis-sentinel *:26379 [sentinel]               

root     18444  0.0  0.3 103244   864 pts/1    S+   16:00   0:00 grep redis

        30秒后哨兵的控制臺輸出：

[root@web1 sentinel]# 18378:X 16 Dec 16:01:16.452 # +sdown master mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.452 # +odown master mymaster 192.168.110.134 6379 #quorum 1/1

18378:X 16 Dec 16:01:16.452 # +new-epoch 9

18378:X 16 Dec 16:01:16.452 # +try-failover master mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.470 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 9

18378:X 16 Dec 16:01:16.470 # +elected-leader master mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.470 # +failover-state-select-slave master mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.546 # +selected-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.546 * +failover-state-send-slaveof-noone slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.605 * +failover-state-wait-promotion slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.699 # +promoted-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.699 # +failover-state-reconf-slaves master mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.759 # +failover-end master mymaster 192.168.110.134 6379

18378:X 16 Dec 16:01:16.759 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379

18378:X 16 Dec 16:01:16.760 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

18378:X 16 Dec 16:01:46.858 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

30秒后哨兵的控制臺輸出：

18378:X 16 Dec 16:01:16.759 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379

   18378:X 16 Dec 16:01:16.760 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

   18378:X 16 Dec 16:01:46.858 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

說明已經監控到剛才我人kill掉的master宕機了

        可以看出，slave重新已經轉換成立主庫 -sdown：說明是恢復服務。

        [root@web2 sentinel]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

         把192.168.110.134恢復：

         [root@web1 sentinel]# redis-server /usr/local/redis/etc/redis.conf 

[root@web1 sentinel]# 18378:X 16 Dec 16:06:59.825 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

18378:X 16 Dec 16:07:09.806 * +convert-to-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

[root@web1 sentinel]# 

[root@web1 sentinel]# 

[root@web1 sentinel]# ps aux|grep redis

root     18378  0.2  1.0 133528  2636 pts/1    Sl   15:46   0:02 redis-sentinel *:26379 [sentinel]               

root     18487  0.1  1.0 135572  2564 ?        Ssl  16:06   0:00 redis-server *:6379                         

root     18491  0.0  0.3 103244   864 pts/1    S+   16:08   0:00 grep redis

         192.168.110.134是作為從庫加入到復制中的（主庫不會切換過來）：

測試下：

[root@web2 sentinel]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.110.134,port=6379,state=online,offset=2835,lag=1

master_repl_offset:2835

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:2834

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.110.134,port=6379,state=online,offset=9687,lag=0

master_repl_offset:9832

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:9831

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.110.134,port=6379,state=online,offset=36450,lag=0

master_repl_offset:36595

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:36594

127.0.0.1:6379> set name2 lisi

OK

127.0.0.1:6379> set age2 22

OK

127.0.0.1:6379> get name2

"lisi"

127.0.0.1:6379> get age2

"22"

127.0.0.1:6379> 

[root@web1 sentinel]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:slave

master_host:192.168.110.135

master_port:6379

master_link_status:up

master_last_io_seconds_ago:0

master_sync_in_progress:0

slave_repl_offset:73863

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> get name2

"lisi"

127.0.0.1:6379> get set2

(nil)

127.0.0.1:6379> get age2

"22"

127.0.0.1:6379> 

          效果不錯：

          把主庫切換回192.168.110.134

          [root@web2 sentinel]# ps aux|grep redis

root      7063  0.1  1.2 135572  3116 ?        Ssl  14:35   0:13 redis-server *:6379         

root      7649  0.0  0.3 103244   864 pts/1    S+   16:48   0:00 grep redis

 [root@web2 sentinel]# kill 7063

 [root@web1 sentinel]# 18522:X 16 Dec 16:49:05.725 # +sdown master mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:05.725 # +odown master mymaster 192.168.110.135 6379 #quorum 1/1

18522:X 16 Dec 16:49:05.725 # +new-epoch 10

18522:X 16 Dec 16:49:05.725 # +try-failover master mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:05.736 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 10

18522:X 16 Dec 16:49:05.736 # +elected-leader master mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:05.736 # +failover-state-select-slave master mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:05.791 # +selected-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:05.791 * +failover-state-send-slaveof-noone slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:05.875 * +failover-state-wait-promotion slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:06.233 # +promoted-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:06.233 # +failover-state-reconf-slaves master mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:06.331 # +failover-end master mymaster 192.168.110.135 6379

18522:X 16 Dec 16:49:06.331 # +switch-master mymaster 192.168.110.135 6379 192.168.110.134 6379

18522:X 16 Dec 16:49:06.331 * +slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

18522:X 16 Dec 16:49:36.394 # +sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

切換成功

[root@web1 sentinel]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

恢復192.168.110.135的redis：

[root@web2 sentinel]# redis-server /etc/redis.conf

127.0.0.1:6379> 18522:X 16 Dec 16:52:32.351 # -sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

18522:X 16 Dec 16:52:42.305 * +convert-to-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

        127.0.0.1:6379> 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.110.135,port=6379,state=online,offset=13653,lag=1

master_repl_offset:13667

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:13666

127.0.0.1:6379> 

                                                                                                                                    1

        [root@web2 sentinel]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:slave

master_host:192.168.110.134

master_port:6379

master_link_status:up

master_last_io_seconds_ago:2

master_sync_in_progress:0

slave_repl_offset:16502

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

     2）、配置兩個哨兵Sentinel的模式： 

         [root@web1 sentinel]# grep -v "^$"  /usr/local/sentinel/sentinel.conf |grep -v "^#"

port 26379

dir "/tmp"

sentinel myid 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

sentinel monitor mymaster 192.168.110.134 6379 2

sentinel config-epoch mymaster 10

sentinel leader-epoch mymaster 10

sentinel known-slave mymaster 192.168.110.135 6379

sentinel known-sentinel mymaster 192.168.110.135 26379 16286f52de2675345d1b11bc8946adc5d3f0c360

sentinel current-epoch 10

         [root@web2 sentinel]# grep -v "^$"  /usr/local/sentinel/sentinel.conf |grep -v "^#"

port 26379

dir "/tmp"

sentinel myid 16286f52de2675345d1b11bc8946adc5d3f0c360

sentinel monitor mymaster 192.168.110.134 6379 2

sentinel config-epoch mymaster 0

sentinel leader-epoch mymaster 6

sentinel known-slave mymaster 192.168.110.135 6379

sentinel known-sentinel mymaster 192.168.110.134 26379 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

sentinel current-epoch 10

         主要是這一項：

         sentinel monitor mymaster 192.168.110.134 6379 2

         最后一個’2’代表我要啟動只要有2個sentinel認為master下線，就認為該master客觀下線，啟動failover并選舉產生新的master。

通常最后一個參數不能多于啟動的sentinel實例數。

         [root@web1 sentinel]# redis-sentinel /usr/local/sentinel/sentinel.conf &

[1] 18747

[root@web1 sentinel]# 18747:X 16 Dec 17:04:30.597 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-``__ ''-._                                             

      _.-``    `.  `_.  ''-._           Redis 3.2.6 (00000000/0) 64 bit

  .-`` .-```.  ```\/    _.,_ ''-._                                   

 (    '      ,       .-`  | `,    )     Running in sentinel mode

 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379

 |    `-._   `._    /     _.-'    |     PID: 18747

  `-._    `-._  `-./  _.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |           http://redis.io        

  `-._    `-._`-.__.-'_.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |                                  

  `-._    `-._`-.__.-'_.-'    _.-'                                   

      `-._    `-.__.-'    _.-'                                       

          `-._        _.-'                                           

              `-.__.-'                                               

18747:X 16 Dec 17:04:30.599 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

18747:X 16 Dec 17:04:30.599 # Sentinel ID is 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

18747:X 16 Dec 17:04:30.599 # +monitor master mymaster 192.168.110.134 6379 quorum 2

18747:X 16 Dec 17:05:00.659 # +sdown sentinel 16286f52de2675345d1b11bc8946adc5d3f0c360 192.168.110.135 26379 @ mymaster 192.168.110.134 6379

[root@web2 sentinel]# redis-sentinel /usr/local/sentinel/sentinel.conf &

[1] 7726

You have new mail in /var/spool/mail/root

[root@web2 sentinel]# 7726:X 16 Dec 17:04:33.835 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-``__ ''-._                                             

      _.-``    `.  `_.  ''-._           Redis 3.2.6 (00000000/0) 64 bit

  .-`` .-```.  ```\/    _.,_ ''-._                                   

 (    '      ,       .-`  | `,    )     Running in sentinel mode

 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379

 |    `-._   `._    /     _.-'    |     PID: 7726

  `-._    `-._  `-./  _.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |           http://redis.io        

  `-._    `-._`-.__.-'_.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |                                  

  `-._    `-._`-.__.-'_.-'    _.-'                                   

      `-._    `-.__.-'    _.-'                                       

          `-._        _.-'                                           

              `-.__.-'                                               

7726:X 16 Dec 17:04:33.836 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

7726:X 16 Dec 17:04:33.836 # Sentinel ID is 16286f52de2675345d1b11bc8946adc5d3f0c360

7726:X 16 Dec 17:04:33.836 # +monitor master mymaster 192.168.110.134 6379 quorum 2

7726:X 16 Dec 17:04:34.668 # +new-epoch 10

7726:X 16 Dec 17:05:03.836 # +sdown sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.134 6379

   上面是有問題的：原因是bind 默認是僅僅監控本機和 192.168.1.1這兩個IP地址

   # bind 127.0.0.1 192.168.1.1

   添加： bind 0.0.0.0

   就沒問題了

   配置文件：

   [root@web1 ~]# grep -v "^$" /usr/local/sentinel/sentinel.conf |grep -v "^#"

bind 0.0.0.0

port 26379

dir "/tmp"

sentinel myid 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

sentinel monitor mymaster 192.168.110.135 6379 2

sentinel config-epoch mymaster 14

sentinel leader-epoch mymaster 14

sentinel known-slave mymaster 192.168.110.134 6379

sentinel known-sentinel mymaster 192.168.110.135 26379 16286f52de2675345d1b11bc8946adc5d3f0c360

sentinel current-epoch 14

   主要是下面兩項：

   bind 0.0.0.0

   sentinel monitor mymaster 192.168.110.135 6379 2

   如果設置了密碼則添加下面一項：

   # sentinel auth-pass <master-name> <password>

   sentinel auth-pass def_master 012_345^678-90

   [root@web1 ~]# redis-sentinel /usr/local/sentinel/sentinel.conf &

[1] 1963

[root@web1 ~]# 1963:X 19 Dec 11:55:29.670 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-``__ ''-._                                             

      _.-``    `.  `_.  ''-._           Redis 3.2.6 (00000000/0) 64 bit

  .-`` .-```.  ```\/    _.,_ ''-._                                   

 (    '      ,       .-`  | `,    )     Running in sentinel mode

 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379

 |    `-._   `._    /     _.-'    |     PID: 1963

  `-._    `-._  `-./  _.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |           http://redis.io        

  `-._    `-._`-.__.-'_.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |                                  

  `-._    `-._`-.__.-'_.-'    _.-'                                   

      `-._    `-.__.-'    _.-'                                       

          `-._        _.-'                                           

              `-.__.-'                                               

1963:X 19 Dec 11:55:29.671 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

1963:X 19 Dec 11:55:29.671 # Sentinel ID is 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

1963:X 19 Dec 11:55:29.672 # +monitor master mymaster 192.168.110.134 6379 quorum 2

   上面表示啟動沒問題的：

   kill掉一個從redis，觀察sentinel控制臺輸出信息：

[root@web1 ~]# kill 1912

[root@web1 ~]# ps aux|grep redis

root      1963  0.2  1.0 133528  2588 pts/0    Sl   11:55   0:00 redis-sentinel 0.0.0.0:26379 [sentinel]         

root      1992  0.0  0.3 103244   864 pts/0    S+   11:58   0:00 grep redis

[root@web1 ~]# 1963:X 19 Dec 11:59:06.906 # +sdown master mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:06.963 # +odown master mymaster 192.168.110.134 6379 #quorum 2/2

1963:X 19 Dec 11:59:06.963 # +new-epoch 14

1963:X 19 Dec 11:59:06.963 # +try-failover master mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:06.972 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 14

1963:X 19 Dec 11:59:06.992 # 16286f52de2675345d1b11bc8946adc5d3f0c360 voted for 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 14

1963:X 19 Dec 11:59:07.035 # +elected-leader master mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:07.035 # +failover-state-select-slave master mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:07.136 # +selected-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:07.136 * +failover-state-send-slaveof-noone slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:07.199 * +failover-state-wait-promotion slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:08.171 # +promoted-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:08.171 # +failover-state-reconf-slaves master mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:08.260 # +failover-end master mymaster 192.168.110.134 6379

1963:X 19 Dec 11:59:08.260 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379

1963:X 19 Dec 11:59:08.260 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

1963:X 19 Dec 11:59:38.290 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

另一臺：

[root@web2 ~]# redis-sentinel /usr/local/sentinel/sentinel.conf &

[1] 1809

[root@web2 ~]# 1809:X 19 Dec 11:55:35.698 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-``__ ''-._                                             

      _.-``    `.  `_.  ''-._           Redis 3.2.6 (00000000/0) 64 bit

  .-`` .-```.  ```\/    _.,_ ''-._                                   

 (    '      ,       .-`  | `,    )     Running in sentinel mode

 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379

 |    `-._   `._    /     _.-'    |     PID: 1809

  `-._    `-._  `-./  _.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |           http://redis.io        

  `-._    `-._`-.__.-'_.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |                                  

  `-._    `-._`-.__.-'_.-'    _.-'                                   

      `-._    `-.__.-'    _.-'                                       

          `-._        _.-'                                           

              `-.__.-'                                               

1809:X 19 Dec 11:55:35.700 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

1809:X 19 Dec 11:55:35.700 # Sentinel ID is 16286f52de2675345d1b11bc8946adc5d3f0c360

1809:X 19 Dec 11:55:35.700 # +monitor master mymaster 192.168.110.134 6379 quorum 2

1809:X 19 Dec 11:59:06.895 # +sdown master mymaster 192.168.110.134 6379

1809:X 19 Dec 11:59:06.999 # +new-epoch 14

1809:X 19 Dec 11:59:07.001 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 14

1809:X 19 Dec 11:59:08.020 # +odown master mymaster 192.168.110.134 6379 #quorum 2/2

1809:X 19 Dec 11:59:08.020 # Next failover delay: I will not start a failover before Mon Dec 19 12:05:07 2016

1809:X 19 Dec 11:59:08.284 # +config-update-from sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.134 6379

1809:X 19 Dec 11:59:08.284 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379

1809:X 19 Dec 11:59:08.284 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

1809:X 19 Dec 11:59:38.344 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

[root@web2 ~]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

恢復192.168.110.134：

[root@web1 ~]# redis-server /usr/local/redis/etc/redis.conf 

[root@web1 ~]# 1963:X 19 Dec 12:14:24.393 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

[root@web1 ~]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:slave

master_host:192.168.110.135

master_port:6379

master_link_status:up

master_last_io_seconds_ago:1

master_sync_in_progress:0

slave_repl_offset:38769

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

135上的顯示：

repl_backlog_histlen:0

127.0.0.1:6379> 1809:X 19 Dec 12:14:24.105 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

127.0.0.1:6379> 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.110.134,port=6379,state=online,offset=2404,lag=1

master_repl_offset:2404

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:2403

也是僅僅作為從。

         2）、配置三個哨兵Sentinel的模式：

            [root@web2 ~]# scp redis-3.2.6.tar.gz 192.168.110.133:/root

The authenticity of host '192.168.110.133 (192.168.110.133)' can't be established.

RSA key fingerprint is 60:6c:0d:f4:3b:f8:42:b1:c9:7e:ab:c4:bc:83:d1:09.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.110.133' (RSA) to the list of known hosts.

root@192.168.110.133's password: 

redis-3.2.6.tar.gz                                                                                 100% 1509KB   1.5MB/s   00:00

            [root@node2 ~]# iptables -I INPUT 6 -m state --state NEW -p tcp --dport 6379 -j ACCEPT

            [root@node2 ~]# iptables -I INPUT 7 -m state --state NEW -p tcp --dport 26379 -j ACCEPT

            [root@node2 ~]# tar -zxvf redis-3.2.6.tar.gz -C /usr/local/src/

            [root@node2 ~]# cd /usr/local/src/redis-3.2.6/

            [root@node2 redis-3.2.6]# make

            [root@node2 redis-3.2.6]# make install

            root@node2 redis-3.2.6]# mkdir /etc/redis

[root@node2 redis-3.2.6]# mkdir -p /redis/data

[root@node2 redis-3.2.6]# mkdir -p /redis/log

     2、配置主從：

[root@node2 redis-3.2.6]# grep -v "^$" /etc/redis/redis.conf |grep -v "^#"

bind *

protected-mode yes

port 6379

tcp-backlog 511

timeout 0

tcp-keepalive 300

daemonize yes

supervised no

pidfile /var/run/redis_6379.pid

loglevel notice

logfile "/redis/log/redis.log"

databases 16

save 900 1

save 300 10

save 60 10000

stop-writes-on-bgsave-error yes

rdbcompression yes

rdbchecksum yes

dbfilename dump.rdb

dir /redis/data

slaveof 192.168.110.134 6379 

slave-serve-stale-data yes

slave-read-only yes

repl-diskless-sync no

repl-diskless-sync-delay 5

repl-disable-tcp-nodelay no

slave-priority 100

appendonly no

appendfilename "appendonly.aof"

appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

lua-time-limit 5000

slowlog-log-slower-than 10000

slowlog-max-len 128

latency-monitor-threshold 0

notify-keyspace-events ""

hash-max-ziplist-entries 512

hash-max-ziplist-value 64

list-max-ziplist-size -2

list-compress-depth 0

set-max-intset-entries 512

zset-max-ziplist-entries 128

zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

activerehashing yes

client-output-buffer-limit normal 0 0 0

client-output-buffer-limit slave 256mb 64mb 60

client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

aof-rewrite-incremental-fsync yes

            [root@node2 redis-3.2.6]# redis-server /etc/redis/redis.conf 

[root@node2 redis-3.2.6]# ps aux|grep redis

root      4220  0.1  0.9 133524  2312 ?        Ssl  16:36   0:00 redis-server *:6379               

root      4224  0.0  0.3 103248   864 pts/0    S+   16:36   0:00 grep redis

            [root@web1 ~]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:2

slave0:ip=192.168.110.135,port=6379,state=online,offset=325246,lag=0

slave1:ip=192.168.110.133,port=6379,state=online,offset=325246,lag=0

master_repl_offset:325246

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:325245

127.0.0.1:6379> 

     2、設置哨兵：

            [root@node2 redis-3.2.6]# mkdir /etc/sentinel

[root@node2 redis-3.2.6]# cp sentinel.conf /etc/sentinel/

[root@node2 redis-3.2.6]# grep -v "^$" /etc/sentinel/sentinel.conf |grep -v "^#"

bind 0.0.0.0

port 26379

dir "/tmp"

sentinel myid fd3e2ac303fd06dfa7934718d68429dc88e3b5b8

sentinel monitor mymaster 192.168.110.134 6379 2

sentinel config-epoch mymaster 15

sentinel leader-epoch mymaster 0

sentinel known-slave mymaster 192.168.110.133 6379

sentinel known-slave mymaster 192.168.110.135 6379

sentinel known-sentinel mymaster 192.168.110.135 26379 16286f52de2675345d1b11bc8946adc5d3f0c360

sentinel known-sentinel mymaster 192.168.110.134 26379 28c5f96738e4fe3335088d037a85ef47dfe9ab8f

sentinel current-epoch 15

[root@node2 redis-3.2.6]# redis-sentinel /etc/sentinel/sentinel.conf &

[1] 4240

[root@node2 redis-3.2.6]# 4240:X 18 Dec 16:48:07.492 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-``__ ''-._                                             

      _.-``    `.  `_.  ''-._           Redis 3.2.6 (00000000/0) 64 bit

  .-`` .-```.  ```\/    _.,_ ''-._                                   

 (    '      ,       .-`  | `,    )     Running in sentinel mode

 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379

 |    `-._   `._    /     _.-'    |     PID: 4240

  `-._    `-._  `-./  _.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |           http://redis.io        

  `-._    `-._`-.__.-'_.-'    _.-'                                   

 |`-._`-._    `-.__.-'    _.-'_.-'|                                  

 |    `-._`-._        _.-'_.-'    |                                  

  `-._    `-._`-.__.-'_.-'    _.-'                                   

      `-._    `-.__.-'    _.-'                                       

          `-._        _.-'                                           

              `-.__.-'                                               

4240:X 18 Dec 16:48:07.494 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

4240:X 18 Dec 16:48:07.494 # Sentinel ID is fd3e2ac303fd06dfa7934718d68429dc88e3b5b8

4240:X 18 Dec 16:48:07.494 # +monitor master mymaster 192.168.110.134 6379 quorum 2

4240:X 18 Dec 16:48:07.499 * +slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

4240:X 18 Dec 16:48:07.515 * +slave slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379

4240:X 18 Dec 16:48:08.295 * +sentinel sentinel 16286f52de2675345d1b11bc8946adc5d3f0c360 192.168.110.135 26379 @ mymaster 192.168.110.134 6379

4240:X 18 Dec 16:48:08.328 # +new-epoch 15

4240:X 18 Dec 16:48:09.420 * +sentinel sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.134 6379

          [root@node2 redis-3.2.6]# ps aux|grep redis

root      4220  0.2  1.0 133524  2408 ?        Ssl  16:36   0:03 redis-server *:6379               

root      4240  0.3  1.1 133528  2696 pts/0    Sl   16:48   0:02 redis-sentinel 0.0.0.0:26379 [sentinel]   

root      4264  0.0  0.3 103248   860 pts/0    S+   17:01   0:00 grep redis

          [root@web2 ~]# redis-cli -p 26379

127.0.0.1:26379> info

....           ..........

# Sentinel

sentinel_masters:1

sentinel_tilt:0

sentinel_running_scripts:0

sentinel_scripts_queue_length:0

sentinel_simulate_failure_flags:0

master0:name=mymaster,status=ok,address=192.168.110.134:6379,slaves=2,sentinels=2

127.0.0.1:26379> 1809:X 19 Dec 14:59:24.984 * +sentinel sentinel fd3e2ac303fd06dfa7934718d68429dc88e3b5b8 192.168.110.133 26379 @ mymaster 192.168.110.134 6379

127.0.0.1:26379> 

127.0.0.1:26379> info

# Server

redis_version:3.2.6

redis_git_sha1:00000000

redis_git_dirty:0

redis_build_id:2f0dd1a83d7e7010

redis_mode:sentinel

os:Linux 2.6.32-431.el6.x86_64 x86_64

arch_bits:64

multiplexing_api:epoll

gcc_version:4.4.7

process_id:1809

run_id:519043c8b9e7314a71c425400d01769b3384f77c

tcp_port:26379

uptime_in_seconds:11045

uptime_in_days:0

hz:11

lru_clock:5735772

executable:/root/redis-sentinel

config_file:/usr/local/sentinel/sentinel.conf

# Clients

connected_clients:3

client_longest_output_list:0

client_biggest_input_buf:0

blocked_clients:0

# CPU

used_cpu_sys:26.71

used_cpu_user:2.31

used_cpu_sys_children:0.00

used_cpu_user_children:0.00

# Stats

total_connections_received:3

total_commands_processed:16011

instantaneous_ops_per_sec:4

total_net_input_bytes:937405

total_net_output_bytes:97572

instantaneous_input_kbps:0.26

instantaneous_output_kbps:0.02

rejected_connections:0

sync_full:0

sync_partial_ok:0

sync_partial_err:0

expired_keys:0

evicted_keys:0

keyspace_hits:0

keyspace_misses:0

pubsub_channels:0

pubsub_patterns:0

latest_fork_usec:0

migrate_cached_sockets:0

# Sentinel

sentinel_masters:1

sentinel_tilt:0

sentinel_running_scripts:0

sentinel_scripts_queue_length:0

sentinel_simulate_failure_flags:0

master0:name=mymaster,status=ok,address=192.168.110.134:6379,slaves=2,sentinels=3

127.0.0.1:26379> 

         [root@node2 redis-3.2.6]# redis-cli -p 26379

127.0.0.1:26379> info

# Server

redis_version:3.2.6

redis_git_sha1:00000000

redis_git_dirty:0

redis_build_id:98c29634d36aff37

redis_mode:sentinel

os:Linux 2.6.32-431.el6.x86_64 x86_64

arch_bits:64

multiplexing_api:epoll

gcc_version:4.4.7

process_id:4240

run_id:d0cf3dbc5259fa960ccf02ed101bc69bd878e26d

tcp_port:26379

uptime_in_seconds:630

uptime_in_days:0

hz:10

lru_clock:5656509

executable:/usr/local/src/redis-3.2.6/redis-sentinel

config_file:/etc/sentinel/sentinel.conf

# Clients

connected_clients:3

client_longest_output_list:0

client_biggest_input_buf:0

blocked_clients:0

# CPU

used_cpu_sys:1.88

used_cpu_user:0.12

used_cpu_sys_children:0.00

used_cpu_user_children:0.00

# Stats

total_connections_received:3

total_commands_processed:1819

instantaneous_ops_per_sec:2

total_net_input_bytes:106447

total_net_output_bytes:10937

instantaneous_input_kbps:0.25

instantaneous_output_kbps:0.01

rejected_connections:0

sync_full:0

sync_partial_ok:0

sync_partial_err:0

expired_keys:0

evicted_keys:0

keyspace_hits:0

keyspace_misses:0

pubsub_channels:0

pubsub_patterns:0

latest_fork_usec:0

migrate_cached_sockets:0

# Sentinel

sentinel_masters:1

sentinel_tilt:0

sentinel_running_scripts:0

sentinel_scripts_queue_length:0

sentinel_simulate_failure_flags:0

master0:name=mymaster,status=ok,address=192.168.110.134:6379,slaves=2,sentinels=3

           [root@node2 redis-3.2.6]# redis-cli

127.0.0.1:6379> get name

"zhangsan"

127.0.0.1:6379> get age

"20"

127.0.0.1:6379> get age1

(nil)

127.0.0.1:6379> get age2

"22"

127.0.0.1:6379> 

          宕掉192.168.110.134的主庫：

          [root@web1 ~]# ps aux|grep redis

root      1963  0.2  1.1 133528  2716 pts/0    Sl   11:55   0:31 redis-sentinel 0.0.0.0:26379 [sentinel]         

root      2068  0.1  1.4 135572  3468 ?        Ssl  12:14   0:20 redis-server *:6379                         

root      2817  0.0  0.3 103244   864 pts/0    S+   15:11   0:00 grep redis

[root@web1 ~]# kill 2068

[root@web1 ~]# ps aux|grep redis

root      1963  0.2  1.1 133528  2716 pts/0    Sl   11:55   0:31 redis-sentinel 0.0.0.0:26379 [sentinel]         

root      2826  0.0  0.3 103244   864 pts/0    S+   15:13   0:00 grep redis

[root@web1 ~]# 1963:X 19 Dec 15:13:44.721 # +sdown master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:44.776 # +odown master mymaster 192.168.110.134 6379 #quorum 3/2

1963:X 19 Dec 15:13:44.776 # +new-epoch 16

1963:X 19 Dec 15:13:44.776 # +try-failover master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:44.778 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 16

1963:X 19 Dec 15:13:44.797 # fd3e2ac303fd06dfa7934718d68429dc88e3b5b8 voted for 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 16

1963:X 19 Dec 15:13:44.804 # 16286f52de2675345d1b11bc8946adc5d3f0c360 voted for 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 16

1963:X 19 Dec 15:13:44.855 # +elected-leader master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:44.855 # +failover-state-select-slave master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:44.956 # +selected-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:44.956 * +failover-state-send-slaveof-noone slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:45.023 * +failover-state-wait-promotion slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:45.898 # +promoted-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:45.898 # +failover-state-reconf-slaves master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:45.963 * +slave-reconf-sent slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:46.107 * +slave-reconf-inprog slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:46.911 # -odown master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:47.154 * +slave-reconf-done slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:47.212 # +failover-end master mymaster 192.168.110.134 6379

1963:X 19 Dec 15:13:47.212 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379

1963:X 19 Dec 15:13:47.213 * +slave slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.135 6379

1963:X 19 Dec 15:13:47.213 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

1963:X 19 Dec 15:14:17.223 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379

             135成了主庫：

             [root@web2 ~]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.110.133,port=6379,state=online,offset=27695,lag=0

master_repl_offset:27695

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:27694

127.0.0.1:6379> quit

             [root@node2 redis-3.2.6]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:slave

master_host:192.168.110.135

master_port:6379

master_link_status:up

master_last_io_seconds_ago:0

master_sync_in_progress:0

slave_repl_offset:21626

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

              再宕掉135：

              [root@web2 ~]# ps aux|grep redis

root      1809  0.2  1.1 133528  2764 pts/0    Sl   11:55   0:32 redis-sentinel 0.0.0.0:26379 [sentinel]         

root      2390  0.2  1.1 135572  2872 ?        Ssl  14:11   0:08 redis-server *:6379         

root      2675  0.0  0.3 103244   860 pts/0    S+   15:16   0:00 grep redis

[root@web2 ~]# kill 2390

[root@web2 ~]# ps aux|grep redis

root      1809  0.2  1.1 133528  2768 pts/0    Sl   11:55   0:32 redis-sentinel 0.0.0.0:26379 [sentinel]         

root      2677  0.0  0.2 103216   640 pts/0    R+   15:17   0:00 grep redis

[root@web2 ~]# 1809:X 19 Dec 15:17:31.230 # +sdown master mymaster 192.168.110.135 6379

1809:X 19 Dec 15:17:31.271 # +new-epoch 17

1809:X 19 Dec 15:17:31.286 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 17

1809:X 19 Dec 15:17:31.302 # +odown master mymaster 192.168.110.135 6379 #quorum 3/2

1809:X 19 Dec 15:17:31.302 # Next failover delay: I will not start a failover before Mon Dec 19 15:23:31 2016

1809:X 19 Dec 15:17:32.002 # +config-update-from sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.135 6379

1809:X 19 Dec 15:17:32.003 # +switch-master mymaster 192.168.110.135 6379 192.168.110.133 6379

1809:X 19 Dec 15:17:32.003 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379

1809:X 19 Dec 15:17:32.003 * +slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379

1809:X 19 Dec 15:18:02.004 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379

1809:X 19 Dec 15:18:02.004 # +sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379

               133成立主庫：

                127.0.0.1:6379> 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

              [root@web1 ~]# redis-server /usr/local/redis/etc/redis.conf 

[root@web1 ~]# 1963:X 19 Dec 15:21:47.745 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379

1963:X 19 Dec 15:21:57.749 * +convert-to-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379

              [root@web2 ~]# redis-server /etc/redis.conf 

[root@web2 ~]# 1809:X 19 Dec 15:22:34.441 # -sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379

1809:X 19 Dec 15:22:44.421 * +convert-to-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379

              133上查看的：

              127.0.0.1:6379> 4240:X 18 Dec 17:11:18.953 # -sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379

127.0.0.1:6379> 

127.0.0.1:6379> info replication

# Replication

role:master

connected_slaves:2

slave0:ip=192.168.110.134,port=6379,state=online,offset=24323,lag=1

slave1:ip=192.168.110.135,port=6379,state=online,offset=24470,lag=0

master_repl_offset:24470

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:24469

127.0.0.1:6379> 

               [root@web1 ~]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:slave

master_host:192.168.110.133

master_port:6379

master_link_status:up

master_last_io_seconds_ago:0

master_sync_in_progress:0

slave_repl_offset:113832

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

               [root@web2 ~]# redis-cli 

127.0.0.1:6379> info replication

# Replication

role:slave

master_host:192.168.110.133

master_port:6379

master_link_status:up

master_last_io_seconds_ago:1

master_sync_in_progress:0

slave_repl_offset:122708

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

127.0.0.1:6379> 

    sentinel一些命令介紹 

    要使用sentinel的命令，我們需要用redis-cli命令進入到sentinel：

    1、INFO 

sentinel的基本狀態信息 

2、SENTINEL masters 

列出所有被監視的主服務器，以及這些主服務器的當前狀態 

3、SENTINEL slaves 

列出給定主服務器的所有從服務器，以及這些從服務器的當前狀態 

4、SENTINEL get-master-addr-by-name 

返回給定名字的主服務器的 IP 地址和端口號 

5、SENTINEL reset 

重置所有名字和給定模式 pattern 相匹配的主服務器。重置操作清除主服務器目前的所有狀態， 包括正在執行中的故障轉移， 并移除目前已經發現和關聯的， 主服務器的所有從服務器和 Sentinel 。 

6、SENTINEL failover 

當主服務器失效時， 在不詢問其他 Sentinel 意見的情況下， 強制開始一次自動故障遷移，但是它會給其他sentinel發送一個最新的配置，其他sentinel會根據這個配置進行更新