ceph-deploy 添加/刪除 mon節點

1、簡介

????mon 操作 我還真是比較少，關于 mon 添加刪除 操作 我反正 從來 沒有 在線上環境執行過；哈哈哈，為了以備 不時之需，還是 練練手 了解下（有備無患 嘛）；本文先 基于 ceph-deploy 工具 操作（手動的 反正 我也試過了 -- 沒難度，有興趣的 可以 自己 手動 嘗試下，需要的話 留言 后期 補上 手動 操作文檔）
????

1.1 操作系統

## 本文操作系統 Ubuntu 16.04 amd64
# lsb_release -a

????

1.2 ceph 版本

????ceph Luminous 12.2.5
????

2、刪除 mon

????為什么 先來 刪除 mon，因為 我得 環境 剛好 部署了 三個 mon，先刪除 兩個 mon
????

2.1 刪除

# ceph-deploy --username cstor mon destroy mon2
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cstor/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.39): /usr/bin/ceph-deploy --username cstor mon destroy mon2
[ceph_deploy.cli][INFO  ] ceph-deploy options:
[ceph_deploy.cli][INFO  ]  username                      : cstor
[ceph_deploy.cli][INFO  ]  verbose                       : False
[ceph_deploy.cli][INFO  ]  overwrite_conf                : False
[ceph_deploy.cli][INFO  ]  subcommand                    : destroy
[ceph_deploy.cli][INFO  ]  quiet                         : False
[ceph_deploy.cli][INFO  ]  cd_conf                       : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7effa42b24d0>
[ceph_deploy.cli][INFO  ]  cluster                       : ceph
[ceph_deploy.cli][INFO  ]  mon                           : ['mon2']
[ceph_deploy.cli][INFO  ]  func                          : <function mon at 0x7effa47179b0>
[ceph_deploy.cli][INFO  ]  ceph_conf                     : None
[ceph_deploy.cli][INFO  ]  default_release               : False
[ceph_deploy.mon][DEBUG ] Removing mon from mon2
cstor@mon2's password: 
[mon2][DEBUG ] connection detected need for sudo
cstor@mon2's password: 
[mon2][DEBUG ] connected to host: cstor@mon2 
[mon2][DEBUG ] detect platform information from remote host
[mon2][DEBUG ] detect machine type
[mon2][DEBUG ] find the location of an executable
[mon2][DEBUG ] get remote short hostname
[mon2][INFO  ] Running command: sudo ceph --cluster=ceph -n mon. -k /var/lib/ceph/mon/ceph-mon2/keyring mon remove mon2
[mon2][WARNIN] removing mon.mon2 at XXX.XXX.XXX.XXX:6789/0, there will be 2 monitors
[mon2][INFO  ] polling the daemon to verify it stopped
[mon2][INFO  ] Running command: sudo systemctl stop ceph-mon@mon2.service
[mon2][INFO  ] Running command: sudo mkdir -p /var/lib/ceph/mon-removed
[mon2][DEBUG ] move old monitor data

????

2.2 配置 修改

????我也不知道 不修改 配置 會不會 有什么 影響（最好還是 修改一下吧）-- mon_initial_members、mon_host 這兩個參數（修改 集群 中 所有得配置 -- 雖然麻煩，但是 還需要操作哦）

# cat /etc/ceph/ceph.conf
    ... ...
    mon_initial_members = mon0, mon1
    mon_host = XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX
    ... ...

????

3、添加 mon

????哈哈哈，刪除 mon 之后，現在 開始 添加 mon （開始埋坑）
????

3.1 配置修改

## 修改配置文件時，需要 將集群 得配置文件 統一
# cat /etc/ceph/ceph.conf
    ... ...
    mon_initial_members = mon0, mon1,mon2
    mon_host = XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX
        ... ...

????

3.2 添加 mon

# ceph-deploy --username cstor mon create mon2
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cstor/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.39): /usr/bin/ceph-deploy --username cstor mon create mon2
[ceph_deploy.cli][INFO  ] ceph-deploy options:
[ceph_deploy.cli][INFO  ]  username                      : cstor
[ceph_deploy.cli][INFO  ]  verbose                       : False
[ceph_deploy.cli][INFO  ]  overwrite_conf                : False
[ceph_deploy.cli][INFO  ]  subcommand                    : create
[ceph_deploy.cli][INFO  ]  quiet                         : False
[ceph_deploy.cli][INFO  ]  cd_conf                       : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f1efcd984d0>
[ceph_deploy.cli][INFO  ]  cluster                       : ceph
[ceph_deploy.cli][INFO  ]  mon                           : ['mon2']
[ceph_deploy.cli][INFO  ]  func                          : <function mon at 0x7f1efd1fd9b0>
[ceph_deploy.cli][INFO  ]  ceph_conf                     : None
[ceph_deploy.cli][INFO  ]  keyrings                      : None
[ceph_deploy.cli][INFO  ]  default_release               : False
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts mon2
[ceph_deploy.mon][DEBUG ] detecting platform for host mon2 ...
cstor@mon2's password: 
[mon2][DEBUG ] connection detected need for sudo
cstor@mon2's password: 
[mon2][DEBUG ] connected to host: cstor@mon2 
[mon2][DEBUG ] detect platform information from remote host
[mon2][DEBUG ] detect machine type
[mon2][DEBUG ] find the location of an executable
[ceph_deploy.mon][INFO  ] distro info: Ubuntu 16.04 xenial
[mon2][DEBUG ] determining if provided host has same hostname in remote
[mon2][DEBUG ] get remote short hostname
[mon2][DEBUG ] deploying mon to mon2
[mon2][DEBUG ] get remote short hostname
[mon2][DEBUG ] remote hostname: mon2
[mon2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[mon2][DEBUG ] create the mon path if it does not exist
[mon2][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-mon2/done
[mon2][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-mon2/done
[mon2][INFO  ] creating keyring file: /var/lib/ceph/tmp/ceph-mon2.mon.keyring
[mon2][DEBUG ] create the monitor keyring file
[mon2][INFO  ] Running command: sudo ceph-mon --cluster ceph --mkfs -i mon2 --keyring /var/lib/ceph/tmp/ceph-mon2.mon.keyring --setuser 64045 --setgroup 64045
[mon2][INFO  ] unlinking keyring file /var/lib/ceph/tmp/ceph-mon2.mon.keyring
[mon2][DEBUG ] create a done file to avoid re-doing the mon deployment
[mon2][DEBUG ] create the init path if it does not exist
[mon2][INFO  ] Running command: sudo systemctl enable ceph.target
[mon2][INFO  ] Running command: sudo systemctl enable ceph-mon@mon2
[mon2][INFO  ] Running command: sudo systemctl start ceph-mon@mon2
[mon2][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon2.asok mon_status
[mon2][DEBUG ] ********************************************************************************
[mon2][DEBUG ] status for monitor: mon.mon2
[mon2][DEBUG ] {
[mon2][DEBUG ]   "election_epoch": 0, 
[mon2][DEBUG ]   "extra_probe_peers": [
[mon2][DEBUG ]     "XXX.XXX.XXX.XXX:6789/0", 
[mon2][DEBUG ]     "XXX.XXX.XXX.XXX:6789/0"
[mon2][DEBUG ]   ], 
[mon2][DEBUG ]   "feature_map": {
[mon2][DEBUG ]     "mon": {
[mon2][DEBUG ]       "group": {
[mon2][DEBUG ]         "features": "0x1ffddff8eea4fffb", 
[mon2][DEBUG ]         "num": 1, 
[mon2][DEBUG ]         "release": "luminous"
[mon2][DEBUG ]       }
[mon2][DEBUG ]     }
[mon2][DEBUG ]   }, 
[mon2][DEBUG ]   "features": {
[mon2][DEBUG ]     "quorum_con": "0", 
[mon2][DEBUG ]     "quorum_mon": [], 
[mon2][DEBUG ]     "required_con": "0", 
[mon2][DEBUG ]     "required_mon": [
[mon2][DEBUG ]       "kraken", 
[mon2][DEBUG ]       "luminous"
[mon2][DEBUG ]     ]
[mon2][DEBUG ]   }, 
[mon2][DEBUG ]   "monmap": {
[mon2][DEBUG ]     "created": "2019-01-04 15:43:39.644529", 
[mon2][DEBUG ]     "epoch": 3, 
[mon2][DEBUG ]     "features": {
[mon2][DEBUG ]       "optional": [], 
[mon2][DEBUG ]       "persistent": [
[mon2][DEBUG ]         "kraken", 
[mon2][DEBUG ]         "luminous"
[mon2][DEBUG ]       ]
[mon2][DEBUG ]     }, 
[mon2][DEBUG ]     "fsid": "79fabbdf-b031-4621-952a-3fe880068501", 
[mon2][DEBUG ]     "modified": "2019-02-22 15:47:19.502869", 
[mon2][DEBUG ]     "mons": [
[mon2][DEBUG ]       {
[mon2][DEBUG ]         "addr": "XXX.XXX.XXX.XXX:6789/0", 
[mon2][DEBUG ]         "name": "mon0", 
[mon2][DEBUG ]         "public_addr": "XXX.XXX.XXX.XXX:6789/0", 
[mon2][DEBUG ]         "rank": 0
[mon2][DEBUG ]       }
[mon2][DEBUG ]     ]
[mon2][DEBUG ]   }, 
[mon2][DEBUG ]   "name": "mon2", 
[mon2][DEBUG ]   "outside_quorum": [], 
[mon2][DEBUG ]   "quorum": [], 
[mon2][DEBUG ]   "rank": -1, 
[mon2][DEBUG ]   "state": "probing", 
[mon2][DEBUG ]   "sync_provider": []
[mon2][DEBUG ] }
[mon2][DEBUG ] ********************************************************************************
[mon2][INFO  ] monitor: mon.mon2 is currently at the state of probing
[mon2][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon2.asok mon_status
[mon2][WARNIN] monitor mon2 does not exist in monmap
[mon2][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[mon2][WARNIN] monitors may not be able to form quorum

????

4、mon 狀態 查詢

# ceph -s
# ceph quorum_status [--format json-pretty]
# ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.{mon_name}.asok mon_status