Ceph分布式存儲安裝

Ceph分布式存儲安裝

前言

參照官方文檔中的快速安裝，結合國內環境，將相關安裝源修改為國內鏡像源提高安裝速度。

http://docs.ceph.org.cn/start/

基礎系統環境

添加阿里云YUM源

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo

更新系統

yum update -y```

### 關閉防火墻、SELinux

systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config

### 添加hosts主機名解析（集群采用四臺虛擬機） 
四臺機器均添加

echo "192.168.5.191 admin-node
192.168.5.192 node1
192.168.5.193 node2
192.168.5.194 node3" >> /etc/hosts```

安裝NTP服務，免因時鐘漂移導致故障

yum install ntp ntpdate ntp-doc -y ```

### 免密鑰登錄

admin-node #到node1 node2 node3之間

ssh-keygen #生成密鑰文件```
分別添加至三個node

hostname=(admin-node node1 node2 node3)
for i in ${hostname[*]};do echo yes|ssh-copy-id -i id_rsa.pub root@$i ;done

添加中科大源,否則 ceph-deploy安裝ceph是會缺少依整包而報錯

yum install -y yum-utils && yum-config-manager --add-repo http://mirrors.ustc.edu.cn/epel/7/x86_64/ &&  yum install --nogpgcheck -y epel-release &&  rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 &&  rm -rf /etc/yum.repos.d/mirrors.ustc.edu.cn_epel*

添加 Ceph的國內源，安裝ceph-deploy部署工具

cat >> /etc/yum.repos.d/ceph.repo << EOF 

[Ceph]
name=Ceph packages for $basearch
baseurl=http://mirrors.163.com/ceph/rpm-nautilus/el7/x86_64/
enabled=1
gpgcheck=0
type=rpm-md
gpgkey=http://mirrors.163.com/ceph/keys/release.asc
priority=1

[Ceph-noarch]
name=Ceph noarch packages
baseurl=http://mirrors.163.com/ceph/rpm-nautilus/el7/noarch/
enabled=1 
gpgcheck=0
type=rpm-md
gpgkey=http://mirrors.163.com/ceph/keys/release.asc
priority=1

[ceph-source]
name=Ceph source packages
baseurl=http://mirrors.163.com/ceph/rpm-nautilus/el7/SRPMS/
enabled=1
gpgcheck=0
type=rpm-md
gpgkey=http://mirrors.163.com/ceph/keys/release.asc
priority=1
EOF

yum -y update &&  yum -y install ceph-deploy```

### 使用ceph-deploy在所有節點安裝軟件包

#### 命令清除配置：

ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys

#### 連 Ceph 安裝包一起清除
ceph-deploy purge {ceph-node} [{ceph-node}]

在管理節點(admin-node)

  `mkdir my-cluster && cd my-cluster`

 ##### ceph-deploy  new admin-node node1 node2 node3 創建集群。

sed -i '2a osd pool default size = 2' ceph.conf # 修改默認副本數從 3 改成 2
sed -i '3a public network = 10.0.5.0/24' ceph.conf

#### 修改ceph-deploy安裝過程中的源

export CEPH_DEPLOY_REPO_URL=http://mirrors.163.com/ceph/rpm-nautilus/el7/
export CEPH_DEPLOY_GPG_URL=http://mirrors.163.com/ceph/keys/release.asc

ceph-deploy install admin-node node1 node2 node3

#### 生成密鑰環 

ceph-deploy mon create-initial```

用 ceph-deploy 把配置文件和 admin 密鑰拷貝到管理節點和 Ceph 節點

ceph-deploy admin admin-node node1 node2 node3```

#### 檢查集群健康狀態

ceph health ```

添加OSD進程

 ceph-deploy osd create node1 --data /dev/sdb
 ceph-deploy osd create node1 --data /dev/sdb
 ceph-deploy osd create node3 --data /dev/sdb