Greenplum docker 安裝測試環境

1. 下載docker 拉去docker 鏡像，創建鏡像
   yum -y install docker
   systemctl start dokcer
   docker pull centos：6
   docker run --privileged -dti -p 65000:6500 --name gptest1 centos:6 bash
   docker run --privileged -dti --name gptest2 centos:6 bash
   docker run --privileged -dti --name gptest3 centos:6 bash
   docker run --privileged -dti --name gptest4 centos:6 bash
   --privileges docker 有root權限

2、下載依賴包并啟動ssh,并配置時間同步
docker中默認沒有啟動ssh，為了方便各節點之間的互連，啟動docker的每個節點里面的ssh，并創建相關的認證key
yum install -y net-tools which openssh-clients openssh-server less zip
unzip iproute.x86_64 vim ntp ed
yum install update 更新系統內核
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
/usr/sbin/sshd
ntpdate ntp1.aliyun.com

3. 配置主機域名映射關系
   172.17.0.2 dw-greenplum-1 mdw
   172.17.0.3 dw-greenplum-2 sdw1
   172.17.0.4 dw-greenplum-3 sdw2
   172.17.0.4 dw-greenplum-4 sdw3
   同時，修改所有節點里面的 /etc/sysconfig/network 文件，保持主機名一致
   修改/etc/hostname 的名字
   [root@mdw /]# cat /etc/hostname
   mdw
   [root@mdw /]# cat /etc/sysconfig/network
   NETWORKING=yes
   HOSTNAME=mdw

4. 修改每個節點上面的文件打開數量的限制
   cat /etc/security/limits.conf
   • soft nofile 65536
   • hard nofile 65536
   • soft nproc 131072
   • hard nproc 131072
     [root@mdw /]# cat /etc/security/limits.d/90-nproc.conf
   • soft nproc 131072
     root soft nproc unlimited

5. 修改內核參數
   sysctl -w kernel.sem="500 64000 50 150"
   cat /proc/sys/kernel/sem

6. 關閉所有節點的防火墻和selinux
   service iptables stop
   chkconfig iptables off
   setenforce 0

7. 下載greenplum安裝包
   需要注冊賬號。也可以在公司的機器上面拉取現有的二進制包
   https://network.pivotal.io/products/pivotal-gpdb

7.所有節點創建 greenplum 的用戶和用戶組
groupadd -g 530 gpadmin
useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin
chown -R gpadmin:gpadmin /home/gpadmin
echo 123456 |passwd gpadmin --stdin

8. 在master上面安裝greenplum
   su - gpadmin
   unzip greenplum-db-4.3.14.1-rhel5-x86_64.zip
   sh /greenplum-db-4.3.14.1-rhel5-x86_64.bin
   安裝期間需要修改默認安裝目錄，輸入/home/gpadmin/greenplum-db
   為了方便安裝集群，greenplum提供了批量操作的指令，通過創建配置文件，以使用批處理命令
   創建文件
   [gpadmin@mdw ~]$ cat conf/hostlist
   mdw
   sdw1
   sdw2
   sdw3
   [gpadmin@mdw ~]$ cat conf/seg_hosts
   sdw1
   sdw2

9. 設置環境變量打通所有的節點
   source greenplum-db/greenplum_path.sh
   [gpadmin@mdw ~]$ gpssh-exkeys -f /home/gpadmin/conf/hostlist //傳輸秘鑰
   [STEP 1 of 5] create local ID and authorize on local host
   ... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] authorize current user on remote hosts
... send to sdw1
... send to sdw2

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts
... finished key exchange with sdw1
... finished key exchange with sdw2
... finished key exchange with sdw3
[INFO] completed successfully
使用gpssh-exkeys命令時一定要使用gpadmin身份，因為會在/home/gpadmin/.ssh
中生成ssh的免密碼登錄秘鑰，
[gpadmin@mdw ~]$ gpssh -f /home/gpadmin/conf/hostlist
Note: command history unsupported on this machine ...
=> pwd
[ mdw] /home/gpadmin
[sdw2] /home/gpadmin
[sdw1] /home/gpadmin
[sdw3] /home/gpadmin

10. 分發所有的安裝包到每個值節點
    tar cf greenplum.tar greenplum-db/
    將文件拷貝到每個機器上面
    gpscp -f /home/gpadmin/conf/hostlist greenplum.tar =:/home/gpadmin/
    批量解壓
    tar -xf greenplum.tar
    這樣所有的節點就完成的安裝

11. 初始化創建數據目錄
    [gpadmin@mdw conf]$ gpssh -f hostlist
    => mkdir gpdata
    [ mdw]
    [sdw2]
    [sdw1]
    [sdw3]
    => cd gpdata
    [ mdw]
    [sdw2]
    [sdw1]
    [sdw3]
    => mkdir gpmaster gpdatap1 gpdatap2 gpdatam1 gpdatam2
    [ mdw]
    [sdw2]
    [sdw1]
    [sdw3]
    => exit

每個節點上面配置環境變量
cat .bash_profile
source /home/gpadmin/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/home/gpadmin/gpdata/gpmaster/gpseg-1
export PGPORT=6500
export PGDATABASE=postgres
source .bash_profile

配置初始化文件
[gpadmin@mdw conf]$ cat gpinitsystem_config
ARRAY_NAME="Greenplum"
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts

Segment 的名稱前綴

SEG_PREFIX=gpseg

Primary Segment 起始的端口號

PORT_BASE=33000

指定 Primary Segment 的數據目錄

declare -a DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatap1 /home/gpadmin/gpdata/gpdatap2)

Master 所在機器的 Hostname

MASTER_HOSTNAME=mdw

指定 Master 的數據目錄

MASTER_DIRECTORY=/home/gpadmin/gpdata/gpmaster

Master 的端口

MASTER_PORT=6500

指定Bash的版本

TRUSTED_SHELL=ssh

Mirror Segment起始的端口號

CHECK_POINT_SEGMENTS=256
MIRROR_PORT_BASE=43000

Primary Segment 主備同步的起始端口號

REPLICATION_PORT_BASE=34000

Mirror Segment 主備同步的起始端口號

MIRROR_REPLICATION_PORT_BASE=44000

Mirror Segment 的數據目錄

declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatam1 /home/gpadmin/gpdata/gpdatam2)

初始化數據庫
gpinitsystem -c gpinitsystem_config -h hostlist -s sdw3 -S
//如無報錯就是初始化完成，如果有報錯，就需要根據日志提示修改配置，并重新初始化

[gpadmin@mdw conf]$ psql
psql (8.2.15)
Type "help" for help.

postgres=# select version();
version

PostgreSQL 8.2.15 (Greenplum Database 4.3.10.0 build commit: f413ff3b006655f14b6b9aa217495ec94da5c96c) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compi
led on Oct 21 2016 19:36:26
(1 row)

錯誤截圖，日志和截圖
節點有部分起來，一部分沒有起來。主機資源不夠。

2019-10-28 21:20:25.520556 CST,,,p6788,th-1469757664,,,,0,,,seg-1,,,,,"FATAL","XX000","could not create semaphores: No space left on device (pg_sema.c:132)","Failed system call was semget(43001004, 17, 03600).","This error does not mean that you have run out of disk space.
It occurs when either the system limit for the maximum number of semaphore sets (SEMMNI), or the system wide maximum number of semaphores (SEMMNS), would be exceeded. You need to raise the respective kernel parameter. Alternatively, reduce PostgreSQL's consumption of semaphores by reducing its max_connections parameter (currently 750).
The PostgreSQL documentation contains more information about configuring your system for PostgreSQL.",,,,,,"InternalIpcSemaphoreCreate","pg_sema.c",132,1 0xb0a80e postgres errstart + 0x4de

調整配置文件
vim /home/gpadmin/gpdata/gpdatam1/gpseg6/postgresql.conf
//每臺機器4個segments 都要修改
max_connections = 200 //原值750 ，調整到200
shared_buffers = 500MB //原值275 調整到500
gpstart -a 重啟啟動集群
gpstop -u 重新加載配置文件
#常用操作命令
gpstate:查看數據庫集群狀態，示例:gpstate -a
gpstart:啟動數據庫集群,示例:gpstart -a
gpstop:關閉數據庫集群,示例:gpstop -a -M fast
gpssh:遠程執行shell命令，示例:gpssh -f hosts -e 'date'

部署參考博文
https://www.cnblogs.com/dap570/archive/2015/03/21/greenplum_4node_install.html
https://my.oschina.net/u/876354/blog/1606419