您好,登錄后才能下訂單哦!
一、MySQL 5.6 以后出現的GTID:
GTID概念:
1.GTID是一個由服務器的UUID和事務序號組成的唯一事務序號
例如: UUID:N
1122-3322-1122:1
1122-3322-1122:2
2.GTID會被當做唯每一個事務的首部,將會自動生成并存到二進制日志中
3.GTID可以用來追蹤主從之間的事務傳輸。
4.GTID主要應用于HA的功能。在多主模型中,標示某一個事務是來源于哪個特定的主服務器。
5.從服務器不會修改或者添加新的GTID,即便從服務器被配置為其他從服務器的主服務器,所以可以追蹤事務流。
6.GTID開啟后,會在gtid_executed予以顯示
跟復制相關的工具,需要python2.7環境
1.mysqlreplicate: 添加從節點
2.mysqlcheck: 實現校驗機制
3.mysqlrplshow: 發現并顯示,復制拓撲結構, 幾級復制每一級有多少個服務器
4.mysqlfailover: 講一個從節點提升為一個主節點
5.mysqlrpladmin: 管理工具,做手工調度的,把一個正常從節點調度為主節點
借助于GTID的多線程復制
從服務器可以發出多個1/O線程對主服務器進行對mysqldump請求會使得先后讀出來的信息順序混亂。因此通過分割數據庫來達到多線程,每一個數據庫的事務只能有一個線程復制。 但即便如此如果只有一個主服務器,多線程并不能帶來性能提升, 因為只有一個二進制服務器,并且網絡帶寬也有限。真正使性能提升,需要一從多主模型。多線程slave通常為多個sql,一個I/O線程,多個SQL線程。 通過GTID機制,可以把不同的事務通過不同線程來應用。
配置開啟GTID開啟復制的需需要的參數(配置在[mysqld]段下)
1.binlog-format:二進制日志的格式,有row、statement和mixed幾種類型;
需要注意的是:當設置隔離級別為READ-COMMITED必須設置二進制日志格式為ROW,現在MySQL官方認為STATEMENT這個已經不再適合繼續使用;但mixed類型在默認的事務隔離級別下,可能會導致主從數據不一致;
2.log-slave-updates、gtid-mode、enforce-gtid-consistency、report-port和report-host:用于啟動GTID及滿足附屬的其它需求;
master-info-repository和relay-log-info-repository:啟用此兩項,可用于實現在崩潰時保證二進制及從服務器安全的功能;
3.sync-master-info:啟用之可確保無信息丟失;
4.slave-paralles-workers:設定從服務器的SQL線程數;0表示關閉多線程復制功能;
5.binlog-checksum、master-verify-checksum和slave-sql-verify-checksum:啟用復制有關的所有校驗功能;
6.binlog-rows-query-log-events:啟用之可用于在二進制日志記錄事件相關的信息,可降低故障排除的復雜度;
7.log-bin:啟用二進制日志,這是保證復制功能的基本前提;
8.server-id:同一個復制拓撲中的所有服務器的id號必須惟一;
9.report-host: 需要從服務器的主機名和IP地址在從服務器注冊的時候是否報告給主服務器。 在主服務器上使用SHOW SLAVE HOSTS可以查看
The host name or IP address of the slave to be reported to the master during slave registration. This value appears in the output of SHOW SLAVE HOSTS on the master server.
10.report-port:是否報告從服務器鏈接端口給主服務器
The TCP/IP port number for connecting to the slave, to be reported to the master during slave registration.
11.master-info-repository: 從服務是否把從服務器登錄和連接信息記錄在文件master.info或者記錄在mysql.slave_master_info表中
The setting of this variable determines whether the slave logs master status and connection information to a FILE (master.info), or to a TABLE (mysql.slave_master_info)
12.relay-log-info-repository: relay log相關數據記錄文檔或者表
This option causes the server to log its relay log info to a file or a table.
13.log_slave_updates:是否接受從服務器的更新信息
Whether updates received by a slave server
使用mariadb 10 實現多線程復制
準備環境:
1.系統環境:Centos6.5
2.數據庫版本:10.0.10-MariaDB-log MariaDB Server
3.Host:
Master主機: master.samlee.com 172.16.100.7
Slave主機: slave.samlee.com 172.16.100.8
----------------------------------------------------------------------------------------------------------
實現過程如下:
1.在Master安裝MariaDB及初始化數據庫服務,修改主機名,定義host文件:
--定義主機名 # vim /etc/sysconfig/network NETWORKING=yes HOSTNAME=master.samlee.com --定義host文件,實現主機名解析 # vim /etc/hosts 172.16.100.7 master.samlee.com master 172.16.100.8 slave.samlee.com slave --安裝MariaDB及初始化數據庫服務 --規劃數據庫存儲目錄 # fdisk /dev/sda --新增一個分區大小為:20G 分區類型為:LVM # kpartx /dev/sda # partx -a /dev/sda # kpartx -af /dev/sda # pvcreate /dev/sda3 # vgcreate myvg /dev/sda3 # lvcreate -L 10G -n mydata myvg # mke2fs -t ext4 /dev/myvg/mydata # mkdir /mydata # vim /etc/fstab #在最后加入此行 /dev/myvg/mydata /mydata ext4 defaults,acl 0 0 # mount -a # mkdir /mydata/data # groupadd -r -g 3306 mysql #創建mysql服務用戶組 # useradd -r -g 3306 -d /mydata/data/ -s /sbin/nologin mysql #創建mysql服務用戶 # chown -R mysql.mysql /mydata/data/ # setfacl -m u:mysql:rwx /mydata/data/ --安裝MariaDB及配置初始化 # cd /root/ # tar -xf mariadb-10.0.10-linux-x86_64.tar.gz -C /usr/local/ # cd /usr/local/ # ln -sv mariadb-10.0.10-linux-x86_64 mysql # mkdir /mydata/{binlogs,relaylogs} -pv # chown -R mysql.mysql /mydata/* # mkdir /etc/mysql # cp /usr/local/mysql/support-files/my-large.cnf /etc/mysql/my.cnf 修改/etc/mysql/my.cnf # vim /etc/mysql/my.cnf --在[mysqld]段加入以下選項 datadir=/mydata/data # chown -R mysql.mysql /etc/mysql/* # echo "export PATH=$PATH:/usr/local/mysql/bin" > /etc/profile.d/mysql.sh # source /etc/profile.d/mysql.sh # mkdir /var/lib/mysql # chown -R mysql.mysql /var/lib/mysql/ # cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld # chmod +x /etc/init.d/mysqld # scripts/mysql_install_db --user=mysql --datadir=/mydata/data/ # service mysqld start
2.在Slave安裝MariaDB及初始化數據庫服務,修改主機名,定義host文件:
--定義主機名 # vim /etc/sysconfig/network NETWORKING=yes HOSTNAME=slave.samlee.com --定義host文件,實現主機名解析 # vim /etc/hosts 172.16.100.7 master.samlee.com master 172.16.100.8 slave.samlee.com slave --安裝MariaDB及初始化數據庫服務 --規劃數據庫存儲目錄 # fdisk /dev/sda --新增一個分區大小為:20G 分區類型為:LVM # kpartx /dev/sda # partx -a /dev/sda # kpartx -af /dev/sda # pvcreate /dev/sda3 # vgcreate myvg /dev/sda3 # lvcreate -L 10G -n mydata myvg # mke2fs -t ext4 /dev/myvg/mydata # mkdir /mydata # vim /etc/fstab #在最后加入此行 /dev/myvg/mydata /mydata ext4 defaults,acl 0 0 # mount -a # mkdir /mydata/data # groupadd -r -g 3306 mysql #創建mysql服務用戶組 # useradd -r -g 3306 -d /mydata/data/ -s /sbin/nologin mysql #創建mysql服務用戶 # chown mysql.mysql /mydata/data/ # setfacl -m u:mysql:rwx /mydata/data/ --安裝MariaDB及配置初始化 # cd /root/ # tar -xf mariadb-10.0.10-linux-x86_64.tar.gz -C /usr/local/ # cd /usr/local/ # ln -sv mariadb-10.0.10-linux-x86_64 mysql # mkdir /mydata/{binlogs,relaylogs} -pv # chown -R mysql.mysql /mydata/* # mkdir /etc/mysql # cp /usr/local/mysql/support-files/my-large.cnf /etc/mysql/my.cnf 修改/etc/mysql/my.cnf # vim /etc/mysql/my.cnf --在[mysqld]段加入以下選項 datadir=/mydata/data # chown -R mysql.mysql /etc/mysql/* # echo "export PATH=$PATH:/usr/local/mysql/bin" > /etc/profile.d/mysql.sh # source /etc/profile.d/mysql.sh # mkdir /var/lib/mysql # chown -R mysql.mysql /var/lib/mysql/ # cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld # chmod +x /etc/init.d/mysqld # scripts/mysql_install_db --user=mysql --datadir=/mydata/data/ # service mysqld start
3.配置Master服務器為GTID主服務配置文件(主服務器上操作配置)
# vim /etc/mysql/my.cnf [mysqld] port = 3306 socket = /tmp/mysql.sock skip-external-locking key_buffer_size = 256M max_allowed_packet = 1M table_open_cache = 256 sort_buffer_size = 1M read_buffer_size = 1M read_rnd_buffer_size = 4M myisam_sort_buffer_size = 64M thread_cache_size = 8 query_cache_size= 16M thread_concurrency = 8 datadir=/mydata/data log-bin=/mydata/binlogs/master-bin binlog_format=row server-id = 1 log-slave-updates = True master-info-repository=TABLE relay-log-info-repository=TABLE sync-master-info = 1 slave-parallel-workers =2 binlog-checksum = CRC32 master-verify-checksum = 1 slave-sql-verify-checksum = 1 binlog-rows-query-log-events = 1 report-port = 3306 report-host = master.samlee.com
4.測試Master-GTID服務是否配置成功:(主服務器上操作配置)
MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE '%gtid%'; +------------------------+-------+ | Variable_name | Value | +------------------------+-------+ | gtid_binlog_pos | | | gtid_binlog_state | | | gtid_current_pos | | | gtid_domain_id | 0 | | gtid_ignore_duplicates | OFF | | gtid_slave_pos | | | gtid_strict_mode | OFF | +------------------------+-------+ MariaDB [(none)]> CREATE DATABASE mydb; MariaDB [(none)]> CREATE TABLE mydb.t1(Name CHAR(30)); MariaDB [(none)]> SHOW BINARY LOGS; +-------------------+-----------+ | Log_name | File_size | +-------------------+-----------+ | master-bin.000001 | 344 | | master-bin.000002 | 590 | +-------------------+-----------+ MariaDB [(none)]> SHOW BINLOG EVENTS IN 'master-bin.000002'; +-------------------+-----+-------------------+-----------+-------------+------------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +-------------------+-----+-------------------+-----------+-------------+------------------------------------------------+ | master-bin.000002 | 4 | Format_desc | 1 | 248 | Server ver: 10.0.10-MariaDB-log, Binlog ver: 4 | | master-bin.000002 | 248 | Gtid_list | 1 | 277 | [] | | master-bin.000002 | 277 | Binlog_checkpoint | 1 | 321 | master-bin.000002 | | master-bin.000002 | 321 | Gtid | 1 | 363 | GTID 0-1-1 | | master-bin.000002 | 363 | Query | 1 | 450 | CREATE DATABASE mydb | | master-bin.000002 | 450 | Gtid | 1 | 492 | GTID 0-1-2 | | master-bin.000002 | 492 | Query | 1 | 590 | CREATE TABLE mydb.t1(Name CHAR(30)) | +-------------------+-----+-------------------+-----------+-------------+------------------------------------------------ --現在我們可以看到GTID已經開始記錄了。
5.配置Slave服務器為GTID從服務配置文件(從服務器上操作配置)
[mysqld] port = 3306 socket = /tmp/mysql.sock skip-external-locking key_buffer_size = 256M max_allowed_packet = 1M table_open_cache = 256 sort_buffer_size = 1M read_buffer_size = 1M read_rnd_buffer_size = 4M myisam_sort_buffer_size = 64M thread_cache_size = 8 query_cache_size= 16M thread_concurrency = 8 datadir=/mydata/data log-bin=/mydata/data/master-bin binlog_format=ROW server-id = 200 log-slave-updates=true master-info-repository=TABLE relay-log-info-repository=TABLE sync-master-info=1 slave-parallel-threads=2 binlog-checksum=CRC32 master-verify-checksum=1 slave-sql-verify-checksum=1 binlog-rows-query-log_events=1 report-port=3306 report-host=slave.samlee.com
6.在Master服務器上創建復制用戶(主服務器上操作配置)
MariaDB [(none)]> GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'repluser'@'172.16.%.%' IDENTIFIED BY 'replpass'; MariaDB [(none)]> FLUSH PRIVILEGES; 注意:172.16.%.%是從節點服務器;如果想一次性授權更多的節點,可以自行根據需要修改;
7.為備節點提供初始數據集 (主服務器上操作配置)
鎖定主表,備份主節點上的數據,將其還原至從節點;如果沒有啟用GTID,在備份時需要在master上使用show master status命令查看二進制日志文件名稱及事件位置,以便后面啟動slave節點時使用。
# mysqldump --all-databases --lock-all-tables --flush-logs --master-data=2 > all.sql # scp all.sql 172.16.100.8:/tmp/
8.為備節點恢復初始數據集,連接主節點服務器(從服務器上操作配置)
# mysql < /tmp/all.sql MariaDB [(none)]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | mydb | | mysql | | performance_schema | | test | +--------------------+ --查詢主節點備份的二進制日志名稱及事件位置 # head -n30 /tmp/all.sql -- CHANGE MASTER TO MASTER_LOG_FILE='master-bin.000003', MASTER_LOG_POS=379; --連接主節點服務器 MariaDB [(none)]> CHANGE MASTER TO MASTER_HOST='172.16.100.7',MASTER_USER='repluser',MASTER_PASSWORD='replpass',MASTER_LOG_FILE='master-bin.000003',MASTER_LOG_POS=379; MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.16.100.7 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000003 Read_Master_Log_Pos: 379 Relay_Log_File: slave-relay-bin.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000003 Slave_IO_Running: No Slave_SQL_Running: No --以上我們可以看到我們所指定的信息 -------------------------------------------------------------------------------------- --啟動復制服務 MariaDB [(none)]> START SLAVE; MariaDB [(none)]> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.16.100.7 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000003 Read_Master_Log_Pos: 379 Relay_Log_File: slave-relay-bin.000002 Relay_Log_Pos: 540 Relay_Master_Log_File: master-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes ----------------------------------------------------------------------------------------- --查看SQL線程信息 MariaDB [(none)]> SHOW PROCESSLIST; +----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+----------+ | 5 | root | localhost | NULL | Query | 0 | init | SHOW PROCESSLIST | 0.000 | | 6 | system user | | NULL | Connect | 214 | Waiting for master to send event | NULL | 0.000 | | 7 | system user | | NULL | Connect | 213 | Slave has read all relay log; waiting for the slave I/O thread to update it | NULL | 0.000 | +----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+----------+
9.驗證GTIP復制狀況信息(主服務器與從服務器上操作配置)
--查詢連接上從節點服務器數 MariaDB [(none)]> SHOW SLAVE HOSTS; +-----------+------------------+------+-----------+ | Server_id | Host | Port | Master_id | +-----------+------------------+------+-----------+ | 200 | slave.samlee.com | 3306 | 1 | +-----------+------------------+------+-----------+ --怎么驗證從服務器啟動多個線程呢? (1)進行大批量寫入操作 # mysql < hellodb.sql (2)執行(1)操作后馬上在Slave服務器進行監控 # watch -n .5 "mysql -e 'show processlist\G'"
10.查詢GTID狀態信息及應用調試(從服務器上操作配置)
MariaDB [(none)]> SHOW SLAVE STATUS\G Using_Gtid: No --查詢并行復制機制是否啟動 MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE '%parallel%'; +-------------------------------+--------+ | Variable_name | Value | +-------------------------------+--------+ | slave_domain_parallel_threads | 0 | | slave_parallel_max_queued | 131072 | | slave_parallel_threads | 2 | +-------------------------------+--------+ --查詢sql線程 MariaDB [(none)]> SHOW PROCESSLIST; +----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+----------+ | 3 | system user | | NULL | Connect | 174 | Waiting for work from SQL thread | NULL | 0.000 | | 4 | system user | | NULL | Connect | 174 | Waiting for work from SQL thread | NULL | 0.000 | | 6 | system user | | NULL | Connect | 173 | Slave has read all relay log; waiting for the slave I/O thread to update it | NULL | 0.000 | | 7 | system user | | NULL | Connect | 173 | Waiting for master to send event | NULL | 0.000 | | 28 | root | localhost | NULL | Query | 0 | init | SHOW PROCESSLIST | 0.000 | +----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+----------+ --出現以上內容,我們已經實現了多線程復制功能了
11.使用GTID連接主節點服務器實現主從復制(從服務器上操作配置)
MariaDB [(none)]> STOP SLAVE; MariaDB [(none)]> CHANGE MASTER TO MASTER_HOST='172.16.100.7',MASTER_USER='repluser',MASTER_PASSWORD='replpass',MASTER_USE_GTID=current_pos; MariaDB [(none)]> START SLAVE; MariaDB [(none)]> SHOW SLAVE STATUS\G Using_Gtid: Current_Pos Gtid_IO_Pos: 0-1-39 MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE '%gtid%'; +------------------------+------------------+ | Variable_name | Value | +------------------------+------------------+ | gtid_binlog_pos | 0-1-39 | | gtid_binlog_state | 0-200-128,0-1-39 | | gtid_current_pos | 0-1-39 | | gtid_domain_id | 0 | | gtid_ignore_duplicates | OFF | | gtid_slave_pos | 0-1-39 | | gtid_strict_mode | OFF | +------------------------+------------------+
使用mariadb 10 實現多源復制架構
準備環境:
1.系統環境:Centos6.5
2.數據庫版本:10.0.10-MariaDB-log MariaDB Server
3.Host:
Master1主機: master.samlee.com 172.16.100.7
Master2主機: master1.samlee.com 172.16.100.10
Slave主機: slave.samlee.com 172.16.100.8
實現多分支機構數據庫匯總架構
----------------------------------------------------------------------
(1).Master1主機\Master2主機\Slave主機--定義hosts主機名解析文件,如下所示:
# vim /etc/hosts 172.16.100.7 master.samlee.com master 172.16.100.10 master2.samlee.com master2 172.16.100.8 slave.samlee.com slave
(2).Master1主機\Master2主機配置文件如下:
Master1主機: # vim /etc/mysql/my.cnf server-id = 100 log-bin=mysql-bin Master2主機: # vim /etc/mysql/my.cnf server-id = 200 log-bin=mysql-bin
(3)Slave主機配置文件如下:
# vim /etc/mysql/my.cnf server-id = 300 relay-log=relay-bin
(4)Master1主機\Master2主機上創建復制用戶(主服務器上操作配置)
MariaDB [(none)]> GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'repluser'@'172.16.%.%' IDENTIFIED BY 'replpass'; MariaDB [(none)]> FLUSH PRIVILEGES;
(5)查詢Master1主機\Master2主機二進制文件及事件位置
Master1主機: MariaDB [(none)]> SHOW MASTER STATUS; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql-bin.000006 | 867 | | | +------------------+----------+--------------+------------------+ Master2主機: MariaDB [(none)]> SHOW MASTER STATUS; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql-bin.000006 | 867 | | | +------------------+----------+--------------+------------------+
(6)在Slave主機上連接Master1、Master2主節點
MariaDB [(none)]> CHANGE MASTER 'master' TO MASTER_HOST='172.16.100.7',MASTER_PORT=3306,MASTER_USER='repluser',MASTER_PASSWORD='replpass',MASTER_LOG_FILE='mysql-bin.000006',MASTER_LOG_POS=867; MariaDB [(none)]> CHANGE MASTER 'master1' TO MASTER_HOST='172.16.100.10',MASTER_PORT=3306,MASTER_USER='repluser',MASTER_PASSWORD='replpass',MASTER_LOG_FILE='mysql-bin.000006',MASTER_LOG_POS=867; MariaDB [(none)]> START SLAVE 'master'; MariaDB [(none)]> START SLAVE 'master1'; MariaDB [(none)]> SHOW ALL SLAVES STATUS\G *************************** 1. row *************************** Connection_name: master Slave_SQL_State: Slave has read all relay log; waiting for the slave I/O thread to update it Slave_IO_State: Waiting for master to send event Master_Host: 172.16.100.7 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000006 Read_Master_Log_Pos: 867 Relay_Log_File: relay-bin-master.000002 Relay_Log_Pos: 535 Relay_Master_Log_File: mysql-bin.000006 Slave_IO_Running: Yes Slave_SQL_Running: Yes *************************** 2. row *************************** Connection_name: master1 Slave_SQL_State: Slave has read all relay log; waiting for the slave I/O thread to update it Slave_IO_State: Waiting for master to send event Master_Host: 172.16.100.10 Master_User: repluser Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000006 Read_Master_Log_Pos: 867 Relay_Log_File: relay-bin-master1.000002 Relay_Log_Pos: 535 Relay_Master_Log_File: mysql-bin.000006 Slave_IO_Running: Yes Slave_SQL_Running: Yes MariaDB [(none)]> SHOW SLAVE 'master' STATUS\G
測試如下:
master操作: MariaDB [(none)]> CREATE DATABASE masterdb; master1操作: MariaDB [(none)]> CREATE DATABASE master1db; slave操作: MariaDB [(none)]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | master1db | | masterdb | | mysql | | performance_schema | | test | +--------------------+
經測試顯示已經完成多源復制架構。
總結
1) 和mysql 5.6 相比,mariadb不支持的參數:
gtid-mode=on
enforce-gtid-consistency=true
2)修改的參數:
slave-parallel-workers參數修改為slave-parallel-threads
3)連接至主服務使用的命令:
一個新的參數:MASTER_USER_GTID={current_pos|slave_pos|no}
這個參數在多主一從的試驗中,總是不成功
4)才配置從服務器的時候,最好使用replicate_ignore_db 來忽略掉一些系統庫。
免責聲明:本站發布的內容(圖片、視頻和文字)以原創、轉載和分享為主,文章觀點不代表本網站立場,如果涉及侵權請聯系站長郵箱:is@yisu.com進行舉報,并提供相關證據,一經查實,將立刻刪除涉嫌侵權內容。