MySQL遷移文件的小問題

ZABBIX-監控系統:
------------------------------------
報警內容: Free disk space is less than 20% on volume /
------------------------------------
報警級別: PROBLEM
------------------------------------
監控項目: Free disk space on / (percentage)：1.94 %
------------------------------------
報警時間:2016.02.23-11:01:07
好家伙，這個分區竟然是直接使用了根目錄的空間，所以空間更是緊張了。
這個時候就需要調整數據的目錄地址了。想想也就是調整datadir的地址即可。
首先是調整數據的目錄地址，修改/etc/my.cnf,然后停庫，因為空間的問題，最后沒有剩余空間了，結果從庫的應用直接hang住了，所以直接停庫的時候等了一些時間。
# /etc/init.d/mysql stop
Shutting down MySQL (Percona Server)....................... SUCCESS!
啟庫的時候報了下面的錯誤。
# /etc/init.d/mysql start
Starting MySQL (Percona Server)..... ERROR! The server quit without updating PID file (/U01/app/mysql_3306/mysql.pid).
經過一番排查，發現原來是文件的目錄權限的問題。
修復之后繼續啟動，還是同樣的報錯。
一時沒有思路，就測試了一下，把文件目錄改回了原來的路徑，修改/etc/my.cnf里面的路徑，再次啟庫，這個時候從庫開始接受應用日志，過期的binlog都做了一些刪除。和追庫追平之后，再次停庫就很快了。
# /etc/init.d/mysql stop
Shutting down MySQL (Percona Server)... SUCCESS!
但是遷移文件之后，修改/etc/my.cnf之后再次啟庫就還是同樣的問題了。
[root@shadoop app]# /etc/init.d/mysql start
Starting MySQL (Percona Server)..... ERROR! The server quit without updating PID file (/U01/app/mysql_3306/mysql.pid).
查看error.log發現了下面的這一段內容，和之前一樣，不過有了新的發現。
160223 11:59:38 mysqld_safe mysqld from pid file /U01/app/mysql_3306/mysql.pid ended
160223 11:59:56 mysqld_safe Starting mysqld daemon with databases from /U01/app/mysql_3306
2016-02-23 11:59:56 21600 [Note] Plugin 'FEDERATED' is disabled.
2016-02-23 11:59:56 21600 [Note] InnoDB: The InnoDB memory heap is disabled
2016-02-23 11:59:56 21600 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2016-02-23 11:59:56 21600 [Note] InnoDB: Compressed tables use zlib 1.2.3
2016-02-23 11:59:56 21600 [Note] InnoDB: Using Linux native AIO
2016-02-23 11:59:56 21600 [Note] InnoDB: Using CPU crc32 instructions
2016-02-23 11:59:56 21600 [Note] InnoDB: Initializing buffer pool, size = 4.0G
2016-02-23 11:59:57 21600 [Note] InnoDB: Completed initialization of buffer pool
2016-02-23 11:59:57 21600 [Note] InnoDB: Highest supported file format is Barracuda.
2016-02-23 11:59:57 21600 [Note] InnoDB: 128 rollback segment(s) are active.
2016-02-23 11:59:57 21600 [Note] InnoDB: Waiting for purge to start
2016-02-23 11:59:57 21600 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.14-rel62.0 started; log sequence number 278581
8494
2016-02-23 11:59:57 7ffa261f0700 InnoDB: Loading buffer pool(s) from .//ib_buffer_pool
^G/usr/sbin/mysqld: File '/home/mysql_3306/mysql-bin.000006' not found (Errcode: 2 - No such file or directory)
2016-02-23 11:59:57 21600 [ERROR] Failed to open log (file '/home/mysql_3306/mysql-bin.000006', errno 2)
2016-02-23 11:59:57 21600 [ERROR] Could not open log file
2016-02-23 11:59:57 21600 [ERROR] Can't init tc log
2016-02-23 11:59:57 21600 [ERROR] Aborting
就是mysql會嘗試去找一個binlog /home/mysql_3306/mysql-bin.000006
這部分的信息在哪里呢。
# less relay-index.index
/home/mysql_3306/mysql-relay.000008
/home/mysql_3306/mysql-relay.000009
/U01/app/mysql_3306/mysql-relay.000010
/U01/app/mysql_3306/mysql-relay.000011
帶著新鮮勁，手工修改了一下這個文件，看看能不能生效。
修改為：
# vi relay-index.index
/U01/app/mysql_3306/mysql-relay.000008
/U01/app/mysql_3306/mysql-relay.000009
/U01/app/mysql_3306/mysql-relay.000010
/U01/app/mysql_3306/mysql-relay.000011
然后嘗試change master讓它基于最新的時間點重新同步。
> change master to master_host='10.127.0.xx',master_port =3306,master_user='repl',master_password='slaveuser',master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.00 sec)
啟動slave的時候就報了下面的錯誤。
> start slave;
ERROR 1872 (HY000): Slave failed to initialize relay log info structure from the repository
重啟之后，繼續嘗試start slave,發現錯誤依舊。
這個時候的方法只有reset slave了。
> start slave;
ERROR 1872 (HY000): Slave failed to initialize relay log info structure from the repository
> reset slave;
Query OK, 0 rows affected (0.00 sec)
> change master to master_host='10.127.0.xx',master_port =3306,master_user='repl',master_password='slaveuser',master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.04 sec)
> start slave;
Query OK, 0 rows affected (0.01 sec)
再次查看slave已經和主庫的日志追平了。
> show slave status\G
***************************
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 200
                  Master_UUID: 170281bc-1957-11e4-ad6e-842b2b4841e9
             Master_Info_File: /U01/app/mysql_3306/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
           Master_Retry_Count: 86400
reset slave會使得slave忘記主從復制關系的位置信息。該語句會刪除master.info文件和relay-log.info 文件以及所有的relay log 文件并重新啟用一個新的relaylog文件。
使用reset slave之前必須使用stop slave 命令將復制進程停止，所有的relay log將被刪除不管他們是否被SQL thread進程完全應用。
不過如果延遲不大，這些都不是事。畢竟這個問題解決了總比隔三差五收到報警手工處理要好很多。