數據庫啟動報錯：ORA-00600 [kcratr_nab_less_than_odr]

最近由于機房停電事件，導致數據庫主機異常關閉，主機恢復正常后，啟動數據庫報錯：

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [8120], [21090], [21205], [], [], [], [], [], [], []

查看alert日志，錯誤如下：

...省略...

ALTER DATABASE OPEN  #在open階段報錯了

Beginning crash recovery of 1 threads  #在進行crash recovery

 parallel recovery started with 2 processes

Started redo scan

Completed redo scan

 read 2095 KB redo, 98 data blocks need recovery

Errors in file /opt/oracle/diag/rdbms/stgorcl/stgorcl/trace/stgorcl_ora_22775.trc  (incident=89339):

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [8120], [21090], [21205], [], [], [], [], [], [], []

Incident details in: /opt/oracle/diag/rdbms/stgorcl/stgorcl/incident/incdir_89339/stgorcl_ora_22775_i89339.trc

...省略...

聯系主機異常關閉事件，推測可能由于主機異常關閉引起數據庫文件被損壞，結合數據庫的報錯信息看，懷疑是controlfile損壞了。

通過 kcratr_nab_less_than_odr 關鍵字，在MOS找一篇與此實際相符的文檔Alter database open fails with ORA-00600 kcratr_nab_less_than_odr (文檔 ID 1296264.1)，對比文檔與/opt/oracle/diag/rdbms/stgorcl/stgorcl/incident/incdir_89339/stgorcl_ora_22775_i89339.trc 的 "Call Stack Trace"，基本確認該事件與文檔描述的一致。

----- Call Stack Trace -----

ksedst1 <- ksedst <- dbkedDefDump <- ksedmp <- dbgexPhaseII <- dbgexProcessError <- dbgePostErrorKGE  <- kgeasnmierr <- kcratr_odr_check  <- kcratr <- kctrec <- kcvcrv <- kcfopd <- adbdrv <- opiexe <- opiosq0 <- kpoal8 <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real <- ssthrdmain <- main <- start

該故障是由于停電故障導致控制文件邏輯corruption造成，文檔給出2個解決方案：

Option a: Do cancel based reocvery, and apply 'current online redolog' manually #做recover using backup controlfile

Option b: Recreate the controlfile using the Controlfile recreation script  #重建控制文件

方案a更簡單，采用方案a：

找到CURRENT redo,做recover需要用到這個文件。

sys@STGORCL> select a.member, a.group#, b.status from v$logfile a ,v$log b where a.group#=b.group#;

MEMBER                                       GROUP# STATUS

---------------------------------------- ---------- ------------------------------------------------

/opt/oracle/oradata/stgorcl/redo03.log            3 INACTIVE

/opt/oracle/oradata/stgorcl/redo02.log            2 CURRENT

/opt/oracle/oradata/stgorcl/redo01.log            1 INACTIVE

SQL> recover database using backup controlfile until cancel ;

ORA-00279: change 140911009 generated at 04/24/2016 16:12:46 needed for thread

1

ORA-00289: suggestion :

/opt/oracle/product/11.2.0/db_1/dbs/arch2_8120_857011473.dbf  #該redo實際未歸檔，對應的是/opt/oracle/oradata/stgorcl/redo02.log

ORA-00280: change 140911009 for thread 1 is in sequence #8120

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}

/opt/oracle/oradata/stgorcl/redo02.log

Log applied.

Media recovery complete.

最后使用resetlogs打開數據庫

SQL> alter database open resetlogs;

Database altered.

注：resetlogs后，需要重新備份，重新搭建DG