【故障处理】DBCA建库诡异问题处理--rac环境不能创建rac库

注意:文章内容太多,公众号有限制,小麦苗做了精简,全过程可以参考http://blog.itpub.net/26736162/viewspace-2121863/ 或 http://www.cnblogs.com/lhrbest/articles/5661762.html 或者去云盘下载pdf版本阅读,谢谢大家的理解。

 

 


 

导读和注意事项

各位技术爱好者,看完本文后,你可以掌握如下的技能,也可以学到一些其它你所不知道的知识,~O(∩_∩)O~:

① dbca静默创建rac库

② Inventory目录作用及其2种重建方法(重点)

③ rac环境dbca工具不能创建rac库的解决办法

④ dbca静默建库常见问题处理

⑤ 重建CRS集群环境执行root.sh脚本

 

 

1  故障分析及解决过程

 

1.1  故障环境介绍

 

 项目

source db

db 类型

RAC

db version

11.2.0.4

db 存储

ASM

OS版本及kernel版本

AIX 64位 6.1.0.0

 

 

1.2  故障发生现象及报错信息

小麦苗采用dbca -silent+nodeinifo创建rac库,原来用的好好的命令,结果在这个rac环境上创建出来的库成了单实例的,很是郁闷,,,,且看故障发生的现象:

[ZFZHLHRDB1:oracle]:/oracle>dbca -silent -createDatabase -templateName General_Purpose.dbc -responseFile NO_VALUE \

> -gdbname raclhr  -sid raclhr \

> -sysPassword oracle -systemPassword lhr \

> -datafileDestination 'DATA/' -recoveryAreaDestination 'DATA/' \

> -redoLogFileSize 50 \

> -storageType ASM -asmsnmpPassword lhr  -diskGroupName 'DATA' \

> -characterset ZHS16GBK -nationalCharacterSet AL16UTF16 \

> -sampleSchema true \

> -automaticMemoryManagement true -totalMemory 2048 \

> -databaseType OLTP  \

> -emConfiguration NONE \

> -nodeinfo ZFZHLHRDB1,ZFZHLHRDB2

Copying database files

1% complete

3% complete

10% complete

17% complete

24% complete

35% complete

Creating and starting Oracle instance

37% complete

42% complete

47% complete

52% complete

53% complete

56% complete

58% complete

Registering database with Oracle Restart

64% complete

Completing Database Creation

68% complete

71% complete

75% complete

85% complete

96% complete

100% complete

Look at the log file "/oracle/app/oracle/cfgtoollogs/dbca/raclhr/raclhr.log" for further details.

[ZFZHLHRDB1:oracle]:/oracle>more /oracle/app/oracle/cfgtoollogs/dbca/raclhr/raclhr.log

Copying database files

DBCA_PROGRESS : 1%

DBCA_PROGRESS : 3%

DBCA_PROGRESS : 10%

DBCA_PROGRESS : 17%

DBCA_PROGRESS : 24%

DBCA_PROGRESS : 31%

DBCA_PROGRESS : 35%

Creating and starting Oracle instance

DBCA_PROGRESS : 37%

DBCA_PROGRESS : 42%

DBCA_PROGRESS : 47%

DBCA_PROGRESS : 52%

DBCA_PROGRESS : 53%

DBCA_PROGRESS : 56%

DBCA_PROGRESS : 58%

Registering database with Oracle Restart

DBCA_PROGRESS : 64%

Completing Database Creation

DBCA_PROGRESS : 68%

DBCA_PROGRESS : 71%

DBCA_PROGRESS : 75%

DBCA_PROGRESS : 85%

DBCA_PROGRESS : 96%

DBCA_PROGRESS : 100%

Database creation complete. For details check the logfiles at:

 /oracle/app/oracle/cfgtoollogs/dbca/raclhr.

Database Information:

Global Database Name:raclhr

System Identifier(SID):raclhr

[ZFZHLHRDB1:oracle]:/oracle>

[ZFZHLHRDB1:oracle]:/oracle>ORACLE_SID=raclhr                                              

[ZFZHLHRDB1:oracle]:/oracle>sqlplus / as sysdba                                            

                                                                                           

SQL*Plus: Release 11.2.0.4.0 Production on Tue Jul 5 17:17:42 2016                         

                                                                                           

Copyright (c) 1982, 2013, Oracle.  All rights reserved.                                    

                                                                                           

                                                                                           

Connected to:                                                                              

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production               

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,      

Data Mining and Real Application Testing options                                           

                                                                                           

SYS@raclhr> show parameter cluster                                                         

                                                                                           

NAME                                 TYPE        VALUE                                     

------------------------------------ ----------- ------------------------------            

cluster_database                     boolean     FALSE                                     

cluster_database_instances           integer     1                                         

cluster_interconnects                string                                                

 

大家看到这里的安装百分比是1%->3%->10%-17%.....,而rac库创建的时候进度百分比应该是1%->3%->9%-15%.....,如下图所示:

Copying database files

1% complete

3% complete

9% complete

15% complete

21% complete

30% complete

Creating and starting Oracle instance

32% complete

36% complete

40% complete

44% complete

45% complete

48% complete

50% complete

Creating cluster database views

52% complete

70% complete

Completing Database Creation

73% complete

76% complete

85% complete

94% complete

100% complete

Look at the log file "/oracle/app/oracle/cfgtoollogs/dbca/raclhr/raclhr.log" for further details.

 

接下来小麦苗尝试了如下的几种方式:①dbca采用响应文件的形式,②dbca图形界面的方式,都是同样的情况。

 

我的环境dbca图形界面的第一张图:


而正常rac环境下dbca图形界面的第一步图应该是:


以上都是目前出现的问题,初步预估是哪个配置文件出错了,dbca不能判断集群环境。

 

1.3  故障分析及解决过程

 

起初小麦苗以为是集群出现了问题,在万般无奈的情况下执行root.sh重建了集群,按照以前的经验,这个终极大招执行后集群肯定可以恢复到正常的情况下,结果呢?结果还是令小麦苗很伤心,情况依旧没有解决。

这个终极大招已经使用过了,问题没有解决只能说明不是权限的问题,也不是集群的问题,而是DB环境的问题。

于是硬着头皮分析分析dbca的日志,dbca创建库的时候日志在:11g:$ORACLE_BASE/cfgtoollogs/dbca , 10g:$ORACLE_HOME/cfgtoollogs/dbca,发现可能是监听的问题,于是又重建了监听,还是不行,

有种深深的挫败感,于是解决同事碰到的那个DG问题,参考:http://blog.itpub.net/26736162/viewspace-2121688/。时间就这样过了2天,到了7月8号,手头又没什么事情了,想起来dbca这个问题,想来想去还是得从日志入手,老子一行一行的看,总能看出点东西来,但这次我比对的看,就是找一个dbca可以创建rac的环境,生成日志来分析差异,结果令我震惊了。

当小麦苗看到第5行的日志的时候,发现一个null的问题,Current Version From Inventory: null,截图如下:

错误环境截图:

而正常环境的rac是这个样子的:


可以看到正常环境下,可以获取到Homeinfo和Current Version From Inventory的值,当看到Inventory这个词的时候我就想我已经找到问题的答案了。应该是Inventory这个目录出现了问题,标准点的说法应该是/oracle/app/oraInventory/ContentsXML/inventory.xml这个文件的内容有问题了。Inventory目录存放的是Oracle软件安装的目录信息,Oracle升级也需要这个目录,执行opatch lspatches也需要这个目录。oraInventory目录的位置在/etc/oraInst.loc中记录,11g中我们可以通过$ORACLE_HOME/oui/bin/attachHome.sh来重建这个目录。

 

在其中的1个节点上分别用oracle和grid来执行:

[ZFLHRDB1:root]:/oracle/app/oraInventory>$ORACLE_HOME/oui/bin/attachHome.sh

 

The user is root. Oracle Universal Installer cannot continue installation if the user is root.

: A file or directory in the path name does not exist.

[ZFLHRDB1:root]:/oracle/app/oraInventory>su - grid  

[ZFLHRDB1:grid]:/home/grid>$ORACLE_HOME/oui/bin/attachHome.sh

Starting Oracle Universal Installer...

 

Checking swap space: must be greater than 500 MB.   Actual 8192 MB    Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /oracle/app/oraInventory

'AttachHome' was successful.

[ZFLHRDB1:grid]:/home/grid>cd /oracle/app/oraInventory/ContentsXML

[ZFLHRDB1:grid]:/oracle/app/oraInventory/ContentsXML>more inv*

<?xml version="1.0" standalone="yes" ?>

<!-- Copyright (c) 1999, 2013, Oracle and/or its affiliates.

All rights reserved. -->

<!-- Do not modify the contents of this file by hand. -->

<INVENTORY>

<VERSION_INFO>

   <SAVED_WITH>11.2.0.4.0</SAVED_WITH>

   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>

</VERSION_INFO>

<HOME_LIST>

<HOME NAME="Ora11g_gridinfrahome1" LOC="/oracle/app/11.2.0/grid" TYPE="O" IDX="1" CRS="true">

   <NODE_LIST>

      <NODE NAME="ZFLHRDB1"/>

      <NODE NAME="ZFLHRDB2"/>

   </NODE_LIST>

</HOME>

</HOME_LIST>

<COMPOSITEHOME_LIST>

</COMPOSITEHOME_LIST>

</INVENTORY>

[ZFLHRDB1:grid]:/oracle/app/oraInventory/ContentsXML>exit

[ZFLHRDB1:root]:/oracle/app/oraInventory>su - oracle

[ZFLHRDB1:oracle]:/oracle>$ORACLE_HOME/oui/bin/attachHome.sh

Starting Oracle Universal Installer...

 

Checking swap space: must be greater than 500 MB.   Actual 8192 MB    Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /oracle/app/oraInventory

'AttachHome' was successful.

[ZFLHRDB1:oracle]:/oracle>cd /oracle/app/oraInventory/ContentsXML

[ZFLHRDB1:oracle]:/oracle/app/oraInventory/ContentsXML>more inv*

<?xml version="1.0" standalone="yes" ?>

<!-- Copyright (c) 1999, 2013, Oracle and/or its affiliates.

All rights reserved. -->

<!-- Do not modify the contents of this file by hand. -->

<INVENTORY>

<VERSION_INFO>

   <SAVED_WITH>11.2.0.4.0</SAVED_WITH>

   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>

</VERSION_INFO>

<HOME_LIST>

<HOME NAME="Ora11g_gridinfrahome1" LOC="/oracle/app/11.2.0/grid" TYPE="O" IDX="1" CRS="true">

   <NODE_LIST>

      <NODE NAME="ZFLHRDB1"/>

      <NODE NAME="ZFLHRDB2"/>

   </NODE_LIST>

</HOME>

<HOME NAME="DbHome1" LOC="/oracle/app/oracle/product/11.2.0/db" TYPE="O" IDX="2">

   <NODE_LIST>

      <NODE NAME="ZFLHRDB1"/>

      <NODE NAME="ZFLHRDB2"/>

   </NODE_LIST>

</HOME>

</HOME_LIST>

<COMPOSITEHOME_LIST>

</COMPOSITEHOME_LIST>

</INVENTORY>

 

重建Inventory这个目录后重新执行建库脚本就正常了,唉,o(︶︿︶)o 唉,折腾了这么久原来是这个文件的问题,至于新环境为啥这个目录的配置文件不全,这个估计得追溯到rac的安装过程了,这个就不去纠结了。

 

  About Me

..........................................................................................................................................................................................................

本文作者:小麦苗,只专注于数据库的技术,更注重技术的运用

本文在ITpub(http://blog.itpub.net/26736162)和博客园(http://www.cnblogs.com/lhrbest)有同步更新

本文地址:http://blog.itpub.net/26736162/viewspace-2121863/

本文pdf版:http://yunpan.cn/cdEQedhCs2kFz (提取码:ed9b) 

小麦苗分享的其它资料:http://blog.itpub.net/26736162/viewspace-1624453/

联系我请加QQ好友(642808185),注明添加缘由

于 2016-07-05 16:00~ 2016-07-08 19:00 在中行完成

【版权所有,文章允许转载,但须以链接方式注明源地址,否则追究法律责任】

..........................................................................................................................................................................................................

拿起手机扫描下边的图片来关注小麦苗的微信公众号:xiaomaimiaolhr,学习最实用的数据库技术。


 


本文分享自微信公众号 - DB宝(lhrdba)。
如有侵权,请联系 support@oschina.cn 删除。
本文参与“OSC源创计划”,欢迎正在阅读的你也加入,一起分享。

上一篇:【DB笔试面试733】在Oracle中,RAC中REMOTE_LISTENER的作用是什么?


下一篇:【DB笔试面试738】在Oracle中,将RAC备份集恢复到单实例数据库的步骤有哪些?...