一.1 BLOG文档结构图
一.2 前言部分
一.2.1 导读和注意事项
各位技术爱好者，看完本文后，你可以掌握如下的技能，也可以学到一些其它你所不知道的知识，~O(∩_∩)O~：
① rac中OCR的简介及其作用
② rac中OCR的备份和恢复
③ rac的健忘与脑裂
④ grid用户下的日志文件：$ORACLE_HOME/log文件夹内容被删除导致集群不能启动如何恢复？(重点)
⑤ 如何修复11.2 Grid权限误操作【（How to check and fix file permissions on Grid Infrastructure environment (文档 ID 1931142.1)】
⑥ 如何修复ASM实例名和节点名不一致的情况【How to Change 11.2 ASM Configuration to Match ASM Instance Name to the Node Where It Runs? (example, +ASM2 on Node2, etc) (文档 ID 1419424.1)】
⑦ permission.pl脚本的使用
Tips：
① 本文在ITpub（http://blog.itpub.net/26736162）和博客园(http://www.cnblogs.com/lhrbest)有同步更新
② 文章中用到的所有代码，相关软件，相关资料请前往小麦苗的云盘下载（http://blog.itpub.net/26736162/viewspace-1624453/）
③ 若文章代码格式有错乱，推荐使用搜狗、360或QQ浏览器，也可以下载pdf格式的文档来查看，pdf文档下载地址：http://blog.itpub.net/26736162/viewspace-1624453/
④ 本篇BLOG中命令的输出部分需要特别关注的地方我都用灰色背景和粉红色字体来表示，比如下边的例子中，thread 1的最大归档日志号为33，thread 2的最大归档日志号为43是需要特别关注的地方；而命令一般使用黄色背景和红色字体标注；对代码或代码输出部分的注释一般采用蓝色字体表示。
List of Archived Logs in backup set 11
Thrd Seq Low SCN Low Time Next SCN Next Time
---- ------- ---------- ------------------- ---------- ---------
1 32 1621589 2015-05-29 11:09:52 1625242 2015-05-29 11:15:48
1 33 1625242 2015-05-29 11:15:48 1625293 2015-05-29 11:15:58
2 42 1613951 2015-05-29 10:41:18 1625245 2015-05-29 11:15:49
2 43 1625245 2015-05-29 11:15:49 1625253 2015-05-29 11:15:53
[ZHLHRDB1:root]:/>lsvg -o
T_XDESK_APP1_vg
rootvg
[ZHLHRDB1:root]:/>
00:27:22 SQL> alter tablespace idxtbs read write;
====》2097152*512/1024/1024/1024=1G
本文如有错误或不完善的地方请大家多多指正，ITPUB留言或QQ皆可，您的批评指正是我写作的最大动力。
一.2.2 相关参考文章链接
关于物理和逻辑备份比较好的一篇文章：https://gjilevski.com/2010/12/20/backup-and-restore-of-ocr-in-grid-infrastructure-11g-r2-11-2-2/
一.2.3 本文简介
写这篇blog的情况是这样的，rac启动有问题，但是$GRID_HOME/log下的文件太多了，我就把下边的内容清空了，结果是集群更加不能启动就连基本的日志也没有了，重启OS无效，最后在仔细的想了想查看了一下运行正常的库的$GRID_HOME/log下的文件结构，发现有的文件夹最后有个T或t的权限，由此联想到可能是权限的缘故导致的，于是上MOS搜了搜文章果然搜到几篇文章，11.2.0.3.6以上比较好解决，但是小于这个版本的就不好弄了，自己总结的方法是重新跑root.sh脚本，而跑完后集群中注册的很多资源不存在了，于是这个又涉及到OCR的备份与恢复。另外执行root.sh若是顺序不对，或者其他的原因可能导致asm实例号和主机号不一致，就是rac1上的实例名是+ASM2,这个虽然没有啥影响，但是看着着实不爽，因此更改它们直接的对应关系也比较重要，所以总体而言就引出了文章开头提出的6个问题。
一.3 相关知识点扫盲(摘自网络+个人总结)
Clusterware 在运行期间需要两个文件： OCR 和 Voting Disk. 这 2 个文件必须存放在共享存储上。 OCR 用于解决健忘问题， Voting Disk 用于解决脑裂问题。
一.3.1 OCR Disk
Oracle Clusterware把整个集群的配置信息放在共享存储上，这个存储就是OCR Disk. 在整个集群中，只有一个节点能对OCR Disk 进行读写操作，这个节点叫作Master Node，所有节点都会在内存中保留一份OCR的拷贝，同时有一个OCR Process 从这个内存中读取内容。 OCR 内容发生改变时，由Master Node的OCR Process负责同步到其他节点的OCR Process。
健忘问题是由于每个节点都有配置信息的拷贝，修改节点的配置信息不同步引起的。Oracle 采用的解决方法就是把这个配置文件放在共享的存储上， 这个文件就是 OCR Disk。OCR 中保存整个集群的配置信息，配置信息以"Key-Value" 的形式保存其中。 在 Oracle10g 以前， 这个文件叫作 Server Manageability Repository(SRVM). 在 Oracle 10g， 这部分内容被重新设计，并重名为 OCR.。在 Oracle Clusterware 安装的过程中， 安装程序会提示用户指定 OCR 位置。并且用户指定的这个位置会被记录在/etc/oracle/ocr.loc(Linux System、AIX) 或者/var/opt/oracle/ocr.loc(Solaris System)文件中。 而在 Oracle 9i RAC 中，对等的是 srvConfig.Loc文件。 Oracle Clusterware在启动时会根据这里面的内容从指定位置读入 OCR 内容。
[zfzhlhrdb3:root]:/>cd /etc/oracle
[zfzhlhrdb3:root]:/etc/oracle>ls -l
total 3160
drwxrwx--- 2 root dba 256 Dec 29 14:16 lastgasp
-rw-r--r-- 1 root dba 37 Dec 29 14:10 ocr.loc
-rw-r--r-- 1 root system 0 Dec 29 14:10 ocr.loc.orig
-rw-r--r-- 1 root dba 92 Dec 29 14:10 olr.loc
-rw-r--r-- 1 root system 0 Dec 29 14:10 olr.loc.orig
drwxrwxr-x 5 root dba 256 Dec 29 14:09 oprocd
drwxr-xr-x 3 root dba 256 Dec 29 14:09 scls_scr
-rws--x--- 1 root dba 1606037 Dec 29 14:09 setasmgid
[zfzhlhrdb3:root]:/etc/oracle>
[zfzhlhrdb3:root]:/etc/oracle>more /etc/oracle/ocr.loc
ocrconfig_loc=+DATA
local_only=FALSE
[zfzhlhrdb3:root]:/etc/oracle>
OCR key
整个 OCR 的信息是树形结构，有 3 个大分支。分别是 SYSTEM,DATABASE 和 CRS。每个分支下面又有许多小分支。这些记录的信息只能由 root 用户修改。
一.3.1.1 OCR包含的内容
OCR中通常包含下列内容：
v 节点成员信息
v 数据库实例，节点，以及其他的映射关系
v ASM
v 资源配置信息(vip,services等等)
v 服务特性(Service characteristics)
v Oracle集群中相关进程的信息
v CRS控制的第三方应用程序信息
[zfzhlhrdb1:root]:/>ocrdump -local -stdout -xml|more|grep -i \<name\>|sed -e 's/\<NAME\>//g' -e 's/\<\/NAME\>//g'|awk -F . '{print 1,1,2,$3}'|uniq
SYSTEM
SYSTEM crs
SYSTEM crs usersecurity
SYSTEM crs deny
SYSTEM crs user_default_dir
SYSTEM ORA_CRS_HOME
SYSTEM WALLET
SYSTEM GNS
SYSTEM version
SYSTEM version localhost
SYSTEM version activeversion
SYSTEM GPnP
SYSTEM GPnP profiles
SYSTEM css
SYSTEM css nodenum_hint
SYSTEM network
SYSTEM network haip
SYSTEM OHASD
SYSTEM OHASD DM
SYSTEM OHASD SERVERPOOLS
SYSTEM OHASD SERVERS
SYSTEM OHASD TYPES
SYSTEM OHASD RESOURCES
SYSTEM CRS
SYSTEM CRS JOIN_SIGNATURE
SYSTEM OLR
SYSTEM OLR MANUALBACKUP
SYSTEM OCR
SYSTEM OCR BACKUP
DATABASE
DATABASE NODEAPPS
DATABASE VIP_RANGE
DATABASE LOG
DATABASE ASM
DATABASE DATABASES
CRS
[zfzhlhrdb1:root]:/>ocrdump -stdout -xml|more|grep -i \<name\>|sed -e 's/\<NAME\>//g' -e 's/\<\/NAME\>//g'|awk -F . '{print 1,1,2,$3}'|uniq
SYSTEM
SYSTEM version
SYSTEM version activeversion
SYSTEM version hostnames
SYSTEM versionstring
SYSTEM WALLET
SYSTEM WALLET APPQOSADMIN
SYSTEM GNS
SYSTEM css
SYSTEM css interfaces
SYSTEM crs
SYSTEM crs versions
SYSTEM crs usersecurity
SYSTEM crs deny
SYSTEM crs user_default_dir
SYSTEM crs e2eport
SYSTEM crs uiport
SYSTEM crs 11
SYSTEM ACFS
SYSTEM ORA_CRS_HOME
SYSTEM evm
SYSTEM evm debug
SYSTEM evm cevmkey
SYSTEM evm rmport
SYSTEM evm cevmport
SYSTEM DIAG
SYSTEM DIAG status
SYSTEM local_only
SYSTEM WLM
SYSTEM GPnP
SYSTEM GPnP profiles
SYSTEM JAZNFILE
<name>jazn com</name>
<name>qosadmin</name>
<name>oc4jadmin</name>
<name>JtaAdmin</name>
<name>ascontrol_appadmin</name>
<name>oc4j-administrators</name>
<name>qosadmin</name>
<name>oc4jadmin</name>
<name>JtaAdmin
<name>qos_admin</name>
<name>qosadmin</name>
<name>oc4j-app-administrators</name>
<name>users</name>
<name>ascontrol_monitor</name>
<name>
<name>qosadmin</name>
<name>oc4jadmin</name>
<name>qos_admin</name>
<name>jazn com/oc4j-administrators</name>
<name>login</name>
<name>subject propagation</name>
<name>oracle security jazn
<name>jazn com/*</name>
<name>administration</name>
<name>jazn com</name>
<name>jazn com/ascontrol_admin</name>
<name>login</name>
<name>subject propagation</name>
<name>oracle security jazn
<name>oracl
<name>oracle security jazn
<name>jazn com/*</name>
<name>administration</name>
<name>jazn com</name>
<name>jazn com/oc4j-app-administrators</name>
<name>login</name>
<name>jazn com/users</name>
<name>login</name>
<name>oracle security jazn
<name>coreid password header</name>
<name>coreid resource operation</name>
<name>addAllRoles</name>
<name>coreid password attribute</name>
<name>coreid resource type</name>
<name>coreid name attribute</name>
<name>coreid resource name</name>
<name>core
<name>oracle security jazn
<name>addAllRoles</name>
<name>oracle security wss
<name>addAllRoles</name>
<name>oracle security jazn
<name>addAllRoles</name>
<name>oracle security jazn
<name>addAllRoles</name>
<name>oracle security jazn
<name>addAllRoles</name>
<name>oracle security wss
<name>addAllRoles</name>
<name>issuer name 1</name>
SYSTEM JAZNFILE STATE
SYSTEM CRSADMIN
SYSTEM CRSUSER
SYSTEM CRSD
SYSTEM CRSD DM
SYSTEM CRSD SERVERPOOLS
SYSTEM CRSD SERVERS
SYSTEM CRSD TYPES
SYSTEM CRSD RESOURCES
SYSTEM OCR
SYSTEM OCR BACKUP
DATABASE
DATABASE NODEAPPS
DATABASE NODEAPPS zfzhlhrdb1
DATABASE NODEAPPS zfzhlhrdb2
DATABASE VIP_RANGE
DATABASE LOG
DATABASE ASM
DATABASE ASM zfzhlhrdb1
DATABASE ASM zfzhlhrdb2
DATABASE DATABASES
CRS
CRS CUR
CRS HIS
CRS SEC
CRS STAGE
CRS STAGE node1
CRS STAGE node2
[zfzhlhrdb1:root]:/>
一.3.1.2 OCR存储内容的表现形式
1. 同样地与Windows注册表来类比，OCR其存储内容的表现形式与其相同，是采用键值对的方式来展现。
2. 整个OCR 的信息是树形结构，有3个大分支。分别是SYSTEM,DATABASE 和CRS。
3. 每个分支下面又有许多小分支。这些记录的信息只能由root用户修改。
4. 可以使用ocrdump命令将其内容全部导出或者按分支进行导出。
一.3.2 Voting Disk
Voting Disk 这个文件主要用于记录节点成员状态，在出现脑裂时，决定那个 Partion 获得控制权，其他的 Partion必须从集群中剔除。在安装 Clusterware 时也会提示指定这个位置。安装完成后可以通过如下命令来查看 Voting Disk 位置。
$crsctl query css votedisk
[zfzhlhrdb3:root]:/dev>crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 83cb4909d3254f4ebf1181b024aaf539 (/dev/rhdisk2) [DATA]
Located 1 voting disk(s).
[zfzhlhrdb3:root]:/dev>ls -l /dev/rhdisk*
crw------- 2 root system 19, 0 Dec 29 10:02 /dev/rhdisk0
crw------- 1 root system 19, 4 Dec 29 11:15 /dev/rhdisk1
crw-rw---- 1 grid asmadmin 19, 6 Jan 08 15:17 /dev/rhdisk2
crw-rw---- 1 root system 19, 3 Dec 29 11:15 /dev/rhdisk3
crw-rw---- 1 root system 19, 1 Dec 29 11:15 /dev/rhdisk4
crw------- 1 root system 19, 7 Dec 29 11:15 /dev/rhdisk5
crw------- 1 root system 19, 8 Dec 29 11:15 /dev/rhdisk6
crw------- 1 root system 19, 2 Dec 29 11:15 /dev/rhdisk7
crw------- 1 root system 19, 5 Dec 29 11:15 /dev/rhdisk8
[zfzhlhrdb3:root]:/dev>
一.3.3 健忘症(Amnesia)
集群环境配置文件不是集中存放的，而是每个节点都有一个本地副本，在集群正常运行时，用户可以在任何节点更改集群的配置，并且这种更改会自动同步到其他节点。有一种特殊情况： 节点 A 正常关闭， 在节点 B 上修改配置， 关闭结点 A，启动结点B。 这种情况下，修改的配置文件是丢失的， 就是所谓的健忘症。OCR 用于解决健忘问题。
健忘是由于某个节点更新了OCR中的内容，而集群中的另外一些节点此时处于关闭，维护或重启阶段，OCR Master进程来不及将其信息更新到这些异常节点缓存而导致的不一致。譬如，在A节点发出了添加ocr镜像的命令，在这个时候B节点处于重启阶段。重启后A已经更新完毕，而此时B并不知道已经为ocr增加了一个新的镜像磁盘，健忘由此而生。
如下例，节点bo2dbp添加了新的ocr之后，配置文件发生了变化，此时节点bo2dbs的ocr.loc会被更新，如果bo2dbs处于关闭或重启阶段，则该文件得不到该更新，此即位健忘一例。
一.3.4 脑裂(Split Brain)
在集群中，节点间通过某种机制(心跳)了解彼此的健康状态，以确保各节点协调工作。假设只有"心跳"出现问题， 各个节点还在正常运行， 这时，每个节点都认为其他的节点宕机了， 自己是整个集群环境中的"唯一建在者"，自己应该获得整个集群的"控制权"。 在集群环境中，存储设备都是共享的， 这就意味着数据灾难， 这种情况就是"脑裂"解决这个问题的通常办法是使用投票算法(Quorum Algorithm). 它的算法机理如下：集群中各个节点需要心跳机制来通报彼此的"健康状态"，假设每收到一个节点的"通报"代表一票。对于三个节点的集群，正常运行时，每个节点都会有 3 票。 当结点 A 心跳出现故障但节点 A 还在运行，这时整个集群就会分裂成 2 个小的 partition。 节点 A 是一个，剩下的 2 个是一个。 这是必须剔除一个 partition 才能保障集群的健康运行。对于有 3个节点的集群， A 心跳出现问题后， B 和 C 是一个 partion，有 2 票， A只有 1 票。 按照投票算法， B 和 C 组成的集群获得控制权， A 被剔除。如果只有 2 个节点，投票算法就失效了。 因为每个节点上都只有 1 票。 这时就需要引入第三个设备： Quorum Device. Quorum Device 通常采用饿是共享磁盘，这个磁盘也叫作Quorum disk。 这个 Quorum Disk 也代表一票。 当 2 个结点的心跳出现问题时， 2 个节点同时去争取 Quorum Disk 这一票， 最早到达的请求被最先满足。 故最先获得 Quorum Disk的节点就获得2 票。另一个节点就会被剔除。
一.3.5 OCR命令系列
一.3.5.1 ocrdump
该命令能以ASCII的方式打印出OCR的内容，但是这个命令不能用作OCR的备份恢复，也就是说产生的文件只能用作阅读，而不能用于恢复。
命令格式： ocrdump [-stdout] [filename] [-keyname name] [-xml]
参数说明：
-stdout: 把内容打印输出到屏幕上
Filename：内容输出到文件中
-keyname：只打印某个键及其子健内容
-xml：以xml格式打印输出
示例：把system.css键的内容以.xml格式打印输出到屏幕
[root@raw1 bin]# ./ocrdump -stdout -keyname system.css -xml|more
<OCRDUMP>
<TIMESTAMP>03/08/2010 04:28:41</TIMESTAMP>
<DEVICE>/dev/raw/raw1</DEVICE>
<COMMAND>./ocrdump.bin -stdout -keyname system.css -xml </COMMAND>
......
这个命令在执行过程中，会在$CRS_HOME/log/<node_name>/client 目录下产生日志文件，文件名ocrdump_<pid>.log,如果命令执行出现问题，可以从这个日志查看问题原因。
一.3.5.2 ocrcheck
Ocrcheck 命令用于检查OCR内容的一致性，命令执行过程会在$CRS_HOME/log/nodename/client 目录下产生ocrcheck_pid.log 日志文件。 这个命令不需要参数。
[zfzhlhrdb1:root]:/>ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3176
Available space (kbytes) : 258944
ID : 362503260
Device/File Name : +DATA
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
一.3.5.3 ocrconfig
该命令用于维护OCR 磁盘，安装clusterware过程中，如果选择External Redundancy冗余方式，则只能输入一个OCR磁盘位置。 但是Oracle允许配置两个OCR 磁盘互为镜像，以防止OCR 磁盘的单点故障。 OCR 磁盘和Votedisk磁盘不一样，OCR磁盘最多只能有两个，一个Primary OCR 和一个Mirror OCR。
[root@raw1 bin]# ./ocrconfig --help
Name:
ocrconfig - Configuration tool for Oracle Cluster Registry.
Synopsis:
ocrconfig [option]
option:
-export <filename> [-s online]
- Export cluster register contents to a file
-import <filename> - Import cluster registry contents from a file
-upgrade [<user> [<group>]]
- Upgrade cluster registry from previous version
-downgrade [-version <version string>]
- Downgrade cluster registry to the specified version
-backuploc <dirname> - Configure periodic backup location
-showbackup - Show backup information
-restore <filename> - Restore from physical backup
-replace ocr|ocrmirror [<filename>] - Add/replace/remove a OCR device/file
-overwrite - Overwrite OCR configuration on disk
-repair ocr|ocrmirror <filename> - Repair local OCR configuration
-help - Print out this help information
Note:
A log file will be created in
$ORACLE_HOME/log/<hostname>/client/ocrconfig_<pid>.log. Please ensure
you have file creation privileges in the above directory before
running this tool.
一.3.6 Oracle RAC OCR 的备份与恢复
Oracle Clusterware把整个集群的配置信息放在共享存储上，这些信息包括了集群节点的列表、集群数据库实例到节点的映射以及CRS应用程序资源信息。也即是存放在ocr 磁盘(或者ocfs文件)上。因此对于这个配置文件的重要性是不言而喻的。任意使得ocr配置发生变化的操作在操作之间或之后都建议立即备份ocr。
因为OCR的内容如此重要，Oracle 每4个小时对其做一次备份，并且保留最后的3个备份，以及前一天，前一周的最后一个备份。 这个备份由Master Node CRSD进程完成，备份的默认位置是$CRS_HOME/crs/cdata/<cluster_name>目录下。 每次备份后，备份文件名自动更改，以反应备份时间顺序，最近一次的备份叫作backup00.ocr。这些备份文件除了保存在本地，DBA还应该在其他存储设备上保留一份，以防止意外的存储故障。
与Oracle数据库备份恢复相似，OCR的备份也有物理备份或逻辑备份的概念，因此有两种备份方式，两种恢复方式。
常用命令：
crsctl query css votedisk
lquerypv -h /dev/rhdisk2
crsctl stop has -f
crsctl start has
crsctl stat res -t
一.3.6.1 dd备份恢复
备份表决磁盘：
dd if=/dev/raw/raw3 of=/tmp/votedisk_lhr.bak bs=1024k count=4
恢复表决磁盘：
dd if=/tmp/votedisk_lhr.bak of=/dev/raw/raw3 bs=1024k count=4
注：11g不推荐使用dd来进行备份恢复，盘头一般是前4K
一.3.6.2 kfed恢复磁盘头
dd if=/dev/rhdisk2 of=/asm_rhdisk2_dd.bak bs=1024 count=4
dd if=/dev/zero of=/dev/rhdisk2 bs=1024 count=4
kfed repair /dev/rhdisk2
关于kfed、kfod、amdu的更多内容可以参考：http://blog.itpub.net/26736162/viewspace-1694198/
http://blog.itpub.net/26736162/viewspace-1694199/
一.3.6.3 md_backup和md_restore恢复磁盘头
asmcmd md_backup /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_md_backup.bak
asmcmd md_restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_md_backup.bak
dd if=/dev/rhdisk2 of=/asm_rhdisk2_dd.bak bs=1024k count=4
dd if=/dev/zero of=/dev/rhdisk2 bs=1024k count=4
crsctl stop has -f
crsctl start has
ASMCMD [+] > startup force nomount;
ASMCMD [+] > md_restore /asm_rhdisk2_dd.bak
ASMCMD [+] > md_backup /rman/asm_md.bak
dd if=/dev/zero of=/dev/rhdisk2 bs=1024 count=4
crsctl stop has -f
crsctl start has
ASMCMD [+] > startup force nomount;
ASMCMD [+] > md_restore /rman/asm_md.bak
关于md_backup和md_restore更多内容可以参考：http://blog.itpub.net/26736162/viewspace-2121309/
一.3.6.4 物理备份与恢复(自动备份)
缺省情况下，Oracle 每4个小时对其做一次备份，并且保留最后的3个副本，以及前一天，前一周的最后一个备份副本。用户不能自定义备份频率以及备份文件的副本数。
对于OCR的备份由是由Master Node CRSD进程完成，因此备份的默认位置是$CRS_HOME/crs/cdata/<cluster_name>目录下。
备份的文件会自动更名，以反应备份时间顺序，最近一次的备份叫作backup00.ocr。
由于是在Master Node的节点之上进行备份，因此备份文件仅存在于Master Node节点。
对于Master Node的节点crash之后则由剩余节点接管。
备份目录可以通过ocrconfig -backuploc <directory_name> 命令修改。
OCR磁盘最多只能有两个，一个Primary OCR 和一个Mirror OCR。两者互为镜像以避免单点故障。
对于物理备份恢复，不能简单的使用操作系统级别的复制命令(使用ocr文件时)来完成，该操作将导致ocr不可用。
对于物理备份，仅仅只能使用restore方式来进行恢复，而不支持import方式
1，查看备份磁盘
[zfzhlhrdb2:grid]:/home/grid>ocrconfig -showbackup
zfzhlhrdb1 2016/06/30 15:13:46 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup00.ocr
zfzhlhrdb1 2016/06/30 11:13:45 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup01.ocr
zfzhlhrdb1 2016/06/30 07:13:45 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup02.ocr
zfzhlhrdb1 2016/06/29 03:13:41 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/day.ocr
zfzhlhrdb1 2016/06/20 03:13:08 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/week.ocr
PROT-25: Manual backups for the Oracle Cluster Registry are not available
[zfzhlhrdb2:grid]:/home/grid>oerr prot 25
00025, 0, "Manual backups for the Oracle Cluster Registry are not available"
// *Cause: Manual backups for the Oracle Cluster Registry were not yet created.
// *Action: Manual backups can be created using 'ocrconfig -manualbackup'
// command.
[zfzhlhrdb2:grid]:/home/grid>ocrconfig -manualbackup
PROT-20: Insufficient permission to proceed. Require privileged user
[zfzhlhrdb2:grid]:/home/grid>exit
[zfzhlhrdb2:root]:/>
[zfzhlhrdb2:root]:/>
[zfzhlhrdb2:root]:/>ocrconfig -manualbackup
zfzhlhrdb1 2016/06/30 16:21:34 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup_20160630_162134.ocr
[zfzhlhrdb2:root]:/>ocrconfig -showbackup
zfzhlhrdb1 2016/06/30 15:13:46 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup00.ocr
zfzhlhrdb1 2016/06/30 11:13:45 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup01.ocr
zfzhlhrdb1 2016/06/30 07:13:45 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup02.ocr
zfzhlhrdb1 2016/06/29 03:13:41 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/day.ocr
zfzhlhrdb1 2016/06/20 03:13:08 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/week.ocr
zfzhlhrdb1 2016/06/30 16:21:34 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup_20160630_162134.ocr
在节点一执行，可以看到2个节点得到的内容一致：
[zfzhlhrdb1:root]:/>ocrconfig -showbackup
zfzhlhrdb1 2016/06/30 15:13:46 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup00.ocr
zfzhlhrdb1 2016/06/30 11:13:45 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup01.ocr
zfzhlhrdb1 2016/06/30 07:13:45 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup02.ocr
zfzhlhrdb1 2016/06/29 03:13:41 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/day.ocr
zfzhlhrdb1 2016/06/20 03:13:08 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/week.ocr
zfzhlhrdb1 2016/06/30 16:21:34 /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs/backup_20160630_162134.ocr
[zfzhlhrdb1:root]:/>
2，恢复
ocrconfig -restore /app/crs/product/11.0.6/crs/cdata/racluster/backup01.ocr
3，查看配置
[grid@rac1 ~]$ more /etc/oracle/ocr.loc
ocrconfig_loc=+DATA
local_only=FALSE
[grid@rac1 ~]$
4，查看进程
[grid@rac1 ~]$ ps -ef | grep d.bin
root 4694 1 0 10:00 ? 00:00:13 /u01/grid/bin/ohasd.bin reboot
grid 4821 1 0 10:00 ? 00:00:28 /u01/grid/bin/oraagent.bin
root 4823 1 0 10:00 ? 00:00:04 /u01/grid/bin/orarootagent.bin
grid 4846 1 0 10:00 ? 00:00:00 /u01/grid/bin/gipcd.bin
grid 4859 1 0 10:00 ? 00:00:00 /u01/grid/bin/mdnsd.bin
grid 4874 1 0 10:00 ? 00:00:01 /u01/grid/bin/gpnpd.bin
root 8645 1 0 10:47 ? 00:00:04 /u01/grid/bin/cssdmonitor
root 8662 1 0 10:48 ? 00:00:05 /u01/grid/bin/cssdagent
grid 8664 1 0 10:48 ? 00:00:01 /u01/grid/bin/diskmon.bin -d -f
grid 8688 1 0 10:48 ? 00:00:40 /u01/grid/bin/ocssd.bin
root 8754 1 0 10:50 ? 00:00:01 /u01/grid/bin/octssd.bin
grid 8770 1 0 10:50 ? 00:00:02 /u01/grid/bin/evmd.bin
grid 8888 1 0 10:51 ? 00:00:00 /u01/grid/bin/oclskd.bin
root 8920 1 0 10:51 ? 00:00:07 /u01/grid/bin/crsd.bin reboot
root 8966 1 0 10:51 ? 00:00:00 /u01/grid/bin/oclskd.bin
grid 9013 8770 0 10:51 ? 00:00:00 /u01/grid/bin/evmlogger.bin -o /u01/grid/evm/log/evmlogger.info -l /u01/grid/evm/log/evmlogger.log
grid 9055 1 0 10:51 ? 00:00:06 /u01/grid/bin/oraagent.bin
root 9059 1 0 10:51 ? 00:00:42 /u01/grid/bin/orarootagent.bin
grid 9283 1 0 10:52 ? 00:00:00 /u01/grid/bin/tnslsnr LISTENER -inherit
oracle 9549 1 0 10:58 ? 00:00:26 /u01/grid/bin/oraagent.bin
oracle 9773 1 0 11:00 ? 00:00:00 /u01/grid/bin/oclskd.bin
grid 18618 1 0 13:46 ? 00:00:00 /u01/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid 22527 21370 0 14:58 pts/2 00:00:00 grep d.bin
[grid@rac1 ~]$
注：
ocssd:用于管理与协调集群中各个节点的关系，并用于节点通信。该进程非常的重要，如果这个进程异常中止，会导致系统自动重启。在某些极端的情况下，如果ocssd无法正常启动，就会导致操作系统循环重启。
crsd：监控节点各个资源，当某个资源发生异常是，自动重启或者切换该资源。
evmd：是一个基于后台的事件检测程序。
oclskd：该守护进程是Oracle 11g(11.10.6)新增的一个后台进程，主要是用于监控RAC数据库节点实例，当某个实例挂起时，就重启该节点。
一.3.6.5 逻辑备份与恢复(手动备份)
其实OCR也可以通过手动的方式导出、导入、方法如下：
ocrconfig -export /tmp/ocr_bak
ocrconfig -import /tmp/ocr_bak
使用ocrconfig -export 方式产生的备份，统称之为逻辑备份。
对于重大的ocr配置发生变化前后，如添加删除节点，修改集群资源，创建数据库等，都建议使用逻辑备份。
对于由于错误配置而导致的ocr被损坏的情形下，我们可以使用ocrconfig -import方式进行恢复。
对于这种逻辑方式也可以还原丢失或损坏的ocr磁盘(文件)。
一.3.7 如何修复11.2 Grid权限误操作
关于Oracle GRID HOME文件目录的权限问题
① chown -R 误操作了，将整个/u01/app的权限修过了如何恢复？
② 删除了$GRID_HOME/log文件夹下的所有内容，集群不能启动，如何恢复？
使用chown -R 误操作了，将整个/u01/app的权限修过了，导致grid无法启动。，搜了下mos发现了一篇文档：Tips for checking file permissions on GRID environment(ID 1931142.1)
该文档中描述到,$GRID_HOME/crs/utl下面的几个文件中记录了整个GRID_HOME下面的文件和目录的相关权限。
Check the permissions from the following 2 files which are created during Grid Infrastructure installation.
In $GRID_HOME/crs/utl (for 11.2 and 12.1.0.1) and <GRID_HOME>/crs/utl/<hostname> (for 12.1.0.2) directory:
crsconfig_dirs ：which has all directories listed in <GRID_HOME> and their permissions
crsconfig_fileperms ：which has list of files and their permissions and locations in <GRID_HOME>.
我们来看下是否是这样的,cd $ORACLE_HOME/crs/utl：
[root@rac2 bin]# cd /home/grid/app/11.2/grid/crs/utl
[root@rac2 utl]# ls -ltr
total 324
-rw-r–r– 1 root root 1128 Aug 11 09:48 usrvip
-rw-r–r– 1 root root 8437 Aug 11 09:48 srvctl
……
-rw-r–r– 1 root root 12102 Aug 11 09:48 crsconfig_files
-rw-r–r– 1 root root 13468 Aug 11 09:48 crsconfig_fileperms
-rw-r–r– 1 root root 8666 Aug 11 09:48crsconfig_dirs
-rw-r–r– 1 root root 699 Aug 11 09:48 crfsetenv
-rw-r–r– 1 root root 1280 Aug 11 09:48 cmdllroot.sh
-rw-r–r– 1 root root 3680 Aug 11 09:48 cluutil
-rw-r–r– 1 root root 1648 Aug 11 09:48 clsrwrap
-rw-r–r– 1 root root 540 Aug 11 09:48 appvipcfg
[zfzhlhrdb1:grid]:/oracle/app/11.2.0/grid/crs/utl>more crsconfig_dirs
# Copyright (c) 2009, 2013, Oracle and/or its affiliates. All rights reserved.
# The values in each line use the following format:
#
# OSLIST DIRNAME OWNER GROUP CLOSED-PERMS OPEN-PERMS
#
# Note:
# 1) OSLIST is a comma-separated list of platforms on which the directory
# needs to be created. 'all' indicates that the directory needs to be
# created on every platform. OSLIST MUST NOT contain whitespace.
# 2) Permissions need to be specified AS OCTAL NUMBERS. If permissions are
# not specified, default (umask) values will be used.
#
# TBD: OPEN-PERMS need to be added for each dir
all /oracle/app/11.2.0/grid/cdata grid dba 0775
all /oracle/app/11.2.0/grid/cdata/zfzhlhrdb-crs grid dba 0775
all /oracle/app/11.2.0/grid/cfgtoollogs grid dba 0775
all /oracle/app/11.2.0/grid/cfgtoollogs/crsconfig grid dba 0775
all /oracle/app/11.2.0/grid/log grid dba 0775
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1 root dba 01755
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/crsd root dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/ctssd root dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/evmd grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/cssd grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/mdnsd grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/gpnpd grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/gnsd root dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/srvm grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/gipcd grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/diskmon grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/cvu grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/cvu/cvulog grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/cvu/cvutrc grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/acfssec root dba 0755
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/acfsrepl grid dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/acfslog grid dba 0750
all /oracle/app/11.2.0/grid/cdata/localhost grid dba 0755
all /oracle/app/11.2.0/grid/cdata/zfzhlhrdb1 grid dba 0755
all /oracle/app/11.2.0/grid/cv grid dba 0775
all /oracle/app/11.2.0/grid/cv/log grid dba 0775
all /oracle/app/11.2.0/grid/cv/init grid dba 0775
all /oracle/app/11.2.0/grid/cv/report grid dba 0775
all /oracle/app/11.2.0/grid/cv/report/html grid dba 0775
all /oracle/app/11.2.0/grid/cv/report/text grid dba 0775
all /oracle/app/11.2.0/grid/cv/report/xml grid dba 0775
# These dirs must be owned by crsuser in SIHA, and $SUPERUSER in cluster env.
# 'HAS_USER' is set appropriately in roothas.pl and rootcrs.pl for this
# purpose
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/ohasd root dba 0750
all /oracle/app/11.2.0/grid/lib root dba 0755
all /oracle/app/11.2.0/grid/bin root dba 0755
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/agent root dba 01775
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/agent/crsd root dba 01777
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/agent/ohasd root dba 01775
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/client grid dba 01777
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/racg grid dba 01775
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/racg/racgmain grid dba 01777
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/racg/racgeut grid dba 01777
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/racg/racgevtf grid dba 01777
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/admin grid dba 0750
all /oracle/app/11.2.0/grid/log/diag/clients grid asmadmin 01770
all /oracle/app/11.2.0/grid/evm grid dba 0750
all /oracle/app/11.2.0/grid/evm/init grid dba 0750
all /oracle/app/11.2.0/grid/auth/evm/zfzhlhrdb1 root dba 01777
all /oracle/app/11.2.0/grid/evm/log grid dba 01770
all /oracle/app/11.2.0/grid/eons/init grid dba 0750
all /oracle/app/11.2.0/grid/auth/ohasd/zfzhlhrdb1 root dba 01777
all /oracle/app/11.2.0/grid/mdns grid dba 0750
all /oracle/app/11.2.0/grid/mdns/init grid dba 0750
all /oracle/app/11.2.0/grid/gipc grid dba 0750
all /oracle/app/11.2.0/grid/gipc/init grid dba 0750
all /oracle/app/11.2.0/grid/gnsd root dba 0750
all /oracle/app/11.2.0/grid/gnsd/init root dba 0750
all /oracle/app/11.2.0/grid/gpnp grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/init grid dba 0750
all /oracle/app/11.2.0/grid/ohasd grid dba 0750
all /oracle/app/11.2.0/grid/ohasd/init grid dba 0750
all /oracle/app/11.2.0/grid/gpnp grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/profiles grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/profiles/peer grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/wallets grid dba 01750
all /oracle/app/11.2.0/grid/gpnp/wallets/root grid dba 01700
all /oracle/app/11.2.0/grid/gpnp/wallets/prdr grid dba 01750
all /oracle/app/11.2.0/grid/gpnp/wallets/peer grid dba 01700
all /oracle/app/11.2.0/grid/gpnp/wallets/pa grid dba 01700
all /oracle/app/11.2.0/grid/mdns grid dba 0750
all /oracle/app/11.2.0/grid/gpnp grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/profiles grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/profiles/peer grid dba 0750
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/wallets grid dba 01750
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/wallets/root grid dba 01700
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/wallets/prdr grid dba 01750
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/wallets/peer grid dba 01700
all /oracle/app/11.2.0/grid/gpnp/zfzhlhrdb1/wallets/pa grid dba 01700
all /oracle/app/11.2.0/grid/css grid dba 0711
all /oracle/app/11.2.0/grid/css/init grid dba 0711
all /oracle/app/11.2.0/grid/css/log grid dba 0711
all /oracle/app/11.2.0/grid/auth/css/zfzhlhrdb1 root dba 01777
all /oracle/app/11.2.0/grid/crs root dba 0755
all /oracle/app/11.2.0/grid/crs/init root dba 0755
all /oracle/app/11.2.0/grid/crs/profile root dba 0755
all /oracle/app/11.2.0/grid/crs/script root dba 0755
all /oracle/app/11.2.0/grid/crs/template root dba 0755
all /oracle/app/11.2.0/grid/auth/crs/zfzhlhrdb1 root dba 01777
all /oracle/app/11.2.0/grid/crs/log grid dba 01750
all /oracle/app/11.2.0/grid/crs/trace grid dba 01750
all /oracle/app/11.2.0/grid/crs/public grid dba 01777
all /oracle/app/11.2.0/grid/ctss root dba 0755
all /oracle/app/11.2.0/grid/ctss/init root dba 0755
all /oracle/app/11.2.0/grid/racg/usrco grid dba
all /oracle/app/11.2.0/grid/racg/dump grid dba 0775
all /oracle/app/11.2.0/grid/srvm/admin grid dba 0775
all /oracle/app/11.2.0/grid/srvm/log grid dba 0775
all /oracle/app/11.2.0/grid/evm/admin/conf grid dba 0750
all /oracle/app/11.2.0/grid/evm/admin/logger grid dba 0750
all /oracle/app/11.2.0/grid/crf root dba 0750
all /oracle/app/11.2.0/grid/crf/admin root dba 0750
all /oracle/app/11.2.0/grid/crf/admin/run grid dba 0750
all /oracle/app/11.2.0/grid/crf/admin/run/crfmond root dba 0700
all /oracle/app/11.2.0/grid/crf/admin/run/crflogd root dba 0700
all /oracle/app/11.2.0/grid/crf/db root dba 0750
all /oracle/app/11.2.0/grid/crf/db/zfzhlhrdb1 root dba 0750
all /oracle/app/11.2.0/grid/osysmond root dba 0755
all /oracle/app/11.2.0/grid/osysmond/init root dba 0755
all /oracle/app/11.2.0/grid/ologgerd root dba 0755
all /oracle/app/11.2.0/grid/ologgerd/init root dba 0755
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/crfmond root dba 0750
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/crflogd root dba 0750
unix /etc/oracle/oprocd root dba 0775
unix /etc/oracle/oprocd/check root dba 0770
unix /etc/oracle/oprocd/stop root dba 0770
unix /etc/oracle/oprocd/fatal root dba 0770
unix /etc/oracle/scls_scr root dba 0755
unix /etc/oracle/scls_scr/zfzhlhrdb1 root dba 0755
unix /var/tmp/.oracle root dba 01777
unix /tmp/.oracle root dba 01777
unix /oracle/app/11.2.0/grid/log/zfzhlhrdb1/acfsreplroot root dba 0750
# create $ID, if it doesn't exist (applicable only in dev env)
unix /etc root root 0755
unix /oracle/app/11.2.0/grid root dba 0755
# Last Gasp files directory - change "unix" to "all"
# once Windows makes a directory decision.
unix /etc/oracle/lastgasp root dba 0770
unix /etc/rc.d/rc2.d root root 0755
[zfzhlhrdb1:grid]:/oracle/app/11.2.0/grid/crs/utl> more crsconfig_fileperms
# Copyright (c) 2009, 2013, Oracle and/or its affiliates. All rights reserved.
# The values in each line use the following format:
#
# OSLIST FILENAME OWNER GROUP PERMS
#
# Note:
# 1) OSLIST is a comma-separated list of platforms on which the file
# permissions need to be set. 'all' indicates that the directory needs
# to be created on every platform. OSLIST MUST NOT contain whitespace.
# 2) Permissions need to be specified AS OCTAL NUMBERS. If permissions
# are not specified, default (umask) values will be used.
# 3) The fields within each line of this file must be delimited by a single space
#
unix /oracle/app/11.2.0/grid/log/zfzhlhrdb1/alertzfzhlhrdb1.log grid dba 0664
unix /oracle/app/11.2.0/grid/bin/usrvip root dba 0755
unix /oracle/app/11.2.0/grid/bin/appvipcfg root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/preupdate.sh grid dba 0755
unix /oracle/app/11.2.0/grid/crs/install/s_crsconfig_defs grid dba 0755
unix /oracle/app/11.2.0/grid/bin/cluutil grid dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrcheck root dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrcheck.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrconfig root dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrconfig.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrdump root dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrdump.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/ocrpatch root dba 0755
unix /oracle/app/11.2.0/grid/bin/appagent grid dba 0755
unix /oracle/app/11.2.0/grid/bin/clssproxy grid dba 0755
unix /oracle/app/11.2.0/grid/bin/cssvfupgd root dba 0755
unix /oracle/app/11.2.0/grid/bin/cssvfupgd.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/racgwrap grid dba 0755
unix /oracle/app/11.2.0/grid/bin/cemutls grid dba 0755
unix /oracle/app/11.2.0/grid/bin/cemutlo grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_getperm grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_profile grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_register grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_relocate grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_setperm grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_start grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_stat grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_stop grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crs_unregister grid dba 0755
unix /oracle/app/11.2.0/grid/bin/gipcd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/mdnsd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/gpnpd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/gpnptool grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oranetmonitor grid dba 0755
unix /oracle/app/11.2.0/grid/bin/rdtool grid dba 0755
unix /oracle/app/11.2.0/grid/bin/octssd root dba 0741
unix /oracle/app/11.2.0/grid/bin/octssd.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/ohasd root dba 0741
unix /oracle/app/11.2.0/grid/bin/ohasd.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/crsd root dba 0741
unix /oracle/app/11.2.0/grid/bin/crsd.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/evmd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evminfo grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmlogger grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmmkbin grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmmklib grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmpost grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmshow grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmsort grid dba 0755
unix /oracle/app/11.2.0/grid/bin/evmwatch grid dba 0755
unix /oracle/app/11.2.0/grid/bin/lsnodes grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oifcfg grid dba 0755
unix /oracle/app/11.2.0/grid/bin/olsnodes grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oraagent grid dba 0755
unix /oracle/app/11.2.0/grid/bin/orarootagent root dba 0741
unix /oracle/app/11.2.0/grid/bin/orarootagent.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/scriptagent grid dba 0755
unix /oracle/app/11.2.0/grid/bin/lsdb grid dba 0755
unix /oracle/app/11.2.0/grid/bin/emcrsp grid dba 0755
unix /oracle/app/11.2.0/grid/bin/onsctl grid dba 0755
unix /oracle/app/11.2.0/grid/crs/install/onsconfig grid dba 0554
unix /oracle/app/11.2.0/grid/bin/gnsd root dba 0741
unix /oracle/app/11.2.0/grid/bin/gnsd.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/gsd.sh grid dba 0755
unix /oracle/app/11.2.0/grid/bin/gsdctl grid dba 0755
unix /oracle/app/11.2.0/grid/bin/scrctl grid dba 0750
unix /oracle/app/11.2.0/grid/bin/vipca grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oc4jctl grid dba 0755
unix /oracle/app/11.2.0/grid/bin/cvures grid dba 0755
unix /oracle/app/11.2.0/grid/bin/odnsd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/qosctl grid dba 0755
unix /oracle/app/11.2.0/grid/crs/install/cmdllroot.sh grid dba 0755
unix /oracle/app/11.2.0/grid/crs/utl/rootdelete.sh root root 0755
unix /oracle/app/11.2.0/grid/crs/utl/rootdeletenode.sh root root 0755
unix /oracle/app/11.2.0/grid/crs/utl/rootdeinstall.sh root root 0755
unix /oracle/app/11.2.0/grid/crs/utl/rootaddnode.sh root root 0755
unix /oracle/app/11.2.0/grid/lib/libskgxpcompat.so grid dba 0644
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/client/olsnodes.log grid dba 0666
all /oracle/app/11.2.0/grid/log/zfzhlhrdb1/client/oifcfg.log grid dba 0666
unix /oracle/app/11.2.0/grid/bin/srvctl root dba 0755
unix /oracle/app/11.2.0/grid/bin/cluvfy root dba 0755
unix /oracle/app/11.2.0/grid/bin/clsecho root dba 0755
unix /oracle/app/11.2.0/grid/bin/clsecho.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/clscfg root dba 0755
unix /oracle/app/11.2.0/grid/bin/clscfg.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/clsfmt root dba 0755
unix /oracle/app/11.2.0/grid/bin/clsfmt.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/clsid grid dba 0755
unix /oracle/app/11.2.0/grid/bin/crsctl root dba 0755
unix /oracle/app/11.2.0/grid/bin/crsctl.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/ndfnceca grid dba 0750
unix /oracle/app/11.2.0/grid/bin/oclskd root dba 0755
unix /oracle/app/11.2.0/grid/bin/oclskd.bin root dba 0751
unix /oracle/app/11.2.0/grid/bin/oclsomon grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oclsvmon grid dba 0755
unix /oracle/app/11.2.0/grid/bin/ocssd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/cssdagent root dba 0741
unix /oracle/app/11.2.0/grid/bin/cssdagent.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/cssdmonitor root dba 0741
unix /oracle/app/11.2.0/grid/bin/cssdmonitor.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/diskmon root dba 0741
unix /oracle/app/11.2.0/grid/bin/diskmon.bin root dba 0741
unix /oracle/app/11.2.0/grid/bin/diagcollection.sh root dba 0755
unix /oracle/app/11.2.0/grid/bin/oradnssd grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oradnssd.bin grid dba 0755
unix /oracle/app/11.2.0/grid/bin/setasmgidwrap grid dba 0755
unix /oracle/app/11.2.0/grid/bin/oclumon root dba 0750
unix /oracle/app/11.2.0/grid/bin/oclumon.bin root dba 0750
unix /oracle/app/11.2.0/grid/bin/oclumon.pl grid dba 0750
unix /oracle/app/11.2.0/grid/bin/crswrapexece.pl root dba 0744
unix /oracle/app/11.2.0/grid/bin/crfsetenv root dba 0750
unix /oracle/app/11.2.0/grid/bin/osysmond root dba 0750
unix /oracle/app/11.2.0/grid/bin/osysmond.bin root dba 0750
unix /oracle/app/11.2.0/grid/bin/ologgerd root dba 0750
unix /oracle/app/11.2.0/grid/bin/ologdbg grid dba 0750
unix /oracle/app/11.2.0/grid/bin/ologdbg.pl grid dba 0750
unix /etc/oracle/setasmgid root dba 4710
# Jars and shared libraries used by the executables invoked by the root script
unix /oracle/app/11.2.0/grid/jlib/srvm.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/srvmasm.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/srvctl.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/srvmhas.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/gns.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/ons.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/netcfg.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/i18n.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/supercluster.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/supercluster-common.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/antlr-complete.jar root dba 0644
unix /oracle/app/11.2.0/grid/jlib/antlr-3.3-complete.jar root dba 0644
unix /oracle/app/11.2.0/grid/lib/libhasgen11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libocr11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libocrb11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libocrutl11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libclntsh.so.11.1 root dba 0644
unix /oracle/app/11.2.0/grid/lib/libclntshcore.so.11.1 root dba 0644
unix /oracle/app/11.2.0/grid/lib/libskgxn2.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libskgxp11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libasmclntsh11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libcell11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libnnz11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libclsra11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libgns11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libeons.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libonsx.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libeonsserver.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libsrvm11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libsrvmhas11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libsrvmocr11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libuini11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libgnsjni11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/librdjni11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libgnsjni11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libclsce11.so root dba 0644
unix /oracle/app/11.2.0/grid/lib/libcrf11.so root dba 0644
unix /oracle/app/11.2.0/grid/bin/diagcollection.pl root dba 0755
# crs configuration scripts invoked from rootcrs.pl
unix /oracle/app/11.2.0/grid/crs/install/crsconfig_lib.pm root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/s_crsconfig_lib.pm root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/crsdelete.pm root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/crspatch.pm root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/oracss.pm root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/oraacfs.pm root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/hasdconfig.pl root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/rootcrs.pl root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/roothas.pl root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/preupdate.sh root dba 0755
unix /oracle/app/11.2.0/grid/crs/install/rootofs.sh root dba 0755
# XXX: required only for dev env, where inittab ($IT) is not present already
unix /etc/inittab root root 0644
# USM FILES
# Only files which will be installed with executable permissions need
# to be listed.
unix /oracle/app/11.2.0/grid/bin/acfsdriverstate root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsload root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsregistrymount root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsroot root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfssinglefsmount root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsrepl_apply root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsrepl_apply.bin root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsreplcrs grid dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsreplcrs.pl grid dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsrepl_initializer root dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsrepl_monitor grid dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsrepl_preapply grid dba 0755
unix /oracle/app/11.2.0/grid/bin/acfsrepl_transport grid dba 0755
unix /oracle/app/11.2.0/grid/lib/acfsdriverstate.pl root dba 0644
unix /oracle/app/11.2.0/grid/lib/acfsload.pl root dba 0644
unix /oracle/app/11.2.0/grid/lib/acfsregistrymount.pl root dba 0644
unix /oracle/app/11.2.0/grid/lib/acfsroot.pl root dba 0644
unix /oracle/app/11.2.0/grid/lib/acfssinglefsmount.pl root dba 0644
unix /oracle/app/11.2.0/grid/lib/acfstoolsdriver.sh root dba 0755
unix /oracle/app/11.2.0/grid/lib/libusmacfs11.so grid dba 0644
#EVM config files
unix /oracle/app/11.2.0/grid/evm/admin/conf/evm.auth root dba 0644
unix /oracle/app/11.2.0/grid/evm/admin/conf/evmdaemon.conf root dba 0644
unix /oracle/app/11.2.0/grid/evm/admin/conf/evmlogger.conf root dba 0644
# TFA files
unix /oracle/app/11.2.0/grid/crs/install/tfa_setup.sh root dba 0755
unix /oracle/app/11.2.0/grid/cdata/zfzhlhrdb1.olr root dba 0600
unix /etc/oracle/olr.loc root dba 0644
unix /etc/oracle/ocr.loc root dba 0644
[zfzhlhrdb1:grid]:/oracle/app/11.2.0/grid/crs/utl>
我们可以看到，确实是这样，crsconfig_dirs里面记录所有$GRID_HOME相关目录的权限。crsconfig_fileperms中记录了文件的权限.
一.3.7.1 校验权限
我们可以通过命令来校验：Validate the <GRID_HOME> by using cluvfy tool.
$ cluvfy comp software -n all -verbose
[zfzhlhrdb1:grid]:/home/grid>cluvfy comp software -n all -verbose
Verifying software
Check: Software
Component: crs
Node Name: zfzhlhrdb2
/oracle/app/11.2.0/grid/bin/racgeut..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgeut" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/racgmain..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgmain" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/asmproxy..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/asmproxy" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/lib/oc4jctl_common.pm..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/lib/oc4jctl_common.pm" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/lib/oc4jctl_lib.pm..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/lib/oc4jctl_lib.pm" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/appagent.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/appagent.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/clssproxy.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/clssproxy.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_getperm.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_getperm.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_profile.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_profile.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_register.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_register.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_relocate.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_relocate.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_setperm.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_setperm.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_start.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_start.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_stat.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_stat.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_stop.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_stop.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_unregister.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_unregister.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gipcd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gipcd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/mdnsd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/mdnsd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gpnpd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gpnpd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gpnptool.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gpnptool.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oranetmonitor.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oranetmonitor.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evminfo.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evminfo.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmlogger.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmlogger.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmmkbin.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmmkbin.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmmklib.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmmklib.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmpost.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmpost.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmshow.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmshow.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmsort.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmsort.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmwatch.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmwatch.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oraagent.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oraagent.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/racgevtf..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgevtf" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/racgvip..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgvip" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/sclsspawn..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/sclsspawn" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/scriptagent.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/scriptagent.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oprocd..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oprocd" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/emcrsp.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/emcrsp.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/clsid.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/clsid.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/ocssd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/ocssd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crstmpl.scr..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crstmpl.scr" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evt.sh..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evt.sh" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/cemutls.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/cemutls.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/cemutlo.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/cemutlo.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/lsdb.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/lsdb.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oifcfg.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oifcfg.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/olsnodes.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/olsnodes.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gsd..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gsd" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oc4jctl.pl..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oc4jctl.pl" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/lib/s_oc4jctl_lib.pm..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/lib/s_oc4jctl_lib.pm" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/appvipcfg.pl..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/appvipcfg.pl" did not match the expected value. [Expected = "0750" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/lxinst..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/lxinst" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/clone/rootpre/ORCLcluster/lib/libskgxnr.a...No such file or directory
Node Name: zfzhlhrdb1
/oracle/app/11.2.0/grid/bin/racgeut..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgeut" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/racgmain..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgmain" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/asmproxy..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/asmproxy" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/lib/oc4jctl_common.pm..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/lib/oc4jctl_common.pm" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/lib/oc4jctl_lib.pm..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/lib/oc4jctl_lib.pm" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/appagent.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/appagent.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/clssproxy.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/clssproxy.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_getperm.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_getperm.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_profile.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_profile.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_register.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_register.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_relocate.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_relocate.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_setperm.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_setperm.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_start.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_start.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_stat.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_stat.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_stop.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_stop.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crs_unregister.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crs_unregister.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gipcd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gipcd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/mdnsd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/mdnsd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gpnpd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gpnpd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gpnptool.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gpnptool.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oranetmonitor.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oranetmonitor.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evminfo.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evminfo.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmlogger.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmlogger.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmmkbin.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmmkbin.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmmklib.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmmklib.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmpost.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmpost.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmshow.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmshow.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmsort.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmsort.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evmwatch.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evmwatch.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oraagent.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oraagent.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/racgevtf..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgevtf" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/racgvip..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/racgvip" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/sclsspawn..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/sclsspawn" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/scriptagent.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/scriptagent.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oprocd..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oprocd" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/emcrsp.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/emcrsp.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/clsid.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/clsid.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/ocssd.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/ocssd.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/crstmpl.scr..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/crstmpl.scr" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/evt.sh..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/evt.sh" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/cemutls.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/cemutls.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/cemutlo.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/cemutlo.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/lsdb.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/lsdb.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oifcfg.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oifcfg.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/olsnodes.bin..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/olsnodes.bin" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/gsd..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/gsd" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/oc4jctl.pl..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/oc4jctl.pl" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/lib/s_oc4jctl_lib.pm..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/lib/s_oc4jctl_lib.pm" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/appvipcfg.pl..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/appvipcfg.pl" did not match the expected value. [Expected = "0750" ; Found = "0775"]
/oracle/app/11.2.0/grid/bin/lxinst..."Permissions" did not match reference
Permissions of file "/oracle/app/11.2.0/grid/bin/lxinst" did not match the expected value. [Expected = "0755" ; Found = "0775"]
/oracle/app/11.2.0/grid/clone/rootpre/ORCLcluster/lib/libskgxnr.a...No such file or directory
1227 files verified
Software check failed
Verification of software was unsuccessful on all the specified nodes.
一.3.7.2 解决
所以要解决这个问题其实并不难，我们大致可以通过如下几种方法来解决：
1. 根据前面的几个权限配置脚本自己参考进行修改，实际上并不难，直接UE编辑就很快搞定.
2. 根据Mos文档提供的建议通过 $GRID_HOME/crs/install/rootcrs.pl -init 或 roothas.pl -init进行解决. rootcrs.pl –init是在PSU>11.2.0.3.6下执行的，如果PSU<11.2.0.3.6可以执行如下两条命令来实现同样的效果
<GRID_HOME>/crs/install/rootcrs.pl -unlock
<GRID_HOME>/crs/install/rootcrs.pl -patch
For 11.2:
For clustered Grid Infrastructure, as root user
# cd <GRID_HOME>/crs/install/
# ./rootcrs.pl -init
For Standalone Grid Infrastructure, as root user
# cd <GRID_HOME>/crs/install/
# ./roothas.pl -init
For 12c:
For clustered Grid Infrastructure, as root user
# cd <GRID_HOME>/crs/install/
# ./rootcrs.sh -init
For Standalone Grid Infrastructure, as root user
# cd <GRID_HOME>/crs/install/
# ./roothas.sh -init
3.采用MOS1515018.1文档提供的脚本在正常库上生成脚本，然后将生成的脚本在异常库上执行从而来修复权限问题（应该和方法2结合使用）。
4. 11gR2可以deconfig crs的配置，然后重新跑root.sh即可。重新跑root.sh脚本并不影响数据库，所以无需担心（个人推荐的一种方式）.
$ORACLE_HOME/crs/install/rootcrs.pl -deconfig -force -verbose
$ORACLE_HOME/root.sh
5. 如果是rac的某个节点的误操作，那么还可以通过delete node然后add node来做，不过这个相对麻烦太多了，但是或许是最保险的一种方式。oracle也推荐这样，因为你如果人为修改文件权限，很难保证以后运行过程中不会出现什么问题。
补充：
Linux环境中还可以通过getfacl和setfacl来进行操作，如下是例子：
1) getfacl /home/grid/app/11.2/grid > dir_privs.txt
2) set –restore dir_privs.txt
总结：
在安装有GI的环境下，权限、属主是严格被设定的，任何对于它们的错误修改容易引发一系列的问题，而且这些问题往往都很诡异很难按照常规的思路去诊断。万一权限或属主被修改了可以通过rootcrs.pl -init及permission.pl进行修复，rootcrs.pl –init仅修复GI的核心目录，所以其修复速度较快，如果遇到GI无法启动的问题，建议首选这种方法以使GI能够快速启动，但其缺点在于无法全量的进行修复，GI虽然正常了，并不能保证之后的运行过程中不出现这样那样的问题，这时就需要permission.pl出场了，permission.pl的运行模式决定了源库(权限正确的库)与目标库(权限错误的库)间的软件版本尽可能的一致，所以源库一定要选好，否则问题会更糟，另外如果源、目标两个库的安装目录不一样还需要对permission*脚本作调整后再执行。
所以个人建议还是跑root.sh来的稳妥一点。
一.3.7.3 MOS 1515018.1 permission.pl脚本的使用
chmod 755 permission.pl
oracle用户获取ORACLE_HOME： ./permission.pl $ORACLE_HOME
root用户获取GRID_HOME： ./permission.pl $ORACLE_HOME
Script generates two files
a. permission-<time stamp> - This contains file permission in octal value, owner and group information of the files captured
b. restore-perm-<time stamp>.cmd - This contains command to change the permission, owner, and group of the captured files
拷贝到目标主机后分别执行：
chmod 755 restore-perm-<timestamp>.cmd
./restore-perm-<timestamp>.cmd
一.3.8 如何修复ASM实例名和节点名不一致的情况
对于10g的情况，我们可以参考Dave大神的blog：RAC修改ASM实例名的步骤:http://blog.csdn.net/tianlesoftware/article/details/6275827
对于11g的情况，我们只能重新执行root.sh脚本来修复这个问题。
一.3.9 如何彻底清除CRS信息
[ZFTPCCDB1:root]:/>$ORACLE_HOME/crs/install/rootcrs.pl -h
Unknown option: h
Usage:
rootcrs.pl [-verbose] [-upgrade [-force] | -patch]
[-paramfile <parameter-file>]
[-deconfig [-deinstall] [-keepdg] | -downgrade] [-force] [-lastnode]
[-downgrade] [-oldcrshome <old crshome path>] [-version <old crs version>]
[-unlock [-crshome <path to crs home>] [-nocrsstop]]
Options:
-verbose Run this script in verbose mode
-upgrade Oracle HA is being upgraded from previous version
-patch Oracle HA is being upgraded to a patch version
-paramfile Complete path of file specifying HA parameter values
-lastnode Force the node this is executing on to be considered the
last node of the install and perform actions associated
with configuring the last node
-downgrade Downgrade the clusterware
-version For use with downgrade; special handling is required if
downgrading to 9i. This is the old crs version in the format
A.B.C.D.E (e.g 11.1.0.6.0).
-deconfig Remove Oracle Clusterware to allow it to be uninstalled or reinstalled.
-force Force the execution of steps in delete that cannot be verified
to be safe
-deinstall Reset the permissions on CRS home during de-configuration
-keepdg Keep existing diskgroups during de-configuration
-unlock Unlock CRS home
-crshome Complete path of crs home. Use with unlock option.
-oldcrshome For use with downgrade. Complete path of the old crs home.
-nocrsstop used with unlock option to reset permissions on an inactive grid home
If neither -upgrade nor -patch is supplied, a new install is performed
To see the full manpage for this program, execute:
perldoc rootcrs.pl
[ZFTPCCDB1:root]:/>
根据MOS How to Proceed from Failed 11gR2 Grid Infrastructure (CRS) Installation (文档 ID 942166.1) 文档，若要重新执行root.sh脚本，我们可以如下操作：
在除最后一个节点外的所有节点执行：$ORACLE_HOME/crs/install/rootcrs.pl -deconfig -force -verbose
在最后一个节点执行：$ORACLE_HOME/crs/install/rootcrs.pl -deconfig -force -verbose -lastnode
重启OS后再执行：$ORACLE_HOME/root.sh
需要注意的一点是，我们在执行完$ORACLE_HOME/crs/install/rootcrs.pl -deconfig -force -verbose执成之后需要删除如下的文件：
ls -l $ORACLE_BASE/Clusterware/ckptGridHA*
find $ORACLE_HOME/gpnp/* -type f
find $ORACLE_HOME/gpnp/* -type f -exec rm -rf {} \;
其中（find $ORACLE_HOME/gpnp/* -type f）中的文件删除后，我们在重新执行root.sh的时候才会有如下的提示：
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
---------------------------------------------------------------------------------------------------------------------
第二章 实验部分
二.1 实验环境介绍
项目 |
primary db |
db 类型 |
单实例 |
db version |
11.2.0.2.0 |
db 存储 |
ASM |
二.2 实验目标
本次我们模拟6个实验：
1、dd备份恢复OCR
2、OCR的物理备份和恢复
3、OCR的逻辑备份和恢复
4、删除grid用户下的$ORACLE_HOME/log下的文件夹，尝试恢复重新执行root.sh来恢复数据
5、permission.pl脚本的使用
6、清除crs和更换修复实例名就不实验了，因为都是执行root.sh脚本
二.3 实验过程
二.3.1 实验一：dd备份恢复OCR
首先关掉集群并且dd备份ocr磁盘头内容：
[zfzhlhrdb1:root]:/>crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 7e59ff6d88ba4fc0bfe5b6ccdd27ba55 (/dev/rhdisk2) [DATA]
Located 1 voting disk(s).
[zfzhlhrdb1:root]:/>dd if=/dev/rhdisk2 of=/tmp/votedisk_lhr.bak bs=1024k count=4
4+0 records in.
4+0 records out.
关闭集群后，使用dd命令模拟ocr 损坏
[zfzhlhrdb1:root]:/>dd if=/dev/zero of=/dev/rhdisk2 bs=1024k count=4
4+0 records in.
4+0 records out.
[zfzhlhrdb2:root]:/>sh disk*
------------------------------------------------------------------------------------------------------------------------------
| disk | PVID | no_reserve | size(G) | disktype | disk_storage |
------------------------------------------------------------------------------------------------------------------------------
| crw------- root system /dev/rhdisk0 | 00f60f2b47d4b56f | no_reserve | 128 | rootvg | EMC,vscsi,3 |
| crw------- root system /dev/rhdisk1 | 00f60f2bd2147554 | single_path | 128 | T_XDESK_APP2_vg | EMC,fscsi,32 |
| crw-rw---- grid asmadmin /dev/rhdisk2 | 0000000000000000 | no_reserve | 128 | Not_Used | EMC,fscsi,32 |
| crw-rw---- grid asmadmin /dev/rhdisk3 | 0000000000000000 | no_reserve | 128 | ASM:+FRA | EMC,fscsi,32 |
| crw-rw---- root system /dev/rhdisk4 | 0000000000000000 | no_reserve | 128 | gpfs1nsd | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk5 | 00f60f2b6046c20d | single_path | 0 | HeadDisk | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk6 | 00f60f2b6046ca7d | single_path | 0 | HeadDisk | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk7 | 00f60f2b6046d2f8 | single_path | 0 | HeadDisk | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk8 | 00f60f2b6046dae6 | single_path | 0 | HeadDisk | EMC,fscsi,32 |
------------------------------------------------------------------------------------------------------------------------------
| ASMDISK_TOTAL:1 TOTAL_SIZE(GB):128 |
------------------------------------------------------------------------------------------------------------------------------
[zfzhlhrdb2:root]:/>
[zfzhlhrdb2:root]:/>cat disk*
if [ 1 = 1 ] ;then
sum=0;asmnum=0
awk 'BEGIN {printf "------------------------------------------------------------------------------------------------------------------------------\n"; printf "%-43s %-18s %-14s %-8s %-15s %-14s\n","| disk ","| PVID ","| no_reserve ","| size(G)","| disktype ","| disk_storage |"; printf "------------------------------------------------------------------------------------------------------------------------------\n";}'
for diskname in `lspv | grep disk | awk '{print $1}'`;do
mydiskname=`ls -l /dev/rdiskname|grep−w/dev/rdiskname|grep−w/dev/rdiskname| cut -c 1-12,17-38,59-76`
mydiskpvid=`lquerypv -H /dev/$diskname | cut -c 1-16`
if [ "${mydiskpvid}" = "" ];then mydiskpvid="0000000000000000" ; fi 2>/dev/null
mydiskreserve=`lsattr -El $diskname | grep -i reserve_policy | cut -c 17-30`
mydisksize=`bootinfo -s diskname2>/dev/null‘;let"mydisksize1=diskname2>/dev/null‘;let"mydisksize1=mydisksize/1024" 2>/dev/null
mydiskvg=`lspv | grep -w $diskname | awk '{print $3}'`
mydiskasmgroup=`lquerypv -h /dev/r$diskname|head -n 7|tail -n 1|awk -F ' ' '{print $NF}'|sed -e 's/\.//g' -e 's/\|//g' | awk '{ if (1!="")printf"+"1!="")printf"+"1 ; else print "NULL"}'`
mydiskflag=`lquerypv -h /dev/r$diskname 2>/dev/null|grep -i orcldisk|wc -l`
if [ mydisksize−lt1000];thenmydisktype="HeadDisk";elif[mydisksize−lt1000];thenmydisktype="HeadDisk";elif[{mydisksize} -gt 1000 -a mydiskflag−gt0];thenmydisktype="ASM:"mydiskflag−gt0];thenmydisktype="ASM:"mydiskasmgroup; elif [ mydisksize−gt1000−amydisksize−gt1000−a{mydiskflag} -eq 0 -a mydiskvg!="None"];thenmydisktype=mydiskvg!="None"];thenmydisktype=mydiskvg ; else mydisktype="Not_Used"; fi 2>/dev/null
mydiskpath=`lspath -l diskname2>/dev/null|head−1|awk′print$NF′|sed"s/.diskname2>/dev/null|head−1|awk′print$NF′|sed"s/.//"`
mydiskstring=`odmget -q attribute="unique_id" CuAt|egrep "name|value"|paste - -|tr '\t' ' '|grep -w ${diskname}|sed 's/\"//g'`
mydiskstorage=`echo ${mydiskstring} 2> /dev/null|awk '{ if(NF /EMC/)print"EMC"elseif(NF /EMC/)print"EMC"elseif(NF ~ /NETAPP/) {print "NETAPP"} else if($NF ~ /HITACHI/) {print "HDS"}}'`
mydiskdepth=`lsattr -El ${diskname}|grep queue_depth|awk '{print $2}'`
mydiskstorage1=mydiskstorage","mydiskstorage","mydiskpath","$mydiskdepth
[ mydisksize1−gt1−amydisksize1−gt1−a{mydiskflag} -gt 0 ] && { (( sum=sum+mydisksize1));((asmnum=mydisksize1));((asmnum=asmnum+1 )) ;}
echo "mydiskname""mydiskname""mydiskpvid" "mydiskreserve""mydiskreserve""{mydisksize1%.*}" "mydisktype""mydisktype""mydiskstorage1" | awk '{printf "| %-10s %-6s %-8s %-14s | %-17s | %-12s | %-8s| %-15s | %-14s |\n",1,1,2,3,3,4,5,5,6,7,7,8,$9}'
done
awk 'BEGIN {printf "------------------------------------------------------------------------------------------------------------------------------\n";}'
echo "ASMDISK_TOTAL:asmnum""TOTALSIZE(GB):asmnum""TOTALSIZE(GB):sum" |awk '{printf "| %-20s %-101s |\n", 1,1,2}'
awk 'BEGIN {printf "------------------------------------------------------------------------------------------------------------------------------\n";}'
fi
[zfzhlhrdb2:root]:/>
[zfzhlhrdb1:root]:/>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[zfzhlhrdb1:root]:/>crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[zfzhlhrdb1:root]:/>
[zfzhlhrdb1:root]:/>crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE OFFLINE Instance Shutdown
ora.cluster_interconnect.haip
1 ONLINE OFFLINE
ora.crf
1 ONLINE ONLINE zfzhlhrdb1
ora.crsd
1 ONLINE OFFLINE
ora.cssd
1 ONLINE OFFLINE STARTING
ora.cssdmonitor
1 ONLINE ONLINE zfzhlhrdb1
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.drivers.acfs
1 ONLINE OFFLINE
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE zfzhlhrdb1
ora.gpnpd
1 ONLINE ONLINE zfzhlhrdb1
ora.mdnsd
1 ONLINE ONLINE zfzhlhrdb1
[zfzhlhrdb1:root]:/>
[zfzhlhrdb1:grid]:/home/grid>cluvfy comp ocr -n all
Verifying OCR integrity
Unable to retrieve nodelist from Oracle Clusterware
Verification cannot proceed
[zfzhlhrdb2:root]:/>ocrcheck
KGFCHECK kgfnStmtExecute01c: ret == OCI_SUCCESS: FAILED at kgfn.c:1563
KGFCHECK kgfpOpen01c: ok: FAILED at kgfp.c:519
-- trace dump on error exit --
Error [kgfoOpen01] in [kgfokge] at kgfo.c:1697
ORA-17503: ksfdopn:2 Failed to open file +DATA.255.4294967295
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-06512: at line 4
Category: 8
DepInfo: 15056
ADR is not properly configured
-- trace dump end --
-- trace dump on error exit --
Error [kgfoOpen01] in [kgfokge] at kgfo.c:1546
ORA-17503: ksfdopn:2 Failed to open file +DATA.255.4294967295
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-06512: at line 4
Category: 8
DepInfo: 15056
ADR is not properly configured
-- trace dump end --
KGFCHECK kgfnStmtSingle3: ret == OCI_SUCCESS: FAILED at kgfn.c:1770
-- trace dump on error exit --
Error [kgfo] in [kgfoCkMt03] at kgfo.c:2080
diskgroup DATA not mounted ()
Category: 6
DepInfo: 0
ADR is not properly configured
-- trace dump end --
KGFCHECK kgfnStmtSingle3: ret == OCI_SUCCESS: FAILED at kgfn.c:1770
-- trace dump on error exit --
Error [kgfo] in [kgfoCkMt03] at kgfo.c:2080
diskgroup DATA not mounted ()
Category: 6
DepInfo: 0
ADR is not properly configured
-- trace dump end --
PROT-602: Failed to retrieve data from the cluster registry
PROC-26: Error while accessing the physical storage
说明集群已经不能启动了，磁盘头也被清掉了，接下来我们恢复集群：
[zfzhlhrdb1:root]:/>crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1'
。。。。。。。。
[zfzhlhrdb1:root]:/>dd if=/tmp/votedisk_lhr.bak of=/dev/rhdisk2 bs=1024k count=4
4+0 records in.
4+0 records out.
[zfzhlhrdb1:root]:/>sh disk*
------------------------------------------------------------------------------------------------------------------------------
| disk | PVID | no_reserve | size(G) | disktype | disk_storage |
------------------------------------------------------------------------------------------------------------------------------
| crw------- root system /dev/rhdisk0 | 00f63a6147ced87a | no_reserve | 128 | rootvg | EMC,vscsi,3 |
| crw------- root system /dev/rhdisk1 | 00f63a61d2143e86 | single_path | 128 | T_XDESK_APP1_vg | EMC,fscsi,32 |
| crw-rw---- grid asmadmin /dev/rhdisk2 | 0000000000000000 | no_reserve | 128 | ASM:+DATA | EMC,fscsi,32 |
| crw-rw---- grid asmadmin /dev/rhdisk3 | 0000000000000000 | no_reserve | 128 | ASM:+FRA | EMC,fscsi,32 |
| crw-rw---- root system /dev/rhdisk4 | 0000000000000000 | no_reserve | 128 | gpfs1nsd | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk5 | 00f63a61c89dbd11 | single_path | 0 | HeadDisk | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk6 | 00f63a6160469425 | single_path | 0 | HeadDisk | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk7 | 00f63a6160469c21 | single_path | 0 | HeadDisk | EMC,fscsi,32 |
| crw------- root system /dev/rhdisk8 | 0000000000000000 | single_path | 0 | HeadDisk | EMC,fscsi,32 |
------------------------------------------------------------------------------------------------------------------------------
| ASMDISK_TOTAL:2 TOTAL_SIZE(GB):256 |
------------------------------------------------------------------------------------------------------------------------------
[zfzhlhrdb1:root]:/>
[zfzhlhrdb1:root]:/>crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE zfzhlhrdb1 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE zfzhlhrdb1
ora.crf
1 ONLINE ONLINE zfzhlhrdb1
ora.crsd
1 ONLINE ONLINE zfzhlhrdb1 STOPPING
ora.cssd
1 ONLINE ONLINE zfzhlhrdb1
ora.cssdmonitor
1 ONLINE ONLINE zfzhlhrdb1
ora.ctssd
1 ONLINE ONLINE zfzhlhrdb1 OBSERVER
ora.diskmon
1 OFFLINE OFFLINE
ora.drivers.acfs
1 ONLINE ONLINE zfzhlhrdb1
ora.evmd
1 ONLINE ONLINE zfzhlhrdb1
ora.gipcd
1 ONLINE ONLINE zfzhlhrdb1
ora.gpnpd
1 ONLINE ONLINE zfzhlhrdb1
ora.mdnsd
1 ONLINE ONLINE zfzhlhrdb1
[zfzhlhrdb1:root]:/>
[zfzhlhrdb1:root]:/>crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'zfzhlhrdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'zfzhlhrdb1'
CRS-2675: Stop of 'ora.oc4j' on 'zfzhlhrdb1' failed
CRS-2679: Attempting to clean 'ora.oc4j' on 'zfzhlhrdb1'
CRS-2681: Clean of 'ora.oc4j' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.ons' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.net1.network' on 'zfzhlhrdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cssd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'zfzhlhrdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[zfzhlhrdb1:root]:/>ps -ef|grep d.bin
root 4718784 7667788 0 19:47:26 pts/2 0:00 grep d.bin
[zfzhlhrdb1:root]:/>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[zfzhlhrdb1:root]:/>
[zfzhlhrdb2:root]:/>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.FRA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.LISTENER.lsnr
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.LISTENER_LHRDG.lsnr
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.asm
ONLINE ONLINE zfzhlhrdb1 Started
ONLINE ONLINE zfzhlhrdb2 Started
ora.gsd
OFFLINE OFFLINE zfzhlhrdb1
OFFLINE OFFLINE zfzhlhrdb2
ora.net1.network
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.ons
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.registry.acfs
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zfzhlhrdb1
ora.cvu
1 ONLINE ONLINE zfzhlhrdb1
ora.lhrdg.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE Instance Shutdown
ora.oc4j
1 ONLINE ONLINE zfzhlhrdb1
ora.oraesdb.db
1 ONLINE OFFLINE Corrupted Controlfi
le
2 ONLINE OFFLINE Corrupted Controlfi
le
ora.oraeskdb.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE Instance Shutdown
ora.raclhr.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE Instance Shutdown
ora.scan1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb2.vip
1 ONLINE ONLINE zfzhlhrdb2
[zfzhlhrdb2:root]:/>
可以看到集群可以正常启动了，接下来修复几个db的问题即可。
二.3.2 实验二：通过kfed修复磁盘头
备份ASM磁盘头部
# dd if=/dev/rhdisk2 of=/tmp/asm_dd.bak bs=1024 count=4
[zfzhlhrdb1:root]:/>dd if=/dev/rhdisk2 of=/tmp/asm_dd.bak bs=1024 count=4
4+0 records in.
4+0 records out.
破坏ASM磁盘头部
# dd if=/dev/zero of=/dev/rhdisk2 bs=1024 count=4
[zfzhlhrdb1:root]:/>dd if=/dev/zero of=/dev/rhdisk2 bs=1024 count=4
4+0 records in.
4+0 records out.
查看ASM磁盘头部信息（空的）
# lquerypv -h /dev/rhdisk2
[zfzhlhrdb1:root]:/>lquerypv -h /dev/rhdisk2
00000000 00000000 00000000 00000000 00000000 |................|
00000010 00000000 00000000 00000000 00000000 |................|
00000020 00000000 00000000 00000000 00000000 |................|
00000030 00000000 00000000 00000000 00000000 |................|
00000040 00000000 00000000 00000000 00000000 |................|
00000050 00000000 00000000 00000000 00000000 |................|
00000060 00000000 00000000 00000000 00000000 |................|
00000070 00000000 00000000 00000000 00000000 |................|
00000080 00000000 00000000 00000000 00000000 |................|
00000090 00000000 00000000 00000000 00000000 |................|
000000A0 00000000 00000000 00000000 00000000 |................|
000000B0 00000000 00000000 00000000 00000000 |................|
000000C0 00000000 00000000 00000000 00000000 |................|
000000D0 00000000 00000000 00000000 00000000 |................|
000000E0 00000000 00000000 00000000 00000000 |................|
000000F0 00000000 00000000 00000000 00000000 |................|
停止HAS服务
# crsctl stop has -f
[zfzhlhrdb1:root]:/>crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'zfzhlhrdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.LISTENER_LHRDG.lsnr' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.cvu' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.FRA.dg' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oralhrq.db' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cvu' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.FRA.dg' on 'zfzhlhrdb1' succeeded
CRS-2676: Start of 'ora.cvu' on 'zfzhlhrdb2' succeeded
CRS-2677: Stop of 'ora.oralhrq.db' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.LISTENER_LHRDG.lsnr' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.registry.acfs' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.scan1.vip' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb2'
CRS-2676: Start of 'ora.scan1.vip' on 'zfzhlhrdb2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb2'
CRS-2676: Start of 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb2' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'zfzhlhrdb2'
CRS-2676: Start of 'ora.oc4j' on 'zfzhlhrdb2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.ons' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.net1.network' on 'zfzhlhrdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cssd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'zfzhlhrdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[zfzhlhrdb1:root]:/>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[zfzhlhrdb1:root]:/>crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE OFFLINE Instance Shutdown
ora.cluster_interconnect.haip
1 ONLINE OFFLINE
ora.crf
1 ONLINE ONLINE zfzhlhrdb1
ora.crsd
1 ONLINE OFFLINE
ora.cssd
1 ONLINE OFFLINE STARTING
ora.cssdmonitor
1 ONLINE ONLINE zfzhlhrdb1
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.drivers.acfs
1 ONLINE OFFLINE
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE zfzhlhrdb1
ora.gpnpd
1 ONLINE ONLINE zfzhlhrdb1
ora.mdnsd
1 ONLINE ONLINE zfzhlhrdb1
[zfzhlhrdb1:root]:/>
通过KFED命令修复ASM磁盘头
# kfed repair /dev/rhdisk2
[zfzhlhrdb1:root]:/>kfed repair /dev/rhdisk2
[zfzhlhrdb1:root]:/>
查看ASM磁盘头信息
[zfzhlhrdb1:root]:/>lquerypv -h /dev/rhdisk2
00000000 00820101 00000000 80000000 9D6A73D5 |.............js.|
00000010 00000000 00000000 00000000 00000000 |................|
00000020 4F52434C 4449534B 00000000 00000000 |ORCLDISK........|
00000030 00000000 00000000 00000000 00000000 |................|
00000040 0B200000 00000103 44415441 5F303030 |. ......DATA_000|
00000050 30000000 00000000 00000000 00000000 |0...............|
00000060 00000000 00000000 44415441 00000000 |........DATA....|
00000070 00000000 00000000 00000000 00000000 |................|
00000080 00000000 00000000 44415441 5F303030 |........DATA_000|
00000090 30000000 00000000 00000000 00000000 |0...............|
000000A0 00000000 00000000 00000000 00000000 |................|
000000B0 00000000 00000000 00000000 00000000 |................|
000000C0 00000000 00000000 01F81BD3 40FE1000 |............@...|
000000D0 01F81BD4 50600800 02001000 00100000 |....P`..........|
000000E0 0001BC80 0002001C 00000003 00000001 |................|
000000F0 00000002 00000002 00000000 00000000 |................|
[zfzhlhrdb1:root]:/>
启动HAS服务
# crsctl start has
查看服务启动信息
# crsctl stat res -t
[zfzhlhrdb1:root]:/>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.FRA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.LISTENER.lsnr
ONLINE OFFLINE zfzhlhrdb1 STARTING
ONLINE ONLINE zfzhlhrdb2
ora.LISTENER_LHRDG.lsnr
ONLINE OFFLINE zfzhlhrdb1 STARTING
ONLINE ONLINE zfzhlhrdb2
ora.asm
ONLINE ONLINE zfzhlhrdb1 Started
ONLINE ONLINE zfzhlhrdb2 Started
ora.gsd
OFFLINE OFFLINE zfzhlhrdb1
OFFLINE OFFLINE zfzhlhrdb2
ora.net1.network
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.ons
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.registry.acfs
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zfzhlhrdb2
ora.cvu
1 ONLINE ONLINE zfzhlhrdb2
ora.lhrdg.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE
ora.oc4j
1 ONLINE ONLINE zfzhlhrdb2
ora.oraesdb.db
1 ONLINE OFFLINE Corrupted Controlfi
le
2 ONLINE OFFLINE
ora.oraeskdb.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE
ora.oralhrq.db
1 ONLINE ONLINE zfzhlhrdb2 Open
2 ONLINE OFFLINE Instance Shutdown
ora.raclhr.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE
ora.scan1.vip
1 ONLINE ONLINE zfzhlhrdb2
ora.zfzhlhrdb1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb2.vip
1 ONLINE ONLINE zfzhlhrdb2
[zfzhlhrdb1:root]:/>
二.3.3 实验三：物理备份与恢复(自动备份)
ocrconfig -manualbackup
ocrconfig -showbackup
crsctl query css votedisk
crsctl stop crs -f
crsctl start crs -excl
crsctl stop resource ora.crsd -init
ocrconfig -restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
[ZFTPCCDB1:root]:/>su - grid
[ZFTPCCDB1:grid]:/home/grid>cluvfy comp ocr -n all -verbose
Verifying OCR integrity
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations
ASM Running check passed. ASM is running on all specified nodes
Checking OCR config file "/etc/oracle/ocr.loc"...
OCR config file "/etc/oracle/ocr.loc" check successful
Disk group for ocr location "+DATA1" available on all the nodes
NOTE:
This check does not verify the integrity of the OCR contents. Execute 'ocrcheck' as a privileged user to verify the contents of OCR.
OCR integrity check passed
Verification of OCR integrity was successful.
[ZFTPCCDB1:grid]:/home/grid>
[ZFTPCCDB1:grid]:/home/grid>
[ZFTPCCDB1:grid]:/home/grid>exit
[ZFTPCCDB1:root]:/>ocrconfig -showbackup
PROT-24: Auto backups for the Oracle Cluster Registry are not available
zftpccdb1 2016/07/01 15:23:58 /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
[ZFTPCCDB1:root]:/>ocrconfig -manualbackup
zftpccdb2 2016/07/01 16:08:22 /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_160822.ocr
zftpccdb1 2016/07/01 15:23:58 /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
[ZFTPCCDB1:root]:/>ocrconfig -showbackup
PROT-24: Auto backups for the Oracle Cluster Registry are not available
zftpccdb2 2016/07/01 16:08:22 /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_160822.ocr
zftpccdb1 2016/07/01 15:23:58 /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
[ZFTPCCDB1:root]:/>
2个节点都停掉CRS：
[ZFTPCCDB1:root]:/>crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 2da6f80ec3e64f45bfca9dabe0dd65eb (/dev/rhdisk1) [DATA1]
Located 1 voting disk(s).
[ZFTPCCDB1:root]:/>ocrconfig -restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
PROT-19: Cannot proceed while the Cluster Ready Service is running
[ZFTPCCDB1:root]:/>crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'zftpccdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.zftpccdb2.vip' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.cvu' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.oralhr.db' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zftpccdb1'
CRS-2677: Stop of 'ora.zftpccdb2.vip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.zftpccdb2.vip' on 'zftpccdb2'
CRS-2677: Stop of 'ora.cvu' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'zftpccdb2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'zftpccdb2'
CRS-2676: Start of 'ora.cvu' on 'zftpccdb2' succeeded
CRS-2677: Stop of 'ora.oralhr.db' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'zftpccdb1'
CRS-2676: Start of 'ora.zftpccdb2.vip' on 'zftpccdb2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER.lsnr' on 'zftpccdb2'
CRS-2676: Start of 'ora.scan1.vip' on 'zftpccdb2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'zftpccdb2'
CRS-2676: Start of 'ora.LISTENER.lsnr' on 'zftpccdb2' succeeded
CRS-2673: Attempting to stop 'ora.zftpccdb1.vip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.zftpccdb1.vip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.zftpccdb1.vip' on 'zftpccdb2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'zftpccdb2' succeeded
CRS-2677: Stop of 'ora.registry.acfs' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.zftpccdb1.vip' on 'zftpccdb2' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'zftpccdb2'
CRS-2676: Start of 'ora.oc4j' on 'zftpccdb2' succeeded
CRS-2677: Stop of 'ora.DATA1.dg' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'zftpccdb1'
CRS-2677: Stop of 'ora.asm' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'zftpccdb1'
CRS-2677: Stop of 'ora.ons' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'zftpccdb1'
CRS-2677: Stop of 'ora.net1.network' on 'zftpccdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zftpccdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.asm' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.cssd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[ZFTPCCDB1:root]:/>ps -ef|grep d.bin
root 4391306 6094924 0 16:31:03 pts/0 0:00 grep d.bin
[ZFTPCCDB1:root]:/>
[ZFTPCCDB1:root]:/>ocrconfig -restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
-- trace dump on error exit --
Error [kgfoAl06] in [kgfokge] at kgfo.c:1529
ORA-29701: unable to connect to Cluster Synchronization Service
Category: 7
DepInfo: 29701
ADR is not properly configured
-- trace dump end --
-- trace dump on error exit --
Error [kgfoAl06] in [kgfokge] at kgfo.c:1255
ORA-29701: unable to connect to Cluster Synchronization Service
Category: 7
DepInfo: 29701
ADR is not properly configured
-- trace dump end --
-- trace dump on error exit --
Error [kgfoAl06] in [kgfokge] at kgfo.c:2063
ORA-29701: unable to connect to Cluster Synchronization Service
Category: 7
DepInfo: 29701
ADR is not properly configured
-- trace dump end --
PROT-35: The configured Oracle Cluster Registry locations are not accessible
[ZFTPCCDB1:root]:/>crsctl start crs -excl
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'zftpccdb1'
CRS-2676: Start of 'ora.mdnsd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'zftpccdb1'
CRS-2676: Start of 'ora.gpnpd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'zftpccdb1'
CRS-2672: Attempting to start 'ora.gipcd' on 'zftpccdb1'
CRS-2676: Start of 'ora.cssdmonitor' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'zftpccdb1'
CRS-2672: Attempting to start 'ora.diskmon' on 'zftpccdb1'
CRS-2676: Start of 'ora.diskmon' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.cssd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'zftpccdb1'
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2676: Start of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'zftpccdb1'
CRS-2674: Start of 'ora.asm' on 'zftpccdb1' failed
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zftpccdb1'
tempting to stop 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-4000: Command Start failed, or completed with errors.
[ZFTPCCDB1:root]:/>
[ZFTPCCDB1:root]:/>crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE zftpccdb1 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE zftpccdb1
ora.crf
1 OFFLINE OFFLINE
ora.crsd
1 ONLINE INTERMEDIATE zftpccdb1 EXCLUSIVE
ora.cssd
1 ONLINE ONLINE zftpccdb1
ora.cssdmonitor
1 ONLINE ONLINE zftpccdb1
ora.ctssd
1 ONLINE ONLINE zftpccdb1 OBSERVER
ora.diskmon
1 OFFLINE OFFLINE
ora.drivers.acfs
1 ONLINE ONLINE zftpccdb1
ora.evmd
1 OFFLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE zftpccdb1
ora.gpnpd
1 ONLINE ONLINE zftpccdb1
ora.mdnsd
1 ONLINE ONLINE zftpccdb1
[ZFTPCCDB1:root]:/>ocrconfig -restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
PROT-19: Cannot proceed while the Cluster Ready Service is running
[ZFTPCCDB1:root]:/>crsctl stop resource ora.crsd -init
CRS-2673: Attempting to stop 'ora.crsd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.crsd' on 'zftpccdb1' succeeded
[ZFTPCCDB1:root]:/>ocrconfig -restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
[ZFTPCCDB1:root]:/>
[ZFTPCCDB1:root]:/>crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.cssd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zftpccdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[ZFTPCCDB1:root]:/>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[ZFTPCCDB1:root]:/>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.LISTENER.lsnr
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.asm
ONLINE ONLINE zftpccdb1 Started
ONLINE ONLINE zftpccdb2 Started
ora.gsd
OFFLINE OFFLINE zftpccdb1
OFFLINE OFFLINE zftpccdb2
ora.net1.network
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.ons
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.registry.acfs
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zftpccdb2
ora.cvu
1 ONLINE ONLINE zftpccdb2
ora.oc4j
1 ONLINE ONLINE zftpccdb2
ora.oralhr.db
1 ONLINE ONLINE zftpccdb1 Open
2 ONLINE ONLINE zftpccdb2 Open
ora.scan1.vip
1 ONLINE ONLINE zftpccdb2
ora.zftpccdb1.vip
1 ONLINE ONLINE zftpccdb1
ora.zftpccdb2.vip
1 ONLINE ONLINE zftpccdb2
[ZFTPCCDB1:root]:/>
成功恢复。
二.3.4 实验四：逻辑备份与恢复(手动备份)
ocrconfig -export /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
crsctl stop crs
crsctl start crs -excl -nocrs
ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
crsctl start crs
[ZFTPCCDB1:root]:/>ocrconfig -export /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
[ZFTPCCDB1:root]:/>ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
PROT-19: Cannot proceed while the Cluster Ready Service is running
[ZFTPCCDB1:root]:/>crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'zftpccdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.oralhr.db' on 'zftpccdb1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.zftpccdb1.vip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.zftpccdb1.vip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.zftpccdb1.vip' on 'zftpccdb2'
CRS-2677: Stop of 'ora.oralhr.db' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'zftpccdb1'
CRS-2676: Start of 'ora.zftpccdb1.vip' on 'zftpccdb2' succeeded
CRS-2677: Stop of 'ora.registry.acfs' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.DATA1.dg' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'zftpccdb1'
CRS-2677: Stop of 'ora.asm' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'zftpccdb1'
CRS-2677: Stop of 'ora.ons' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'zftpccdb1'
CRS-2677: Stop of 'ora.net1.network' on 'zftpccdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zftpccdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.asm' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.cssd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'zftpccdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[ZFTPCCDB1:root]:/>
[ZFTPCCDB1:root]:/>ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
-- trace dump on error exit --
Error [kgfoAl06] in [kgfokge] at kgfo.c:1529
ORA-15077: could not locate ASM instance serving a required diskgroup
Category: 7
DepInfo: 15077
ADR is not properly configured
-- trace dump end --
-- trace dump on error exit --
Error [kgfoAl06] in [kgfokge] at kgfo.c:1255
ORA-15077: could not locate ASM instance serving a required diskgroup
Category: 7
DepInfo: 15077
ADR is not properly configured
-- trace dump end --
-- trace dump on error exit --
Error [kgfoAl06] in [kgfokge] at kgfo.c:2063
ORA-15077: could not locate ASM instance serving a required diskgroup
Category: 7
DepInfo: 15077
ADR is not properly configured
-- trace dump end --
PROT-1: Failed to initialize ocrconfig
PROC-26: Error while accessing the physical storage
ORA-15077: could not locate ASM instance serving a required diskgroup
[ZFTPCCDB1:root]:/>
[ZFTPCCDB1:root]:/>crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'zftpccdb1'
CRS-2676: Start of 'ora.mdnsd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'zftpccdb1'
CRS-2676: Start of 'ora.gpnpd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'zftpccdb1'
CRS-2672: Attempting to start 'ora.gipcd' on 'zftpccdb1'
CRS-2676: Start of 'ora.cssdmonitor' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'zftpccdb1'
CRS-2672: Attempting to start 'ora.diskmon' on 'zftpccdb1'
CRS-2676: Start of 'ora.diskmon' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.cssd' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2672: Attempting to start 'ora.ctssd' on 'zftpccdb1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2676: Start of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'zftpccdb1'
CRS-2676: Start of 'ora.asm' on 'zftpccdb1' succeeded
[ZFTPCCDB1:root]:/>ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
[ZFTPCCDB1:root]:/>crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zftpccdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zftpccdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.asm' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zftpccdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'zftpccdb1' succeeded
CRS-2677: Stop of 'ora.cssd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zftpccdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zftpccdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zftpccdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zftpccdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[ZFTPCCDB1:root]:/>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[ZFTPCCDB1:root]:/>
[ZFTPCCDB1:root]:/>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.LISTENER.lsnr
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.asm
ONLINE ONLINE zftpccdb1 Started
ONLINE ONLINE zftpccdb2 Started
ora.gsd
OFFLINE OFFLINE zftpccdb1
OFFLINE OFFLINE zftpccdb2
ora.net1.network
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.ons
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
ora.registry.acfs
ONLINE ONLINE zftpccdb1
ONLINE ONLINE zftpccdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zftpccdb2
ora.cvu
1 ONLINE ONLINE zftpccdb1
ora.oc4j
1 ONLINE ONLINE zftpccdb1
ora.oralhr.db
1 ONLINE ONLINE zftpccdb1 Open
2 ONLINE ONLINE zftpccdb2 Open
ora.scan1.vip
1 ONLINE ONLINE zftpccdb2
ora.zftpccdb1.vip
1 ONLINE ONLINE zftpccdb1
ora.zftpccdb2.vip
1 ONLINE ONLINE zftpccdb2
[ZFTPCCDB1:root]:/>
成功恢复。
二.3.5 实验五：删除$ORACLE_HOME/log下的文件夹后的修复
在这个实验中，我们删除grid用户下的日志文件：$ORACLE_HOME/log，文件夹内容被删除导致集群不能启动。
[zfzhlhrdb1:root]:/>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.asm
ONLINE ONLINE zfzhlhrdb1 Started
ONLINE ONLINE zfzhlhrdb2 Started
ora.gsd
OFFLINE OFFLINE zfzhlhrdb1
OFFLINE OFFLINE zfzhlhrdb2
ora.net1.network
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.ons
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.registry.acfs
ONLINE OFFLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zfzhlhrdb1
ora.cvu
1 ONLINE ONLINE zfzhlhrdb1
ora.oc4j
1 ONLINE ONLINE zfzhlhrdb1
ora.oraesdb.db
1 ONLINE ONLINE zfzhlhrdb1 Open
2 ONLINE ONLINE zfzhlhrdb2 Open
ora.oraeskdb.db
1 ONLINE ONLINE zfzhlhrdb1 Open
2 ONLINE ONLINE zfzhlhrdb2 Open
ora.scan1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb2.vip
1 ONLINE ONLINE zfzhlhrdb2
[zfzhlhrdb1:root]:/>cd $ORACLE_HOME/log
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid/log>l
total 32
drwxr-xr-x 2 grid dba 256 Nov 05 2014 crs
drwxrwx--T 6 grid asmadmin 256 Nov 06 2014 diag
drwxr-xr-t 25 root dba 4096 Nov 19 2014 yjyltest2
drwxr-xr-t 25 root dba 4096 Jul 15 2015 zfmcisudb5
drwxr-xr-t 25 root dba 4096 Jul 01 10:51 zfzhlhrdb1
drwxr-xr-t 25 root dba 4096 Nov 05 2014 zt1nuwdb1
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid/log>cd ..
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>mv log log_bk
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'zfzhlhrdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.cvu' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oraesdb.db' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oraeskdb.db' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cvu' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'zfzhlhrdb2'
CRS-2676: Start of 'ora.cvu' on 'zfzhlhrdb2' succeeded
CRS-2677: Stop of 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.scan1.vip' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.oraeskdb.db' on 'zfzhlhrdb1' succeeded
CRS-2676: Start of 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'zfzhlhrdb2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.oraesdb.db' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'zfzhlhrdb1'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'zfzhlhrdb2' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.DATA.dg' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2676: Start of 'ora.oc4j' on 'zfzhlhrdb2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.ons' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.net1.network' on 'zfzhlhrdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cssd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zfzhlhrdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>crsctl start has
《《《《。。。。。。。。这里没有任何输出，我多次回车，只能ctrl+c退出。。。。。。。。》》》》
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>l log
total 0
drwxr-xr-x 4 root system 256 Jul 01 17:35 zfzhlhrdb1
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>
可以看到log文件夹自动创建，但是集群不能启动，log文件下的其他文件夹也不能自动创建，下边我们根据前边的理论知识来恢复由于权限问题导致的集群不能启动的问题，由于我们的环境是11.2.0.4，修复起来比较容易，执行脚本$GRID_HOME/crs/install/rootcrs.pl -init。
---执行速度很快。。。
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>$ORACLE_HOME/crs/install/rootcrs.pl -init
Using configuration parameter file: /oracle/app/11.2.0/grid/crs/install/crsconfig_params
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>l log
total 8
drwxrwx--T 3 grid asmadmin 256 Jul 01 17:40 diag
drwxr-xr-t 24 root dba 4096 Jul 01 17:40 zfzhlhrdb1
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>
可以看到crs可以启动了，等待一会可以看到资源全部启动：
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.asm
ONLINE ONLINE zfzhlhrdb1 Started
ONLINE ONLINE zfzhlhrdb2 Started
ora.gsd
OFFLINE OFFLINE zfzhlhrdb1
OFFLINE OFFLINE zfzhlhrdb2
ora.net1.network
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.ons
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.registry.acfs
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zfzhlhrdb2
ora.cvu
1 ONLINE ONLINE zfzhlhrdb2
ora.oc4j
1 ONLINE ONLINE zfzhlhrdb2
ora.oraesdb.db
1 ONLINE ONLINE zfzhlhrdb1 Open
2 ONLINE ONLINE zfzhlhrdb2 Open
ora.oraeskdb.db
1 ONLINE ONLINE zfzhlhrdb1 Open
2 ONLINE ONLINE zfzhlhrdb2 Open
ora.scan1.vip
1 ONLINE ONLINE zfzhlhrdb2
ora.zfzhlhrdb1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb2.vip
1 ONLINE ONLINE zfzhlhrdb2
[zfzhlhrdb1:root]:/oracle/app/11.2.0/grid>
集群基本已经恢复，且看实验六。
二.3.6 实验六：permission.pl脚本的使用
根据实验五的操作，执行命令$GRID_HOME/crs/install/rootcrs.pl -init，只是修复了$GRID_HOME下的为避免留下后遗症，我们需要将rootcrs.pl弃之不管的目录与文件的权限、属主也修复一下，怎么修复？MOS1515018.1提供了现成的perl脚本，这个脚本使用方法很简单：从一台权限正常的服务器上抓取GRID_HOME、ORACLE_HOME下的所有文件与目录权限，生成shell脚本，然后在权限错误的主机上执行这个脚本，简单演示一下：
<1> 先把permission.pl下载下来复制到一台权限正常的服务器上，并赋予执行权限，这台主机上必须要有perl的执行环境，我们这里直接在另外一台主机rac2上生成脚本，抓取GRID_HOME下的所有目录与文件的属主、权限,必须使用root用户执行：
[zfzhlhrdb2:root]:/>l /tmp/permission*
-rw-r----- 1 root system 2326 Jul 01 00:05 /tmp/permission.pl
[zfzhlhrdb2:root]:/>chmod 755 /tmp/permission.pl
[zfzhlhrdb2:root]:/>
[zfzhlhrdb2:root]:/>/tmp/permission.pl $ORACLE_HOME
Following log files are generated
logfile : permission-Fri-Jul-01-18-16-20-2016
Command file : restore-perm-Fri-Jul-01-18-16-20-2016.cmd
Linecount : 18126
[zfzhlhrdb2:root]:/>l *18-16-20-2016*
-rw-r--r-- 1 root system 1457428 Jul 01 18:16 permission-Fri-Jul-01-18-16-20-2016
-rw-r--r-- 1 root system 2928380 Jul 01 18:16 restore-perm-Fri-Jul-01-18-16-20-2016.cmd
[zfzhlhrdb2:root]:/>
生成了两个文件，其中permission*开头的是/oracle/app/oracle/product/11.2.0/db_1目录及其下的所有子目录与文件列表，例如：
[zfzhlhrdb2:root]:/>more permission-Fri-Jul-01-18-16-20-2016
755 root dba /oracle/app/11.2.0/grid
755 grid dba /oracle/app/11.2.0/grid/JRE
640 grid dba /oracle/app/11.2.0/grid/oraInst.loc
750 grid dba /oracle/app/11.2.0/grid/root.sh
755 grid dba /oracle/app/11.2.0/grid/rootupgrade.sh
750 grid dba /oracle/app/11.2.0/grid/.patch_storage
644 grid dba /oracle/app/11.2.0/grid/.patch_storage/LatestOPatchSession.properties
644 grid dba /oracle/app/11.2.0/grid/.patch_storage/interim_inventory.txt
644 grid dba /oracle/app/11.2.0/grid/.patch_storage/patch_free
644 grid dba /oracle/app/11.2.0/grid/.patch_storage/record_inventory.txt
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50
710 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/rollback.sh
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/bin
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/bin/lxinst
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libclient11.a
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libclient11.a/knoggcap.o
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libgeneric11.a
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libgeneric11.a/qcd.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libgeneric11.a/qcs.o
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libpls11.a
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libpls11.a/pevmexe.o
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libpls11_pic.a
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libpls11_pic.a/pevmexe_pic.o
755 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kcfis.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kf.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kfd.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kfds.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kff.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kjb.o
664 grid dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libserver11.a/kjbl.o
《《《《。。。。。。。。篇幅原因，有省略。。。。。。。。》》》》
restore*开头的包含了执行修改权限修复所需的脚本，例如：
[zfzhlhrdb2:root]:/>more restore-perm-Fri-Jul-01-18-16-20-2016.cmd
chown root:dba /oracle/app/11.2.0/grid
chmod 755 /oracle/app/11.2.0/grid
chown grid:dba /oracle/app/11.2.0/grid/JRE
chmod 755 /oracle/app/11.2.0/grid/JRE
chown grid:dba /oracle/app/11.2.0/grid/oraInst.loc
chmod 640 /oracle/app/11.2.0/grid/oraInst.loc
chown grid:dba /oracle/app/11.2.0/grid/root.sh
chmod 750 /oracle/app/11.2.0/grid/root.sh
chown grid:dba /oracle/app/11.2.0/grid/rootupgrade.sh
chmod 755 /oracle/app/11.2.0/grid/rootupgrade.sh
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage
chmod 750 /oracle/app/11.2.0/grid/.patch_storage
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/LatestOPatchSession.properties
chmod 644 /oracle/app/11.2.0/grid/.patch_storage/LatestOPatchSession.properties
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/interim_inventory.txt
chmod 644 /oracle/app/11.2.0/grid/.patch_storage/interim_inventory.txt
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/patch_free
chmod 644 /oracle/app/11.2.0/grid/.patch_storage/patch_free
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/record_inventory.txt
chmod 644 /oracle/app/11.2.0/grid/.patch_storage/record_inventory.txt
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50
chmod 755 /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/rollback.sh
chmod 710 /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/rollback.sh
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files
chmod 755 /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/bin
chmod 755 /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/bin
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/bin/lxinst
chmod 755 /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/bin/lxinst
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib
chmod 755 /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib
chown grid:dba /oracle/app/11.2.0/grid/.patch_storage/17478514_Dec_30_2013_03_38_50/files/lib/libclient11.a
《《《《。。。。。。。。篇幅原因，有省略。。。。。。。。》》》》
<3> 抓取$ORACLE_HOME下所有目录与文件的属主、权限，可以使用oracle或者root用户
[zfzhlhrdb2:root]:/>su - oracle
[zfzhlhrdb2:oracle]:/oracle>echo $ORACLE_HOME
/oracle/app/oracle/product/11.2.0/db
[zfzhlhrdb2:oracle]:/oracle>exit
[zfzhlhrdb2:root]:/>/tmp/permission.pl /oracle/app/oracle/product/11.2.0/db
Following log files are generated
logfile : permission-Fri-Jul-01-18-21-50-2016
Command file : restore-perm-Fri-Jul-01-18-21-50-2016.cmd
Linecount : 41627
[zfzhlhrdb2:root]:/>
<4> 将生成的4个文件copy到目标主机上，在目标主机上执行restore*开头的两个脚本，root用户执行
[zfzhlhrdb1:root]:/permissions>l
total 32600
-rw-r----- 1 root system 1457428 Jul 01 18:16 permission-Fri-Jul-01-18-16-20-2016
-rw-r----- 1 root system 4114174 Jul 01 18:22 permission-Fri-Jul-01-18-21-50-2016
-rw-r----- 1 root system 2928380 Jul 01 18:16 restore-perm-Fri-Jul-01-18-16-20-2016.cmd
-rw-r----- 1 root system 8184311 Jul 01 18:22 restore-perm-Fri-Jul-01-18-21-50-2016.cmd
[zfzhlhrdb1:root]:/permissions>chmod 755 *.cmd
[zfzhlhrdb1:root]:/permissions>l
total 32600
-rw-r----- 1 root system 1457428 Jul 01 18:16 permission-Fri-Jul-01-18-16-20-2016
-rw-r----- 1 root system 4114174 Jul 01 18:22 permission-Fri-Jul-01-18-21-50-2016
-rwxr-xr-x 1 root system 2928380 Jul 01 18:16 restore-perm-Fri-Jul-01-18-16-20-2016.cmd
-rwxr-xr-x 1 root system 8184311 Jul 01 18:22 restore-perm-Fri-Jul-01-18-21-50-2016.cmd
[zfzhlhrdb1:root]:/permissions>./restore-perm-Fri-Jul-01-18-16-20-2016.cmd
chown: /oracle/app/11.2.0/grid/auth/crs/zfzhlhrdb2: A file or directory in the path name does not exist.
chmod: /oracle/app/11.2.0/grid/auth/crs/zfzhlhrdb2: A file or directory in the path name does not exist.
chown: /oracle/app/11.2.0/grid/auth/css/zfzhlhrdb2: A file or directory in the path name does not exist.
chmod: /oracle/app/11.2.0/grid/auth/css/zfzhlhrdb2: A file or directory in the path name does not exist.
chown: /oracle/app/11.2.0/grid/auth/evm/zfzhlhrdb2: A file or directory in the path name does not exist.
。。。。。。。。。。。。。
[zfzhlhrdb1:root]:/permissions>./restore-perm-Fri-Jul-01-18-21-50-2016.cmd
chown: /oracle/app/oracle/product/11.2.0/db/ccr/state/ORADEDB-RAC.ccr: A file or directory in the path name does not exist.
chmod: /oracle/app/oracle/product/11.2.0/db/ccr/state/ORADEDB-RAC.ccr: A file or directory in the path name does not exist.
chown: /oracle/app/oracle/product/11.2.0/db/ccr/state/ORADEDB-RAC.ll: A file or directory in the path name does not exist.
chmod: /oracle/app/oracle/product/11.2.0/db/ccr/state/ORADEDB-RAC.ll: A file or directory in the path name does not exist.
chown: /oracle/app/oracle/product/11.2.0/db/ccr/state/ORADEDB-RAC.ll-stat: A file or directory in the path name does not exist.
chmod: /oracle/app/oracle/product/11.2.0/db/ccr/state/ORADEDB-RAC.ll-stat: A file or directory in the path name does not exist.
chown: /oracle/app/oracle/product/11.2.0/db/cfgtoollogs/opatch/opatch2016-07-01_12-50-49PM_1.log: A file or directory in the path name does not exist.
chmod: /oracle/app/oracle/product/11.2.0/db/cfgtoollogs/opatch/opatch2016-07-01_12-50-49PM_1.log: A file or directory in the path name does not exist.
chown: /oracle/app/oracle/product/11.2.0/db/dbs/hc_oraESDB2.dat: A file or directory in the path name does not exist.
。。。。。。。。。。。。。。。。。
启动和关闭集群试试：
[zfzhlhrdb1:root]:/>crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'zfzhlhrdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.LISTENER_LHRDG.lsnr' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oraesdb.db' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.oraeskdb.db' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.LISTENER_LHRDG.lsnr' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.oraesdb.db' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb1' succeeded
CRS-2672: Attempting to start 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb2'
CRS-2677: Stop of 'ora.registry.acfs' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.oraeskdb.db' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'zfzhlhrdb1'
CRS-2676: Start of 'ora.zfzhlhrdb1.vip' on 'zfzhlhrdb2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.ons' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.net1.network' on 'zfzhlhrdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zfzhlhrdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'zfzhlhrdb1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'zfzhlhrdb1' succeeded
crsctl start has
CRS-2677: Stop of 'ora.asm' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.cssd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gipcd' on 'zfzhlhrdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'zfzhlhrdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'zfzhlhrdb1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'zfzhlhrdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zfzhlhrdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[zfzhlhrdb1:root]:/>crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[zfzhlhrdb1:root]:/>crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.LISTENER_LHRDG.lsnr
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.asm
ONLINE ONLINE zfzhlhrdb1 Started
ONLINE ONLINE zfzhlhrdb2 Started
ora.gsd
OFFLINE OFFLINE zfzhlhrdb1
OFFLINE OFFLINE zfzhlhrdb2
ora.net1.network
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.ons
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
ora.registry.acfs
ONLINE ONLINE zfzhlhrdb1
ONLINE ONLINE zfzhlhrdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE zfzhlhrdb1
ora.cvu
1 ONLINE ONLINE zfzhlhrdb1
ora.oc4j
1 ONLINE ONLINE zfzhlhrdb1
ora.oraesdb.db
1 ONLINE ONLINE zfzhlhrdb1 Open
2 ONLINE ONLINE zfzhlhrdb2 Open
ora.oraeskdb.db
1 ONLINE ONLINE zfzhlhrdb1 Open
2 ONLINE ONLINE zfzhlhrdb2 Open
ora.scan1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb1.vip
1 ONLINE ONLINE zfzhlhrdb1
ora.zfzhlhrdb2.vip
1 ONLINE ONLINE zfzhlhrdb2
[zfzhlhrdb1:root]:/>
好，搞定，结束，收尾，回家。
二.4 实验总结
本篇blog内容较多，主要包括OCR的备份恢复及其GRID_HOME目录的权限修复2大类问题，实验中的dd命令记得一定要在测试库实验，记得多种备份都要有。
第三章 实验中用到的SQL总结
ocrconfig -export /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm.bak
dd if=/dev/rhdisk1 of=/oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_rhdisk1_dd.bak bs=1024k count=4
asmcmd md_backup /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_md_backup.bak
ocrconfig -manualbackup
ocrconfig -showbackup
ocrconfig -export /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm.bak
ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm.bak
dd if=/dev/rhdisk1 of=/oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_rhdisk1_dd.bak bs=1024k count=4
dd if=/oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_rhdisk1_dd.bak of=/dev/rhdisk1 bs=1024k count=4
kfed repair /dev/rhdisk1
asmcmd md_backup /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_md_backup.bak
asmcmd md_restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/asm_md_backup.bak
ocrconfig -manualbackup
ocrconfig -showbackup
crsctl query css votedisk
crsctl stop crs -f
crsctl start crs -excl
crsctl stop resource ora.crsd -init
ocrconfig -restore /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/backup_20160701_152358.ocr
cluvfy comp ocr -n all -verbose
ocrconfig -export /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
crsctl stop crs
crsctl start crs -excl -nocrs
ocrconfig -import /oracle/app/11.2.0/grid/cdata/ZFTPCCDB-crs/export_asm_lhr.bak
crsctl start crs
$ORACLE_HOME/crs/install/rootcrs.pl -init