环境介绍，
esxi6.0 ，VMware vSphere Client6.0
linux 版本Centos7.6（最小化安装）
Oracle 版本 oracle 11g 11.2.0.4
虚拟化环境搭建，
资源下载：
链接：https://pan.baidu.com/s/1AjXAPi2OBCm3pA88EtDi0Q
提取码：1111
个别机型去要第三方驱动，而在官方又找不到时候，可以选择ESXi-Customizer加载第三方驱动
工具下载链接如下：
链接：https://pan.baidu.com/s/1Ta5ZZK6KtSe05Q1LyPtU6A
提取码：1111
安装步骤省略：
1.配置共享存储，
esxi配置ssh远程连接
我这里配置的与主机一起启动，如果ssh没有启动，点击启动即可
打开ssh连接工具，这里我使用的xshell，截图如下
创建共享磁盘时候忘记截图了，就截图做完毕之后的如下
创建共享磁盘命令如下，（一共是7块，）
vmkfstools -c 10240m -a lsilogic -d eagerzeroedthick mysharedisk_01.vmdk
vmkfstools -c 10240m -a lsilogic -d eagerzeroedthick mysharedisk_02.vmdk
vmkfstools -c 10240m -a lsilogic -d eagerzeroedthick mysharedisk_03.vmdk
vmkfstools -c 10240m -a lsilogic -d eagerzeroedthick mysharedisk_04.vmdk
vmkfstools -c 1024m -a lsilogic -d eagerzeroedthick mysharedisk_05.vmdk
vmkfstools -c 1024m -a lsilogic -d eagerzeroedthick mysharedisk_06.vmdk
vmkfstools -c 1024m -a lsilogic -d eagerzeroedthick mysharedisk_07.vmdk
创建成功后，打开VMware vSphere Client，进入数据存储浏览器打开如下图：
两个linux节点，虚拟机中配置如下：
这里列出rac1的配置：（两台Linux配置相同）
虚拟设备是scsi0:虚拟机中安装linux所用的磁盘。
虚拟设备是scsi1:共享磁盘，下图中硬盘2到硬盘8.
虚拟设备scsi0配置如下，默认即可，scsi总线共享为无，
虚拟设备scsi1配置如下
scsi控制器1配置属性scsi总线共享为：虚拟（修改需要熏鸡关机情况下配置修改）
两台linux 相同的配置如上图。完毕。
两台机器时间同步配置，虚拟化就比较简单了，
如下操作：
将同步客户机时间与主机时间打勾，重启两台虚拟机。
2，安装linux ,选择最小化安装，虚拟机配置双网卡，挂载安装镜像，
配置yum源为本地源
挂载操作系统光盘文件
mkdir /cdrom
mount /dev/cdrom /cdrom
cd /etc/yum.repos.d
mkdir bak
mv *.* bak
编辑yum 配置文件CentOS-Base.repo配置如下：
vi CentOS-Base.repo添加如下
[base]
name=CentOS
baseurl=file:///cdrom
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
联网环境也可使用oracle官方源
使用命令下载:
wget http://public-yum.oracle.com/public-yum-ol7.repo
安装net-tools
此步骤省略，安装net-tools，只是为了使用ifconfig 命令
yum inatall net-tools* -y
安装unzip ,解压oracle安装包使用unzip命令
yum install unzip -y
linux 配置裸设备，
两节点
执行fdisk -l /dev/sdb创建分区
编辑配置文件/usr/lib/udev/rules.d/60-raw.rules
[root@rac1 ~]# ls /usr/lib/udev/rules.d/60-raw.rules
/usr/lib/udev/rules.d/60-raw.rules
[root@rac1 ~]# cat /usr/lib/udev/rules.d/60-raw.rules
#
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/usr/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/usr/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", KERNEL=="sdc1", RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", KERNEL=="sdd1", RUN+="/bin/raw /dev/raw/raw3 %N"
ACTION=="add", KERNEL=="sde1", RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add", KERNEL=="sdf1", RUN+="/bin/raw /dev/raw/raw5 %N"
ACTION=="add", KERNEL=="sdg1", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdh1", RUN+="/bin/raw /dev/raw/raw7 %N"
KERNEL=="raw[1]", MODE="0660", OWNER="grid", GROUP="asmadmin"
KERNEL=="raw[2]", MODE="0660", OWNER="grid", GROUP="asmadmin"
KERNEL=="raw[3]", MODE="0660", OWNER="grid", GROUP="asmadmin"
KERNEL=="raw[4]", MODE="0660", OWNER="grid", GROUP="asmadmin"
KERNEL=="raw[5]", MODE="0660", OWNER="grid", GROUP="asmadmin"
KERNEL=="raw[6]", MODE="0660", OWNER="grid", GROUP="asmadmin"
KERNEL=="raw[7]", MODE="0660", OWNER="grid", GROUP="asmadmin"
[root@rac1 ~]#
重启系统后看到/dev/raw/下出现新配的文件
[root@rac1 ~]# ls -l /dev/raw/
total 0
crw-rw---- 1 grid asmadmin 162, 1 Jul 15 21:23 raw1
crw-rw---- 1 grid asmadmin 162, 2 Jul 14 20:45 raw2
crw-rw---- 1 grid asmadmin 162, 3 Jul 14 20:45 raw3
crw-rw---- 1 grid asmadmin 162, 4 Jul 14 20:45 raw4
crw-rw---- 1 grid asmadmin 162, 5 Jul 14 20:45 raw5
crw-rw---- 1 grid asmadmin 162, 6 Jul 14 20:45 raw6
crw-rw---- 1 grid asmadmin 162, 7 Jul 14 20:45 raw7
crw-rw---- 1 root disk 162, 0 Jul 14 20:45 rawctl
3，下载oracle 安装包，grid 安装包，准备红帽操作系统或者oracle linux 操作系统，（oracle 和grid 需要的包在centos7会缺少compat-libstdc++ ，红帽镜像中有）
本文中使用的oracle 11.2.0.4 安装包及补丁包如下
百度云链接：
链接：https://pan.baidu.com/s/1WvdpiTs9m3es5vyTBzHb4w
提取码：1111
操作系统下载链接：
centos
http://mirrors.sohu.com/centos/7.6.1810/isos/x86_64/
redhat
链接：https://pan.baidu.com/s/16-TgKVk_nAxLeakRsOPKkg
提取码：1111
4,安装grid，创建asm，前配置及规划
环境配置
rac1
网卡1：网卡名：ens192配置如下：
public ip :192.168.2.101
网卡1：网卡名：ens224配置如下：
private vip :10.10.10.2
配置文件省略。。。。。。
rac2
网卡1：网卡名：ens192配置如下：
public ip :192.168.2.102
网卡1：网卡名：ens224配置如下
private vip :10.10.10.3
配置文件省略。。。。。。
两个节点都要配置
修改hosts 文件添加ip,及安装Oracle后生成的虚拟ip
192.168.2.103-104，192.168.2.105 为安装后Oracle配置的ip，
vi /etc/hosts
192.168.2.101 rac1
192.168.2.102 rac2
192.168.2.103 rac1-vip
192.168.2.104 rac2-vip
10.10.10.2 rac1-priv
10.10.10.3 rac2-priv
192.168.2.105 rac-san
安装grid必须的包
yum install -y binutils*
yum install -y compat-libcap1*
yum install -y gcc*
yum install -y gcc-c++*
yum install -y glibc*
yum install -y glibc-devel*
yum install -y ksh*
yum install -y libaio*
yum install -y libgcc*
yum install -y libstdc*
yum install -y libstdc++-devel*
yum install -y libXi*
yum install -y libXtst*
yum install -y make*
yum install -y sysstat*
（环境检查时候会缺少pdksh包，忽略即可）
创建安装目录，创建永辉
/usr/sbin/groupadd -g 1000 oinstall
/usr/sbin/groupadd -g 1020 asmadmin
/usr/sbin/groupadd -g 1021 asmdba
/usr/sbin/groupadd -g 1022 asmoper
/usr/sbin/groupadd -g 1031 dba
/usr/sbin/groupadd -g 1032 oper
useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
useradd -u 1101 -g oinstall -G dba,asmdba,oper oracle
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/grid
mkdir /u01/app/oracle
chown -R grid:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/
最后使用root用户更改grid,oracle 用户的密码
passwd oracle
passwd grid
配置oracle用户，grid用户环境变量
节点1上配置
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=+ASM1
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.0/grid
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
节点2
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=+ASM2
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.0/grid
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
oracle 环境变量
节点1
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=rac1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export TNS_ADMIN=$ORACLE_HOME/network/admin
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
节点2
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=rac2
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export TNS_ADMIN=$ORACLE_HOME/network/admin
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
保存完毕，执行source命令，生效环境变量。
配置两台linux rac1,rac2互信，包括Oracle，grid 用户
oracle 用户和grid 用户大致相同
步骤如下
rac1下操作
ssh-keygen -t rsa
回车默认
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
rac2下操作
ssh-keygen -t rsa
回车默认
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
scp .ssh/authorized_keys rac2:
在rac2上操作
cat authorized_keys >> .ssh/authorized_keys
执行
scp .ssh/authorized_keys rac1:
rac1上操作
cat authorized_keys >.ssh/authorized_keys
rac1互信配置完毕配置文件，权限如下
[grid@rac1 ~]$ ll .ssh/ -d
drwx------ 2 grid oinstall 80 Jul 13 05:02 .ssh/
[grid@rac1 ~]$ ll .ssh/
total 16
-rw-r--r-- 1 grid oinstall 782 Jul 13 05:04 authorized_keys
-rw------- 1 grid oinstall 1675 Jul 13 05:01 id_rsa
-rw-r--r-- 1 grid oinstall 391 Jul 13 05:01 id_rsa.pub
-rw-r--r-- 1 grid oinstall 360 Jul 13 05:02 known_hosts
[grid@rac1 ~]$ cat .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVuiEr9Dug/tLJ+rZbdUr0ZDIlviaRPAoKcPoPf2EBCCRuYa3O6sqcOqKlwMNbitRl6eTaGxktVFOru7r2AG+DlajNaZo1B5jizx+atKCZzxoils8XGZNEPbEbNZN/NwgYEq2DYv1RkjcRXMvvPhEKskkBV3GT6BOPow0YTsRwajBmaLSamhg7fHnBTkzV6cjhDlLnLCyPevvlFMcKm1Y338ApSQMNBaPO9DhprYhUaEbUK8SpqZVOOWKHpAsFqc2iAPdnXJX36W/pc+dZf/FzuaJU1bPz6nzizEBB1zsnY3MWopKRt2j6TzjHpO4HLWCLDdJLYIMsFDCL0kJEIgl/ grid@rac1
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOTMVA9hhXpMZQX9YK282J0QWxCawAwZCP/AH3zIuXYXA/q5GHYhnFoG2xAM7wt9Vf2+LXYKV0QrVBBpZAlS9MzCna2upqqfB32QtBBMeAIxyBE++9ufPUz+QgvfkpET5YA9vxgm1plX0jB5OftO0YXXQzE5m0+2TI+WRsDCSpqZJTZzfZr0ncLfT6olguTwok70ls4dv+VLDW5RyG4yef8T5+PEdQa9igPD8iqsrAcunmhzsf+Hroz7Mkkl7IiFrFZc3OOPEkOa1GK3doOZeuu75fEHDd89IEMfDPJdW5Hfv+xOTRBd3mOv0jn/oiL+9wxP9TqfamJ9XRZpUpDFeR grid@rac2
[grid@rac1 ~]$
rac2上rac1互信配置完毕配置文件，权限如下
[grid@rac2 ~]$ ll .ssh/ -d
drwx------. 2 grid oinstall 80 Jul 12 23:23 .ssh/
[grid@rac2 ~]$ ll .ssh/
total 16
-rw-r--r--. 1 grid oinstall 782 Jul 13 05:04 authorized_keys
-rw-------. 1 grid oinstall 1679 Jul 12 23:22 id_rsa
-rw-r--r--. 1 grid oinstall 391 Jul 12 23:22 id_rsa.pub
-rw-r--r--. 1 grid oinstall 360 Jul 13 05:02 known_hosts
[grid@rac2 ~]$ cat .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVuiEr9Dug/tLJ+rZbdUr0ZDIlviaRPAoKcPoPf2EBCCRuYa3O6sqcOqKlwMNbitRl6eTaGxktVFOru7r2AG+DlajNaZo1B5jizx+atKCZzxoils8XGZNEPbEbNZN/NwgYEq2DYv1RkjcRXMvvPhEKskkBV3GT6BOPow0YTsRwajBmaLSamhg7fHnBTkzV6cjhDlLnLCyPevvlFMcKm1Y338ApSQMNBaPO9DhprYhUaEbUK8SpqZVOOWKHpAsFqc2iAPdnXJX36W/pc+dZf/FzuaJU1bPz6nzizEBB1zsnY3MWopKRt2j6TzjHpO4HLWCLDdJLYIMsFDCL0kJEIgl/ grid@rac1
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOTMVA9hhXpMZQX9YK282J0QWxCawAwZCP/AH3zIuXYXA/q5GHYhnFoG2xAM7wt9Vf2+LXYKV0QrVBBpZAlS9MzCna2upqqfB32QtBBMeAIxyBE++9ufPUz+QgvfkpET5YA9vxgm1plX0jB5OftO0YXXQzE5m0+2TI+WRsDCSpqZJTZzfZr0ncLfT6olguTwok70ls4dv+VLDW5RyG4yef8T5+PEdQa9igPD8iqsrAcunmhzsf+Hroz7Mkkl7IiFrFZc3OOPEkOa1GK3doOZeuu75fEHDd89IEMfDPJdW5Hfv+xOTRBd3mOv0jn/oiL+9wxP9TqfamJ9XRZpUpDFeR grid@rac2
[grid@rac2 ~]$
方法供参考，目的是两台机器的密钥文件写入到authorized_keys，拷贝到两台机器上，并且一致
互信测试执行
rac1
[grid@rac1 ~]$ ssh rac1
Last login: Wed Jul 15 03:30:57 2020
[grid@rac1 ~]$
[grid@rac1 ~]$ ssh rac2
Last login: Wed Jul 15 03:04:23 2020 from rac1
[grid@rac2 ~]$
rac2
grid@rac2 ~]$ ssh rac2
Last login: Wed Jul 15 03:44:49 2020 from rac1
[grid@rac2 ~]$ ssh rac1
Last login: Wed Jul 15 03:44:31 2020 from rac1
[grid@rac1 ~]$
如上输出即可正常。
配置完毕执行grid环境检查脚本
[grid@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "rac1"
Destination Node Reachable?
------------------------------------ ------------------------
rac2 yes
rac1 yes
Result: Node reachability check passed from node "rac1"
Checking user equivalence...
Check: User equivalence for user "grid"
Node Name Status
------------------------------------ ------------------------
rac2 passed
rac1 failed
Result: PRVF-4007 : User equivalence check failed for user "grid"
WARNING:
User equivalence is not set for nodes:
rac1
Verification will proceed with nodes:
rac2
Checking node connectivity...
Checking hosts config file...
Node Name Status
------------------------------------ ------------------------
rac2 passed
Verification of the hosts config file successful
Interface information for node "rac2"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
ens192 192.168.2.102 192.168.2.0 0.0.0.0 10.10.10.1 00:0C:29:9D:C5:91 1500
ens224 10.10.10.3 10.10.10.0 0.0.0.0 10.10.10.1 00:0C:29:9D:C5:9B 1500
Check: Node connectivity of subnet "192.168.2.0"
Result: Node connectivity passed for subnet "192.168.2.0" with node(s) rac2
Check: TCP connectivity of subnet "192.168.2.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
rac1:192.168.2.101 rac2:192.168.2.102 failed
ERROR:
PRVF-7617 : Node connectivity between "rac1 : 192.168.2.101" and "rac2 : 192.168.2.102" failed
Result: TCP connectivity check failed for subnet "192.168.2.0"
Check: Node connectivity of subnet "10.10.10.0"
Result: Node connectivity passed for subnet "10.10.10.0" with node(s) rac2
Check: TCP connectivity of subnet "10.10.10.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
rac1:192.168.2.101 rac2:10.10.10.3 failed
ERROR:
PRVF-7617 : Node connectivity between "rac1 : 192.168.2.101" and "rac2 : 10.10.10.3" failed
Result: TCP connectivity check failed for subnet "10.10.10.0"
Interfaces found on subnet "10.10.10.0" that are likely candidates for VIP are:
rac2 ens224:10.10.10.3
WARNING:
Could not find a suitable set of interfaces for the private interconnect
Result: Node connectivity check failed
Checking multicast communication...
Checking subnet "192.168.2.0" for multicast communication with multicast group "230.0.1.0"...
PRVG-11134 : Interface "192.168.2.102" on node "rac2" is not able to communicate with interface "192.168.2.102" on node "rac2"
Checking subnet "192.168.2.0" for multicast communication with multicast group "224.0.0.251"...
PRVG-11134 : Interface "192.168.2.102" on node "rac2" is not able to communicate with interface "192.168.2.102" on node "rac2"
Checking subnet "10.10.10.0" for multicast communication with multicast group "230.0.1.0"...
PRVG-11134 : Interface "10.10.10.3" on node "rac2" is not able to communicate with interface "10.10.10.3" on node "rac2"
Checking subnet "10.10.10.0" for multicast communication with multicast group "224.0.0.251"...
PRVG-11134 : Interface "10.10.10.3" on node "rac2" is not able to communicate with interface "10.10.10.3" on node "rac2"
Checking ASMLib configuration.
Node Name Status
------------------------------------ ------------------------
rac2 passed
Result: Check for ASMLib configuration passed.
Check: Total memory
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 1.7954GB (1882624.0KB) 1.5GB (1572864.0KB) passed
Result: Total memory check passed
Check: Available memory
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 1.5934GB (1670784.0KB) 50MB (51200.0KB) passed
Result: Available memory check passed
Check: Swap space
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 4.0039GB (4198392.0KB) 2.6931GB (2823936.0KB) passed
Result: Swap space check passed
Check: Free disk space for "rac2:/tmp"
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/tmp rac2 / 4.54GB 1GB passed
Result: Free disk space check passed for "rac2:/tmp"
Check: User existence for "grid"
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed exists(1100)
Checking for multiple users with UID value 1100
Result: Check for multiple users with UID value 1100 passed
Result: User existence check passed for "grid"
Check: Group existence for "oinstall"
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed exists
Result: Group existence check passed for "oinstall"
Check: Group existence for "dba"
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed exists
Result: Group existence check passed for "dba"
Check: Membership of user "grid" in group "oinstall" [as Primary]
Node Name User Exists Group Exists User in Group Primary Status
---------------- ------------ ------------ ------------ ------------ ------------
rac2 yes yes yes yes passed
Result: Membership check for user "grid" in group "oinstall" [as Primary] passed
Check: Membership of user "grid" in group "dba"
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
rac2 yes yes yes passed
Result: Membership check for user "grid" in group "dba" passed
Check: Run level
Node Name run level Required Status
------------ ------------------------ ------------------------ ----------
rac2 3 3,5 passed
Result: Run level check passed
Check: Hard limits for "maximum open file descriptors"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 hard 65536 65536 passed
Result: Hard limits check passed for "maximum open file descriptors"
Check: Soft limits for "maximum open file descriptors"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 soft 1024 1024 passed
Result: Soft limits check passed for "maximum open file descriptors"
Check: Hard limits for "maximum user processes"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 hard 16384 16384 passed
Result: Hard limits check passed for "maximum user processes"
Check: Soft limits for "maximum user processes"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 soft 2047 2047 passed
Result: Soft limits check passed for "maximum user processes"
Check: System architecture
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 x86_64 x86_64 passed
Result: System architecture check passed
Check: Kernel version
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 3.10.0-957.el7.x86_64 2.6.9 passed
Result: Kernel version check passed
Check: Kernel parameter for "semmsl"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 250 250 250 passed
Result: Kernel parameter check passed for "semmsl"
Check: Kernel parameter for "semmns"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 32000 32000 32000 passed
Result: Kernel parameter check passed for "semmns"
Check: Kernel parameter for "semopm"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 100 100 100 passed
Result: Kernel parameter check passed for "semopm"
Check: Kernel parameter for "semmni"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 128 128 128 passed
Result: Kernel parameter check passed for "semmni"
Check: Kernel parameter for "shmmax"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 2002012160 2002012160 963903488 passed
Result: Kernel parameter check passed for "shmmax"
Check: Kernel parameter for "shmmni"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 4096 4096 4096 passed
Result: Kernel parameter check passed for "shmmni"
Check: Kernel parameter for "shmall"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 2097152 2097152 2097152 passed
Result: Kernel parameter check passed for "shmall"
Check: Kernel parameter for "file-max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 6815744 6815744 6815744 passed
Result: Kernel parameter check passed for "file-max"
Check: Kernel parameter for "ip_local_port_range"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed
Result: Kernel parameter check passed for "ip_local_port_range"
Check: Kernel parameter for "rmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 262144 262144 262144 passed
Result: Kernel parameter check passed for "rmem_default"
Check: Kernel parameter for "rmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 4194304 4194304 4194304 passed
Result: Kernel parameter check passed for "rmem_max"
Check: Kernel parameter for "wmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 262144 262144 262144 passed
Result: Kernel parameter check passed for "wmem_default"
Check: Kernel parameter for "wmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 1048586 1048586 1048576 passed
Result: Kernel parameter check passed for "wmem_max"
Check: Kernel parameter for "aio-max-nr"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 1048576 1048576 1048576 passed
Result: Kernel parameter check passed for "aio-max-nr"
Check: Package existence for "make"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 make-3.82-23.el7 make-3.80 passed
Result: Package existence check passed for "make"
Check: Package existence for "binutils"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 binutils-2.27-34.base.el7 binutils-2.15.92.0.2 passed
Result: Package existence check passed for "binutils"
Check: Package existence for "gcc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 gcc(x86_64)-4.8.5-36.el7 gcc(x86_64)-3.4.6 passed
Result: Package existence check passed for "gcc(x86_64)"
Check: Package existence for "libaio(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.105 passed
Result: Package existence check passed for "libaio(x86_64)"
Check: Package existence for "glibc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc(x86_64)-2.17-260.el7 glibc(x86_64)-2.3.4-2.41 passed
Result: Package existence check passed for "glibc(x86_64)"
Check: Package existence for "compat-libstdc++-33(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 missing compat-libstdc++-33(x86_64)-3.2.3 failed
Result: Package existence check failed for "compat-libstdc++-33(x86_64)"
Check: Package existence for "elfutils-libelf(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 elfutils-libelf(x86_64)-0.172-2.el7 elfutils-libelf(x86_64)-0.97 passed
Result: Package existence check passed for "elfutils-libelf(x86_64)"
Check: Package existence for "elfutils-libelf-devel"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 missing elfutils-libelf-devel-0.97 failed
Result: Package existence check failed for "elfutils-libelf-devel"
Check: Package existence for "glibc-common"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc-common-2.17-260.el7 glibc-common-2.3.4 passed
Result: Package existence check passed for "glibc-common"
Check: Package existence for "glibc-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc-devel(x86_64)-2.17-260.el7 glibc-devel(x86_64)-2.3.4 passed
Result: Package existence check passed for "glibc-devel(x86_64)"
Check: Package existence for "glibc-headers"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc-headers-2.17-260.el7 glibc-headers-2.3.4 passed
Result: Package existence check passed for "glibc-headers"
Check: Package existence for "gcc-c++(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 gcc-c++(x86_64)-4.8.5-36.el7 gcc-c++(x86_64)-3.4.6 passed
Result: Package existence check passed for "gcc-c++(x86_64)"
Check: Package existence for "libaio-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.105 passed
Result: Package existence check passed for "libaio-devel(x86_64)"
Check: Package existence for "libgcc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libgcc(x86_64)-4.8.5-36.el7 libgcc(x86_64)-3.4.6 passed
Result: Package existence check passed for "libgcc(x86_64)"
Check: Package existence for "libstdc++(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libstdc++(x86_64)-4.8.5-36.el7 libstdc++(x86_64)-3.4.6 passed
Result: Package existence check passed for "libstdc++(x86_64)"
Check: Package existence for "libstdc++-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libstdc++-devel(x86_64)-4.8.5-36.el7 libstdc++-devel(x86_64)-3.4.6 passed
Result: Package existence check passed for "libstdc++-devel(x86_64)"
Check: Package existence for "sysstat"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 sysstat-10.1.5-17.el7 sysstat-5.0.5 passed
Result: Package existence check passed for "sysstat"
Check: Package existence for "pdksh"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 missing pdksh-5.2.14 failed
Result: Package existence check failed for "pdksh"
Check: Package existence for "expat(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 expat(x86_64)-2.1.0-10.el7_3 expat(x86_64)-1.95.7 passed
Result: Package existence check passed for "expat(x86_64)"
Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed
Check: Current group ID
Result: Current group ID check passed
Starting check for consistency of primary group of root user
Node Name Status
------------------------------------ ------------------------
rac2 passed
Check for consistency of root user's primary group passed
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes
No NTP Daemons or Services were found to be running
Result: Clock synchronization check using Network Time Protocol(NTP) passed
Checking Core file name pattern consistency...
Core file name pattern consistency check passed.
Checking to make sure user "grid" is not in "root" group
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed does not exist
Result: User "grid" is not part of "root" group. Check passed
Check default user file creation mask
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
rac2 0022 0022 passed
Result: Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes
Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined
File "/etc/resolv.conf" does not have both domain and search entries defined
Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes...
domain entry in file "/etc/resolv.conf" is consistent across nodes
Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes...
search entry in file "/etc/resolv.conf" is consistent across nodes
Checking DNS response time for an unreachable node
Node Name Status
------------------------------------ ------------------------
rac2 passed
The DNS response time for an unreachable node is within acceptable limit on all nodes
File "/etc/resolv.conf" is consistent across nodes
Check: Time zone consistency
Result: Time zone consistency check passed
Pre-check for cluster services setup was unsuccessful on all the nodes.
5,安装Oracle软件，及创建数据库
解压grid安装包
静默安装grid配置文件：（这里配置安装grid软件配置ASM）编辑grid_install.rsp,如下仅供参考：
oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v11_2_0
ORACLE_HOSTNAME=rac1
INVENTORY_LOCATION=/u01/app/oraInventory
SELECTED_LANGUAGES=en
oracle.install.option=CRS_CONFIG
ORACLE_BASE=/u01/app/grid
ORACLE_HOME/u01/app/11.2.0/grid
oracle.install.asm.OSDBA=asmdba
oracle.install.asm.OSOPER=asmoper
oracle.install.asm.OSASM=asmadmin
oracle.install.crs.config.gpnp.scanName=rac-san
oracle.install.crs.config.gpnp.scanPort=1521
oracle.install.crs.config.clusterName=cluster-san
oracle.install.crs.config.gpnp.configureGNS=false
oracle.install.crs.config.gpnp.gnsSubDomain=
oracle.install.crs.config.gpnp.gnsVIPAddress=
oracle.install.crs.config.autoConfigureClusterNodeVIP=false
oracle.install.crs.config.clusterNodes=rac1:rac1-vip,rac2:rac2-vip
oracle.install.crs.config.networkInterfaceList=ens192:192.168.2.101:1,ens224:10.10.10.2:2
oracle.install.crs.config.storageOption=ASM_STORAGE
oracle.install.crs.config.sharedFileSystemStorage.diskDriveMapping=
oracle.install.crs.config.sharedFileSystemStorage.votingDiskLocations=
oracle.install.crs.config.sharedFileSystemStorage.votingDiskRedundancy=NORMAL
oracle.install.crs.config.sharedFileSystemStorage.ocrLocations=
oracle.install.crs.config.sharedFileSystemStorage.ocrRedundancy=NORMAL
oracle.install.crs.config.useIPMI=false
oracle.install.crs.config.ipmi.bmcUsername=
oracle.install.crs.config.ipmi.bmcPassword=
#配置ASM
oracle.install.asm.SYSASMPassword=oracle
oracle.install.asm.diskGroup.name=DATA
oracle.install.asm.diskGroup.redundancy=EXTERNAL
oracle.install.asm.diskGroup.AUSize=1
oracle.install.asm.diskGroup.disks=/dev/raw/raw1
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/raw/*
oracle.install.asm.monitorPassword=oracle
oracle.install.crs.upgrade.clusterNodes=
oracle.install.asm.upgradeASM=false
oracle.installer.autoupdates.option=SKIP_UPDATES
oracle.installer.autoupdates.downloadUpdatesLoc=
AUTOUPDATES_MYORACLESUPPORT_USERNAME=
AUTOUPDATES_MYORACLESUPPORT_PASSWORD=
PROXY_HOST=
PROXY_PORT=
PROXY_USER=
PROXY_PWD=
PROXY_REALM=
安装oracle 软件
db_install.rsp文件配置如下：
oracle.install.responseFileVersion=/oracle/install/rspfmt_dbinstall_response_schema_v11_2_0
oracle.install.option=INSTALL_DB_AND_CONFIG
ORACLE_HOSTNAME=rac2
UNIX_GROUP_NAME=oinstall
INVENTORY_LOCATION=/u01/app/oraInventory
SELECTED_LANGUAGES=en
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_BASE=/u01/app/oracle
oracle.install.db.InstallEdition=EE
oracle.install.db.EEOptionsSelection=false
oracle.install.db.optionalComponents=oracle.rdbms.partitioning:11.2.0.4.0,oracle.oraolap:11.2.0.4.0,oracle.rdbms.dm:11.2.0.4.0,oracle.rdbms.dv:11.2.0.4.0,oracle.rdbms.lbac:11.2.0.4.0,oracle.rdbms.rat:11.2.0.4.0
oracle.install.db.DBA_GROUP=dba
oracle.install.db.OPER_GROUP=oper
oracle.install.db.CLUSTER_NODES=rac1,rac2
oracle.install.db.isRACOneInstall=
oracle.install.db.racOneServiceName=
oracle.install.db.config.starterdb.type=GENERAL_PURPOSE
oracle.install.db.config.starterdb.globalDBName=rac
oracle.install.db.config.starterdb.SID=rac
oracle.install.db.config.starterdb.characterSet=ZHS16GBK
oracle.install.db.config.starterdb.memoryOption=true
oracle.install.db.config.starterdb.memoryLimit=512
oracle.install.db.config.starterdb.installExampleSchemas=false
oracle.install.db.config.starterdb.enableSecuritySettings=true
oracle.install.db.config.starterdb.password.ALL=oracle
oracle.install.db.config.starterdb.password.SYS=
oracle.install.db.config.starterdb.password.SYSTEM=
oracle.install.db.config.starterdb.password.SYSMAN=
oracle.install.db.config.starterdb.password.DBSNMP=
oracle.install.db.config.starterdb.control=DB_CONTROL
oracle.install.db.config.starterdb.gridcontrol.gridControlServiceURL=
oracle.install.db.config.starterdb.automatedBackup.enable=false
oracle.install.db.config.starterdb.automatedBackup.osuid=
oracle.install.db.config.starterdb.automatedBackup.ospwd=
oracle.install.db.config.starterdb.storageType=ASM_STORAGE
oracle.install.db.config.starterdb.fileSystemStorage.dataLocation=
oracle.install.db.config.starterdb.fileSystemStorage.recoveryLocation=
oracle.install.db.config.asm.diskGroup=DATA
oracle.install.db.config.asm.ASMSNMPPassword=oracle
MYORACLESUPPORT_USERNAME=
MYORACLESUPPORT_PASSWORD=
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false
DECLINE_SECURITY_UPDATES=true
PROXY_HOST=
PROXY_PORT=
PROXY_USER=
PROXY_PWD=
PROXY_REALM=
COLLECTOR_SUPPORTHUB_URL=
oracle.installer.autoupdates.option=SKIP_UPDATES
oracle.installer.autoupdates.downloadUpdatesLoc=
AUTOUPDATES_MYORACLESUPPORT_USERNAME=
AUTOUPDATES_MYORACLESUPPORT_PASSWORD=
执行安装./runInstaller -silent -showProgress -responseFile /home/oracle/db_install.rsp
创建数据库
dbca.rsp配置文件如下
[GENERAL]
RESPONSEFILE_VERSION = "11.2.0"
OPERATION_TYPE = "createDatabase"
[CREATEDATABASE]
GDBNAME = "rac"
SID = "rac"
NODELIST=rac1,rac2
TEMPLATENAME = "General_Purpose.dbc"
SYSPASSWORD = "oracle"
SYSTEMPASSWORD = "oracle"
SYSMANPASSWORD = "oracle"
DBSNMPPASSWORD = "oracle"
STORAGETYPE=ASM
DISKGROUPNAME=DATA
ASMSNMP_PASSWORD="oracle"
#RECOVERYGROUPNAME=ARCH
CHARACTERSET = "ZHS16GBK"
NATIONALCHARACTERSET= "UTF8"
执行安装./runInstaller -silent -responseFile /home/oracle/dbca.rsp
参数配置：TEMPLATENAME = "General_Purpose.dbc" General_Purpose.dbc文件可在$ORACLE_HOME//assistants/dbca/templates/中找到：
[oracle@rac2 response]$ ls -lh /u01/app/oracle/product/11.2.0/db_1/assistants/dbca/templates/
total 295M
-rw-r--r-- 1 oracle oinstall 5.0K Aug 24 2013 Data_Warehouse.dbc
-rwxr-xr-x 1 oracle oinstall 21M Aug 27 2013 example01.dfb
-rwxr-xr-x 1 oracle oinstall 1.5M Aug 27 2013 example.dmp
-rw-r--r-- 1 oracle oinstall 4.9K Aug 24 2013 General_Purpose.dbc
-rw-r--r-- 1 oracle oinstall 12K May 1 2013 New_Database.dbt
-rwxr-xr-x 1 oracle oinstall 9.3M Aug 27 2013 Seed_Database.ctl
-rwxr-xr-x 1 oracle oinstall 263M Aug 27 2013 Seed_Database.dfb
[oracle@rac2 response]$
6，查看Oracle集群状态，测试Oracle连接
测试oracle 操作系统验证执行如下
[oracle@rac1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Wed Jul 15 22:37:05 2020
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL>
查看节点信息
SQL> desc gv$instance
Name Null? Type
----------------------------------------- -------- ----------------------------
INST_ID NUMBER
INSTANCE_NUMBER NUMBER
INSTANCE_NAME VARCHAR2(16)
HOST_NAME VARCHAR2(64)
VERSION VARCHAR2(17)
STARTUP_TIME DATE
STATUS VARCHAR2(12)
PARALLEL VARCHAR2(3)
THREAD# NUMBER
ARCHIVER VARCHAR2(7)
LOG_SWITCH_WAIT VARCHAR2(15)
LOGINS VARCHAR2(10)
SHUTDOWN_PENDING VARCHAR2(3)
DATABASE_STATUS VARCHAR2(17)
INSTANCE_ROLE VARCHAR2(18)
ACTIVE_STATE VARCHAR2(9)
BLOCKED VARCHAR2(3)
SQL> column host_name format a10
SQL> select host_name ,instance_name,status,database_status from gv$instance
2 /
HOST_NAME INSTANCE_NAME STATUS DATABASE_STATUS
---------- ---------------- ------------ -----------------
rac1 rac1 OPEN ACTIVE
rac2 rac2 OPEN ACTIVE
SQL>
使用ip端口服务名连接
[oracle@rac1 ~]$ sqlplus system/oracle@192.168.2.105:1521/rac
SQL*Plus: Release 11.2.0.4.0 Production on Wed Jul 15 22:38:34 2020
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL>
查看监听运行在节点rac2上，
[grid@rac2 ~]$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node rac2
测试把rac2上的两个网卡全部停用,然后再测oracle连接
执行如下命令 ifdown ens192, ifdown ens224
rac2的远程已经失去连接
登录rac1测试登录Oracle
[oracle@rac1 ~]$ sqlplus system/oracle@192.168.2.105:1521/rac
SQL*Plus: Release 11.2.0.4.0 Production on Wed Jul 15 23:21:56 2020
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> desc gv$instance
Name Null? Type
----------------------------------------- -------- ----------------------------
INST_ID NUMBER
INSTANCE_NUMBER NUMBER
INSTANCE_NAME VARCHAR2(16)
HOST_NAME VARCHAR2(64)
VERSION VARCHAR2(17)
STARTUP_TIME DATE
STATUS VARCHAR2(12)
PARALLEL VARCHAR2(3)
THREAD# NUMBER
ARCHIVER VARCHAR2(7)
LOG_SWITCH_WAIT VARCHAR2(15)
LOGINS VARCHAR2(10)
SHUTDOWN_PENDING VARCHAR2(3)
DATABASE_STATUS VARCHAR2(17)
INSTANCE_ROLE VARCHAR2(18)
ACTIVE_STATE VARCHAR2(9)
BLOCKED VARCHAR2(3)
SQL> column host_name format a10
SQL> select host_name,instance_name.status.database_status from gv$instance
2 /
select host_name,instance_name.status.database_status from gv$instance
*
ERROR at line 1:
ORA-00904: "INSTANCE_NAME"."STATUS"."DATABASE_STATUS": invalid identifier
SQL> c/./,
1* select host_name,instance_name,status.database_status from gv$instance
SQL> c/./,
1* select host_name,instance_name,status,database_status from gv$instance
SQL> /
HOST_NAME INSTANCE_NAME STATUS DATABASE_STATUS
---------- ---------------- ------------ -----------------
rac1 rac1 OPEN ACTIVE
SQL>
查看rac1上的ip地址，查看192.168.2.105 rac-san已经被rac1节点接管，
测试完毕
7，安装问题解决及总结
1：安装完毕grid，执行root.sh脚本时候报错
ohasd failed to start
Failed to start the Clusterware. Last 20 lines of the alert log follow:
解决方案：安装oracle补丁p18370031_112040_Linux-x86-64.zip，（两个节点都要执行）
1root下执行yum install perl*
2.执行脚本 回退root.sh操作/u01/app/11.2.0/grid/crs/install/roothas.pl -deconfig -force -verbose
3.解压压缩包，切换到18370031目录执行 /u01/app/11.2.0/grid/OPatch/opatch apply 完成更新，输出没有报错信息，重新运行root脚本
2：安装oracle软件时候报错：INS-35354: The system on which you are attempting to install Oracle RAC is not part of a valid cluster.
解决方案：编辑/u01/app/oraInventory/ContentsXML/inventory.xml，找到属性 <HOME NAME="Ora11g_gridinfrahome1" LOC="/u01/app/11.2.0/grid" TYPE="O" IDX="1">添加CRS="true"属性，两个节点都要修改，修改完毕重新执行安装
如下
[oracle@rac2 ~]$ cat /u01/app/oraInventory/ContentsXML/inventory.xml
<?xml version="1.0" standalone="yes" ?>
<!-- Copyright (c) 1999, 2013, Oracle and/or its affiliates.
All rights reserved. -->
<!-- Do not modify the contents of this file by hand. -->
<INVENTORY>
<VERSION_INFO>
<SAVED_WITH>11.2.0.4.0</SAVED_WITH>
<MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
</VERSION_INFO>
<HOME_LIST>
<HOME NAME="Ora11g_gridinfrahome1" LOC="/u01/app/11.2.0/grid" TYPE="O" IDX="1" CRS="true">
<NODE_LIST>
<NODE NAME="rac1"/>
<NODE NAME="rac2"/>
</NODE_LIST>
</HOME>
<HOME NAME="OraDb11g_home1" LOC="/u01/app/oracle/product/11.2.0/db_1" TYPE="O" IDX="2">
<NODE_LIST>
<NODE NAME="rac1"/>
<NODE NAME="rac2"/>
</NODE_LIST>
</HOME>
</HOME_LIST>
<COMPOSITEHOME_LIST>
</COMPOSITEHOME_LIST>
</INVENTORY>
[oracle@rac2 ~]$