Oracle Database (RDBMS) on Unix AIX,HP-UX,Linux,Solaris and MS Windows Operating Systems Installation and Configuration Requirements Quick Reference (12.1/12.2/18c/19c) (Doc ID 1587357.1) 

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/cwlin/grid-infrastructure-installation-and-upgrade-guide-linux.pdf

Patch Set Updates for Oracle Products (Doc ID 854428.1)    

Database 12.2.0.1 Proactive Patch Information (Doc ID 2285557.1) 

How to Apply a Grid Infrastructure Patch Before Grid Infrastructure Configuration (before root.sh or rootupgrade.sh or gridsetup.bat) is Executed (Doc ID 1410202.1)	 

##以下是遇到的坑，以及MOS文章解决方案：
1、ASM device sharedness check 
Shared Storage Accessibility:/dev/asm_ocr ...FAILED (PRVG-11506)
可参照MOS文档：
12.2: PRVG-0802 : Storage type for path "/dev/mapper/asm011p1" could not be determined (Doc ID 2251322.1)    

2、CLSRSC-400: A system reboot is required to continue installing.
可参照MOS文档：
ALERT: root.sh Fails With "CLSRSC-400" While Installing GI 12.2.0.1 on RHEL or OL with RedHat Compatible Kernel (RHCK) 7.3 (Doc ID 2284463.1)  

3、kgfnGetConnDetails requires 4 parameters at/u01/app/12.2.0/grid/lib/asmcmdbase.pm line 5704.
可参照MOS文档：
install.sh Hung And root.sh Is Failing At asmcmd lsdg --suppressheader While Installing A Zone With Clusterware (Doc ID 2414241.1)    
ASMCMD Failing With "KGFNGETCONNDETAILS Requires 4 Parameters at <GI_HOME>/lib/asmcmdbase.pm (Doc ID 2748316.1)

0、参数准备

1、HostName、ClusterName、ScanName

2、IP（Public，Private，Virtual，Scan）

3、ASMDISK（OCR，ARCH，DATA；Redundancy）

4、ORACLE_SID

5、User Password（root，oracle，grid）

6、CharacterSet（CharacterSet，NationalCharacterSet）

7、Software（ORACLE，GRID，REDHAT）

8、ISCDB

一、环境检查

1、Release Info

dmidecode | grep "Product Name"
more /etc/system-release

2、Kernel Info

uname -r

3、CPU Info

grep "model name" /proc/cpuinfo
grep "model name" /proc/cpuinfo | wc -l

4、Memory Info

##查看内存条数、大小
dmidecode|grep -P -A5 "Memory\s+Device"|grep Size|grep -v Range
grep MemTotal /proc/meminfo | awk '{print $2}'

5、Swap Info

grep SwapTotal /proc/meminfo | awk '{print $2}'

6、ZONE

timedatectl | grep zone

7、tmp Info

df -h /tmp

8、Disk Info

df -h
lsblk

9、Network Info

nmcli connection show
ip addr

##查看网卡驱动的速率
iplink=`cat /proc/net/dev | awk '{i++; if(i>2){print $1}}' | sed 's/^[\t]*//g' | sed 's/[:]*$//g'`
for i in $iplink
do
printf "%s %s\n" "$i :"
ethtool -i $i

10、Transparent HugePages

cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

11、vm.min_free_kbytes

cat /proc/sys/vm/min_free_kbytes

12、vm.swappiness

cat /proc/sys/vm/swappiness

13、SELINUX

grep ^SELINUX= /etc/selinux/config

14、Firewalld

systemctl status firewalld | grep Active

15、avahi-daemon

systemctl status avahi-daemon | grep Active

16、NUMA

grep NUMA /proc/cmdline

二、环境配置

1、RPM PACKAGE

关于oracle-database-server-12cR2-preinstall

yum install oracle-database-server-12cR2-preinstall-1.0-5.el7.src.rpm

oracle-database-server-12cR2-preinstall为我们做了什么

（1）自动安装oracle所需的RPM包

（2）自动创建oracle用户和dba、oinstall组

（3）自动配置/etc/sysctl.conf内核参数

（4）自动配置/etc/security/limits.conf参数

（5）关闭NUMA=OFF   （关闭非一致内存访问）

（6）关闭NOZEROCONF=yes

（7）修改系统启动版本信息

摘自官方MOS文档：

Oracle Database (RDBMS) on Unix AIX,HP-UX,Linux,Solaris and MS Windows Operating Systems Installation and Configuration Requirements Quick Reference (12.1/12.2/18c/19c) (Doc ID 1587357.1) Oracle Linux 7 / RHEL 7

Oracle Linux 7/RHEL 7
OS Version	Patches/Packages	Kernel settings
Oracle Linux 7 with the Unbreakable Enterprise Kernel 3: 3.8.13-35.3.1.el7uek.x86_64 or later Oracle Linux 7.2 with the Unbreakable Enterprise Kernel 4: 4.1.12-32.2.3.el7uek.x86_64 or later Oracle Linux 7 with the Red Hat Compatible kernel: 3.10.0-123.el7.x86_64 or later Red Hat Enterprise Linux 7: 3.10.0-123.el7.x86_64 or later Reference : Note 1961997.1	binutils-2.23.52.0.1-12.el7 (x86_64) compat-libcap1-1.10-3.el7 (x86_64) compat-libstdc++-33-3.2.3-71.el7 (i686) compat-libstdc++-33-3.2.3-71.el7 (x86_64) glibc-2.17-36.el7 (i686) glibc-2.17-36.el7 (x86_64) glibc-devel-2.17-36.el7 (i686) glibc-devel-2.17-36.el7 (x86_64) ksh libaio-0.3.109-9.el7 (i686) libaio-0.3.109-9.el7 (x86_64) libaio-devel-0.3.109-9.el7 (i686) libaio-devel-0.3.109-9.el7 (x86_64) libX11-1.6.0-2.1.el7 (i686) libX11-1.6.0-2.1.el7 (x86_64) libXau-1.0.8-2.1.el7 (i686) libXau-1.0.8-2.1.el7 (x86_64) libXi-1.7.2-1.el7 (i686) libXi-1.7.2-1.el7 (x86_64) libXtst-1.2.2-1.el7 (i686) libXtst-1.2.2-1.el7 (x86_64) libgcc-4.8.2-3.el7 (i686) libgcc-4.8.2-3.el7 (x86_64) libstdc++-4.8.2-3.el7 (i686) libstdc++-4.8.2-3.el7 (x86_64) libstdc++-devel-4.8.2-3.el7 (i686) libstdc++-devel-4.8.2-3.el7 (x86_64) libxcb-1.9-5.el7 (i686) libxcb-1.9-5.el7 (x86_64) make-3.82-19.el7 (x86_64) nfs-utils-1.3.0-0.21.el7.x86_64 (for Oracle ACFS) net-tools-2.0-0.17.20131004git.el7 (x86_64) (for Oracle RAC and Oracle Clusterware) smartmontools-6.2-4.el7 (x86_64) sysstat-10.1.5-1.el7 (x86_64) Ensure that OpenSSH is installed on your servers. OpenSSH is the required SSH software.	semmsl 250 semmns 32000 semopm 100 semmni 128 shmall 40 percent of the size of physical memory in pages shmmax Half the size of physical memory in bytes shmmni 4096 panic_on_oops 1 file-max 6815744 aio-max-nr 1048576 ip_local_port_range Minimum: 9000 Maximum: 65500 rmem_default 262144 rmem_max 4194304 wmem_default 262144 wmem_max 1048576 ip_local_port_range 32768 61000 /etc/security/limits.conf oracle soft nproc 2047 (at least) oracle hard nproc 16384 (at least) oracle soft nofile 1024 (at least) oracle hard nofile 65536 (at least) oracle soft stack 10240 KB (at least) oracle hard stack 32768 KB (at most) memlock Soft: at least 90 percent of the current RAM when HugePages memory is enabled and at least 3145728 KB (3 GB) when HugePages memory is disabled hard: at least 90 percent of the current RAM when HugePages memory is enabled and at least 3145728 KB (3 GB) when HugePages memory is disabled

以下摘自官方文档：

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/cwlin/grid-infrastructure-installation-and-upgrade-guide-linux.pdf

Operating System Requirements for x86-64 Linux Platforms

Red Hat Enterprise Linux 7 Distributions for x86-64：
1、SSH Requirement Ensure that OpenSSH is installed on your servers.

2、Red Hat Enterprise Linux 7: 3.10.0-123.el7.x86_64 or later

3、he following packages (or later versions) must be installed:
bc
binutils-2.23.52.0.1-12.el7 (x86_64)
compat-libcap1-1.10-3.el7 (x86_64)
compat-libstdc++-33-3.2.3-71.el7 (i686)
compat-libstdc++-33-3.2.3-71.el7 (x86_64)
glibc-2.17-36.el7 (i686)
glibc-2.17-36.el7 (x86_64)
glibc-devel-2.17-36.el7 (i686)
glibc-devel-2.17-36.el7 (x86_64)
ksh
libaio-0.3.109-9.el7 (i686)
libaio-0.3.109-9.el7 (x86_64)
libaio-devel-0.3.109-9.el7 (i686)
libaio-devel-0.3.109-9.el7 (x86_64)
libgcc-4.8.2-3.el7 (i686)
libgcc-4.8.2-3.el7 (x86_64)
libstdc++-4.8.2-3.el7 (i686)
libstdc++-4.8.2-3.el7 (x86_64)
libstdc++-devel-4.8.2-3.el7 (i686)
libstdc++-devel-4.8.2-3.el7 (x86_64)
libxcb-1.9-5.el7 (i686)
libxcb-1.9-5.el7 (x86_64)
libX11-1.6.0-2.1.el7 (i686)
libX11-1.6.0-2.1.el7 (x86_64)
libXau-1.0.8-2.1.el7 (i686)
libXau-1.0.8-2.1.el7 (x86_64)
libXi-1.7.2-1.el7 (i686)
libXi-1.7.2-1.el7 (x86_64)
libXtst-1.2.2-1.el7 (i686)
libXtst-1.2.2-1.el7 (x86_64)
libXrender (i686)
libXrender (x86_64)
libXrender-devel (i686)
libXrender-devel (x86_64)
make-3.82-19.el7 (x86_64)
net-tools-2.0-0.17.20131004git.el7 (x86_64) (for Oracle RAC
and Oracle Clusterware)
nfs-utils-1.3.0-0.21.el7.x86_64 (for Oracle ACFS)
smartmontools-6.2-4.el7 (x86_64)
sysstat-10.1.5-1.el7 (x86_64)

--redhat内核支持
Red Hat Enterprise Linux 7: 3.10.0-123.el7.x86_64 or later

--镜像挂载
mount /dev/cdrom /mnt
mv /etc/yum.repos.d/* /tmp/

echo "[local]" >> /etc/yum.repos.d/local.repo
echo "name = local" >> /etc/yum.repos.d/local.repo
echo "baseurl = file:///mnt/" >> /etc/yum.repos.d/local.repo
echo "enabled = 1" >> /etc/yum.repos.d/local.repo
echo "gpgcheck = 0" >> /etc/yum.repos.d/local.repo

yum clean all
yum repolist all
--安装图形化界面
yum groupinstall -y "Server with GUI"

--安装oracle所需依赖包
yum install -y bc binutils compat-libcap1 compat-libstdc++ glibc glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel libxcb libX11 libXau libXi libXtst libXrender libXrender-devel make net-tools nfs-utils smartmontools sysstat openssh-clients

--检查是否安装成功
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' bc binutils compat-libcap1 compat-libstdc++ glibc glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel libxcb libX11 libXau libXi libXtst libXrender libXrender-devel make net-tools nfs-utils smartmontools sysstat openssh-clients

--需要额外安装compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm包
rpm -ivh compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm

2、Configuring Kernel Parameters for Linux

The /etc/sysctl.conf file has been deprecated

/etc/sysctl.conf文件在OL7和RHEL7中已被弃用

cat <<EOF >> /etc/sysctl.d/97-oracledatabase-sysctl.conf
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152 ##equal SGA，mininum is 2097152,advise 40 percent of the size ofphysical memory in pages
kernel.shmmax = 4294967295 ##advise Half the size of physical memory in bytes
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.conf.eno5.rp_filter = 2
net.ipv4.conf.eno6.rp_filter = 2
EOF

3、Resource Limits

/etc/security/limits.conf

oracle   soft    nproc    2047 (at least)
oracle   hard   nproc   16384 (at least)
oracle   soft    nofile    1024 (at least)
oracle   hard   nofile    65536 (at least)
oracle   soft    stack    10240 KB (at least)
oracle   hard   stack    32768 KB (at most)

memlock    
Soft: at least 90 percent of the current RAM when HugePages memory is enabled and at least 3145728 KB (3 GB) when HugePages memory is disabled     
hard: at least 90 percent of the current RAM when HugePages memory is enabled and at least 3145728 KB (3 GB) when HugePages memory is disabled

cat <<EOF >>/etc/security/limits.conf
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768
grid soft nproc 2047
grid hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768
oracle soft nproc 2047
oracle hard nproc 16384
###如果内存小于8G，下面大内存页配置可忽略
grid hard memlock 3145728
grid soft memlock 3145728
oracle hard memlock 3145728
oracle soft memlock 3145728
EOF

cat <<EOF >>/etc/pam.d/login
session required pam_limits.so 
session required /lib64/security/pam_limits.so
EOF

4、USER CREATE

Creating the OSASM Group
# groupadd -g 54329 asmadmin
Creating the OSDBA for ASM Group
# /usr/sbin/groupadd -g 54327 asmdba
Creating the OSOPER for ASM Group
# /usr/sbin/groupadd -g 54328 asmoper
Creating the OSDBA Group for Database Installations
# /usr/sbin/groupadd -g 54322 dba
Creating an OSOPER Group for Database Installations
# groupadd -g 54323 oper
Creating the OSBACKUPDBA Group for Database Installations
# /usr/sbin/groupadd -g 54324 backupdba
Creating the OSDGDBA Group for Database Installations
# /usr/sbin/groupadd -g 54325 dgdba
Creating the OSKMDBA Group for Database Installations
# /usr/sbin/groupadd -g 54326 kmdba
Creating the OSRACDBA Group for Database Installations
# /usr/sbin/groupadd -g 54330 racdba

$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)
$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)

##创建用户组
/usr/sbin/groupadd -g 54321 oinstall
/usr/sbin/groupadd -g 54322 dba
/usr/sbin/groupadd -g 54323 oper
/usr/sbin/groupadd -g 54324 backupdba
/usr/sbin/groupadd -g 54325 dgdba
/usr/sbin/groupadd -g 54326 kmdba
/usr/sbin/groupadd -g 54327 asmdba
/usr/sbin/groupadd -g 54328 asmoper
/usr/sbin/groupadd -g 54329 asmadmin
/usr/sbin/groupadd -g 54330 racdba

##创建用户
/usr/sbin/useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba oracle
/usr/sbin/useradd -u 54331 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
 
#修改密码，这边密码均设置为oracle,密码可以按自己需要修改
echo "oracle" |passwd grid --stdin
echo "oracle" |passwd oracle --stdin

5、Oracle Home and Oracle Base Directory CREATE

mkdir -p /u01/app/grid
mkdir -p /u01/app/12.2.0/grid
mkdir -p /u01/app/oracle/product/12.2.0/db
mkdir -p /u01/app/oraInventory
chown -R grid:oinstall /u01/app/grid
chown -R grid:oinstall /u01/app/12.2.0
chown -R grid:oinstall /u01/app/oraInventory
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01

6、Disable Firewalld and SELLINUX

--关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

--关闭selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

7、HOSTS

cat >> /etc/hosts <<EOF
#Public IP
192.168.31.100	p12c01
192.168.31.101	p12c02
 
#Private IP
1.1.1.1	p12c01-priv1
1.1.1.2	p12c02-priv1

2.2.2.1	p12c01-priv2
2.2.2.2	p12c02-priv2
 
#Virtual IP
192.168.31.200	p12c01-vip
192.168.31.201	p12c02-vip
 
#SCAN IP
192.168.31.50	p12c-scan
EOF

8、Disable Transparent Hugepages & NUMA

For example, for Oracle Linux 7.3, the procedure to disable Transparent HugePages involves editing the /etc/default/grub file and then running the command grub2-mkconfig

sed -i 's/quiet/quiet transparent_hugepage=never numa=off/' /etc/default/grub

##重建grub 配置文件：
##fdisk -l可以查看主分区的Disk label type: gpt
##MBR 分区表
##grub2-mkconfig -o /etc/grub2.cfg\

##efi 引导模式,efi+GPT分区表
grub2-mkconfig -o /etc/grub2-efi.cfg

##reboot之后查看是否生效
numactl --hardware
dmesg|grep -i numa
cat /proc/cmdline

##临时生效
echo never >/sys/kernel/mm/transparent_hugepage/enabled
echo never >/sys/kernel/mm/transparent_hugepage/defrag

9、Disable NOZEROCONF

echo "NOZEROCONF=yes" >> /etc/sysconfig/network

10、Disable NetworkManager

systemctl stop NetworkManager.service
systemctl disable NetworkManager.service

11、Disable avahi-daemon

systemctl stop avahi-daemon.socket
systemctl stop avahi-daemon.service
systemctl disable avahi-daemon.service
ps -ef|grep avahi-daemon

12、Disable chronyd

By default, the NTP service available on Oracle Linux 7 and Red Hat Linux 7 is chronyd and the NTP service available on Oracle Linux 6 and Red Hat Linux 6 or lower is ntpd.

systemctl stop chronyd
systemctl disable chronyd
mv /etc/resolv.conf /etc/resolv.conf.bak
mv /etc/chrony.conf /etc/chrony.conf.bak

13、配置/dev/shm

cat <<EOF >> /etc/fstab
tmpfs /dev/shm tmpfs size=4G 0 0
EOF
mount -o remount /dev/shm

14、UDEV+MULTIPATH

##安装multipath
yum install -y device-mapper*
mpathconf --enable --with_multipathd y

##查看磁盘wwid
/usr/lib/udev/scsi_id -g -u /dev/sdb

--配置multipath.confc
cat <<EOF >> /etc/multipath.conf
defaults {
    user_friendly_names yes
}
 
blacklist {
  devnode "^sda"
}
 
multipaths {
  multipath {
  wwid "36000c299c61a189afae17172197fd20e"
  alias asm_OCR
  }
  multipath {
  wwid "36000c29511ad8bc6c69c3d63a8f4e6c8"
  alias asm_DATA
  }
}
EOF

--获取ENV{DM_UUID}
cd /dev/mapper
>udev_info
for i in `ls asm*`;
do
printf "%s %s\n" "$i" "$(udevadm info --query=all --name=/dev/mapper/$i |grep -i dm_uuid)">>udev_info;
done

--配置udev规则文件、99-oracle-asmdevices.rules文件(复制到节点二)
cat /dev/mapper/udev_info |while read line 
do
dm_uuid=`echo $line |awk -F'=' '{print $2}'`;
disk_name=`echo $line|awk '{print $1}'`;
echo "KERNEL==\"dm-*\",ENV{DM_UUID}==\"${dm_uuid}\",SYMLINK+=\"asmdisks/${disk_name}\",OWNER=\"grid\",GROUP=\"asmadmin\",MODE=\"0660\"">>/etc/udev/rules.d/99-oracle-asmdevices.rules
done

--启动udev服务
udevadm control --reload-rules
udevadm trigger

--查看udev绑盘是否成功
ls -l /dev/asmdisks/*

15、环境变量设置

ROOT:

 cat >>/etc/profile <<"EOF"
################# Root Rrofile Add ############################
if [ $USER = "oracle" ] || [ $USER = "grid" ] || [ $USER = "root" ]; then
        if  [ $SHELL = "/bin/ksh" ];  then
              ulimit -p 16384
              ulimit -n 65536
        else
              ulimit -u 16384 -n 65536
        fi
fi
alias so='su - oracle'
alias sg='su - grid'
alias crsctl='/u01/app/11.2.0/grid/bin/crsctl'
################# Root Rrofile End ############################
EOF

GRID:

cat <<EOF >> /home/grid/.bash_profile
################ Grid Profile Add #########################
umask 022
export TMP=/tmp
export TMPDIR=\$TMP
export NLS_LANG=AMERICAN_AMERICA.$CharacterSet
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/12.2.0/grid
export TNS_ADMIN=\$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=\$ORACLE_HOME/lib:/lib:/usr/lib
export ORACLE_SID=$GRID_SIDTemp
export PATH=/usr/sbin:\$PATH
export PATH=\$ORACLE_HOME/bin:\$ORACLE_HOME/OPatch:\$PATH
#alias sqlplus='rlwrap sqlplus'
#alias rman='rlwrap rman'
#alias lsnrctl='rlwrap lsnrctl'
#alias asmcmd='rlwrap asmcmd'
#alias adrci='rlwrap adrci'
alias sas='sqlplus / as sysasm'
################ Grid Profile End #########################
EOF

ORALCE:

  cat <<EOF >>/home/oracle/.bash_profile
################ Oracle Profile Add #########################
umask 022
export TMP=/tmp
export TMPDIR=\$TMP
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=\$ORACLE_BASE/product/12.2.0/db
export TNS_ADMIN=\$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=\$ORACLE_HOME/lib:/lib:/usr/lib
export ORACLE_SID=orcl
export PATH=/usr/sbin:\$PATH
export PATH=\$ORACLE_HOME/bin:\$ORACLE_HOME/OPatch:\$PATH
#alias sqlplus='rlwrap sqlplus'
#alias rman='rlwrap rman'
#alias lsnrctl='rlwrap lsnrctl'
#alias asmcmd='rlwrap asmcmd'
#alias adrci='rlwrap adrci'
alias sas='sqlplus / as sysdba'
#alias ggsci='rlwrap ggsci'
alias alert='tail -500f \$ORACLE_BASE/diag/rdbms/$ORACLE_SIDTemp/$ORACLE_SIDTemp/trace/alert_$ORACLE_SIDTemp.log|more'
################ Oracle Profile End #########################
EOF

Notes：安装前执行reboot。

三、安装GRID软件

1、解压缩Grid软件

chown -R grid:oinstall /soft
su - grid -c "unzip /soft/linuxx64_12201_grid_home.zip -d /u01/app/12.2.0/grid"

2、RPM CVU 安装

rpm -ivh /soft/compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm
rpm -ivh /soft/cvuqdisk-1.0.10-1.rpm

3、配置ssh互信

#SSH配置 (自root用户下执行)
/u01/app/12.2.0/grid/oui/prov/resources/scripts/sshUserSetup.sh -user oracle -hosts "fdcdb01 fdcdb02 fdcdb01-priv1 fdcdb01-priv2 fdcdb02-priv1 fdcdb02-priv2" -advanced -noPromptPassphrase -exverify -confirm
/u01/app/12.2.0/grid/oui/prov/resources/scripts/sshUserSetup.sh -user grid -hosts "fdcdb01 fdcdb02 fdcdb01-priv1 fdcdb01-priv2 fdcdb02-priv1 fdcdb02-priv2" -advanced -noPromptPassphrase -exverify -confirm

4、安装前runcluvfy检查

su - grid -c "/u01/app/12.2.0/grid/runcluvfy.sh stage -pre crsinst -n fdcdb01,fdcdb02 -fixup -verbose"


Pre-check for cluster services setup was unsuccessful on all the nodes. 


Failures were encountered during execution of CVU verification request "stage -pre crsinst".

Verifying Network Time Protocol (NTP) ...FAILED

CVU operation performed:      stage -pre crsinst
Date:                         Apr 2, 2021 5:27:37 AM
CVU home:                     /u01/app/12.2.0/grid/
User:                         grid

5、安装GRID软件

Notes：提前剧透下，一定要看：如果安装命令是执行./gridSetup.sh，执行root.sh时，必然报错：CLSRSC-400: A system reboot is required to continue installing.需要卸载重新安装。

解决方案：通过12C新特性 -applyPSU 提前打上最新PSU补丁修复该BUG，前提是需要更换OPatch包：

How to Apply a Grid Infrastructure Patch Before Grid Infrastructure Configuration (before root.sh or rootupgrade.sh or gridsetup.bat) is Executed (Doc ID 1410202.1)

./gridSetup.sh -applyPSU /soft/32226491

以下是完整的安装以及填坑步骤：

12.2: PRVG-0802 : Storage type for path "/dev/mapper/asm011p1" could not be determined (Doc ID 2251322.1)

[root@fdcdb02 dev]# ll /etc/multipath.conf 
-rw------- 1 root root 3209 Apr  1 14:22 /etc/multipath.conf
[root@fdcdb02 dev]# chmod 644 /etc/multipath.conf 
[root@fdcdb02 dev]# ll /etc/multipath.conf 
-rw-r--r-- 1 root root 3209 Apr  1 14:22 /etc/multipath.conf

6、执行ROOT.SH

先节点一执行：

[root@fdcdb01 grid]# /u01/app/12.2.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/fdcdb01/crsconfig/rootcrs_fdcdb01_2021-04-02_06-09-31AM.log
2021/04/02 06:09:34 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2021/04/02 06:09:34 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2021/04/02 06:09:59 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2021/04/02 06:09:59 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2021/04/02 06:10:05 CLSRSC-363: User ignored prerequisites during installation
2021/04/02 06:10:05 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2021/04/02 06:10:08 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2021/04/02 06:10:09 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2021/04/02 06:10:16 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2021/04/02 06:10:31 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2021/04/02 06:10:31 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2021/04/02 06:10:47 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2021/04/02 06:10:53 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2021/04/02 06:10:53 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2021/04/02 06:10:57 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2021/04/02 06:11:13 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2021/04/02 06:11:57 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2021/04/02 06:12:01 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb01'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2021/04/02 06:12:50 CLSRSC-400: A system reboot is required to continue installing.
The command '/u01/app/12.2.0/grid/perl/bin/perl -I/u01/app/12.2.0/grid/perl/lib -I/u01/app/12.2.0/grid/crs/install /u01/app/12.2.0/grid/crs/install/rootcrs.pl ' execution failed

ALERT: root.sh Fails With "CLSRSC-400" While Installing GI 12.2.0.1 on RHEL or OL with RedHat Compatible Kernel (RHCK) 7.3 (Doc ID 2284463.1)

根据mos文档的方式，打补丁之后，依然报错。

使用deinstall卸载GRID软件，重新安装。

Notes：这里是12201GRID的一个bug，尝试使用applyPSU的方式在安装GRID软件之前打好PSU补丁：

##更换OPatch包
su - grid -c "/u01/app/12.2.0/grid/OPatch/opatch version"
OPatch Version: 12.2.0.1.6

##You must use the OPatch utility version 12.2.0.1.23 or later to apply this patch.

mv /u01/app/12.2.0/grid/OPatch /u01/app/12.2.0/grid/OPatchbak
su - grid -c "unzip /soft/p6880880_122010_Linux-x86-64.zip -d /u01/app/12.2.0/grid/"

su - grid -c "/u01/app/12.2.0/grid/OPatch/opatch version"
OPatch Version: 12.2.0.1.24

su - grid
./gridSetup.sh -applyPSU /soft/32226491

以上安装步骤同上，省略、、、

打完补丁之后checks不再报以上错误，继续执行root.sh。

先节点一执行root.sh：

[root@fdcdb01 ~]# /u01/app/12.2.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/fdcdb01/crsconfig/rootcrs_fdcdb01_2021-04-01_04-49-48PM.log
2021/04/01 16:49:50 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2021/04/01 16:49:50 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2021/04/01 16:50:40 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2021/04/01 16:50:40 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2021/04/01 16:50:48 CLSRSC-363: User ignored prerequisites during installation
2021/04/01 16:50:48 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2021/04/01 16:50:50 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2021/04/01 16:50:56 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2021/04/01 16:51:11 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2021/04/01 16:51:24 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2021/04/01 16:51:24 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2021/04/01 16:51:50 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2021/04/01 16:52:10 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2021/04/01 16:52:11 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2021/04/01 16:52:25 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2021/04/01 16:52:40 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2021/04/01 16:53:19 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2021/04/01 16:53:33 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb01'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2021/04/01 16:54:20 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2021/04/01 16:54:34 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb01'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.mdnsd' on 'fdcdb01'
CRS-2676: Start of 'ora.mdnsd' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.evmd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'fdcdb01'
CRS-2676: Start of 'ora.gpnpd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.gipcd' on 'fdcdb01'
CRS-2676: Start of 'ora.cssdmonitor' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.gipcd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.diskmon' on 'fdcdb01'
CRS-2676: Start of 'ora.diskmon' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.cssd' on 'fdcdb01' succeeded

Disk groups created successfully. Check /u01/app/grid/cfgtoollogs/asmca/asmca-210401PM045515.log for details.


2021/04/01 16:55:49 CLSRSC-482: Running command: '/u01/app/12.2.0/grid/bin/ocrconfig -upgrade grid oinstall'
CRS-2672: Attempting to start 'ora.crf' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.storage' on 'fdcdb01'
CRS-2676: Start of 'ora.storage' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.crf' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'fdcdb01'
CRS-2676: Start of 'ora.crsd' on 'fdcdb01' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk e19e9dd1dab24fc1bfa53d8fea38c5cd.
Successfully replaced voting disk group with +OCR.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   e19e9dd1dab24fc1bfa53d8fea38c5cd (/dev/asmdisks/asm_ocr) [OCR]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.crsd' on 'fdcdb01'
CRS-2677: Stop of 'ora.crsd' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.crf' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'fdcdb01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.crf' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.storage' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'fdcdb01'
CRS-2677: Stop of 'ora.mdnsd' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.asm' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'fdcdb01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.evmd' on 'fdcdb01'
CRS-2677: Stop of 'ora.evmd' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'fdcdb01'
CRS-2677: Stop of 'ora.cssd' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'fdcdb01'
CRS-2677: Stop of 'ora.gipcd' on 'fdcdb01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2021/04/01 16:57:24 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.evmd' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.mdnsd' on 'fdcdb01'
CRS-2676: Start of 'ora.mdnsd' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.evmd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'fdcdb01'
CRS-2676: Start of 'ora.gpnpd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'fdcdb01'
CRS-2676: Start of 'ora.gipcd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fdcdb01'
CRS-2676: Start of 'ora.cssdmonitor' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.diskmon' on 'fdcdb01'
CRS-2676: Start of 'ora.diskmon' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.cssd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.ctssd' on 'fdcdb01'
CRS-2676: Start of 'ora.ctssd' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fdcdb01'
CRS-2676: Start of 'ora.asm' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'fdcdb01'
CRS-2676: Start of 'ora.storage' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'fdcdb01'
CRS-2676: Start of 'ora.crf' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'fdcdb01'
CRS-2676: Start of 'ora.crsd' on 'fdcdb01' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: fdcdb01
CRS-6016: Resource auto-start has completed for server fdcdb01
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2021/04/01 16:58:53 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/04/01 16:58:53 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.ASMNET2LSNR_ASM.lsnr' on 'fdcdb01'
CRS-2676: Start of 'ora.ASMNET2LSNR_ASM.lsnr' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fdcdb01'
CRS-2676: Start of 'ora.asm' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.OCR.dg' on 'fdcdb01'
CRS-2676: Start of 'ora.OCR.dg' on 'fdcdb01' succeeded
2021/04/01 16:59:59 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/04/01 17:05:03 CLSRSC-614: failed to get the list of configured diskgroups
Died at /u01/app/12.2.0/grid/crs/install/oraasm.pm line 2069.
The command '/u01/app/12.2.0/grid/perl/bin/perl -I/u01/app/12.2.0/grid/perl/lib -I/u01/app/12.2.0/grid/crs/install /u01/app/12.2.0/grid/crs/install/rootcrs.pl ' execution failed

节点一执行root.sh报错，并且执行asmcmd报错如下：

[grid@fdcdb01 ~]$ asmcmd -p
kgfnGetConnDetails requires 4 parameters at /u01/app/12.2.0/grid/lib/asmcmdbase.pm line 5704.

可参考MOS文档：

Install.sh Hung And root.sh Is Failing At asmcmd lsdg --suppressheader While Installing A Zone With Clusterware (Doc ID 2414241.1)

ASMCMD Failing With "KGFNGETCONNDETAILS Requires 4 Parameters at <GI_HOME>/lib/asmcmdbase.pm (Doc ID 2748316.1)

解决方案：（需要在执行root.sh之前执行该命令）

MOS文档Workround:
As GI User:

$ export ORACLE_HOME=<Oracle Home>
$ /usr/bin/make -f ins_rdbms.mk client_sharedlib libasmclntsh12.ohso
libasmperl12.ohso ORACLE_HOME=$ORACLE_HOME

##执行过程:
[grid@fdcdb01 ~]$ /usr/bin/make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk client_sharedlib libasmclntsh12.ohso libasmperl12.ohso ORACLE_HOME=/u01/app/12.2.0/grid
/u01/app/12.2.0/grid/bin/genclntsh
/bin/rm: cannot remove '/u01/app/12.2.0/grid/lib/libclntshcore.so.12.1': Permission denied
genclntsh: Failed to remove /u01/app/12.2.0/grid/lib/libclntshcore.so.12.1
make: *** [client_sharedlib] Error 1

##用grid用户执行会报错，需要以root用户执行：

/usr/bin/make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk client_sharedlib libasmclntsh12.ohso libasmperl12.ohso ORACLE_HOME=/u01/app/12.2.0/grid

##执行成功后可以进入asmcmd
[grid@fdcdb01 ~]$ asmcmd -p
ASMCMD [+] > 
ASMCMD [+] > 

##命令执行结果如下：
[root@fdcdb01 install]# /usr/bin/make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk client_sharedlib libasmclntsh12.ohso libasmperl12.ohso ORACLE_HOME=/u01/app/12.2.0/grid
/u01/app/12.2.0/grid/bin/genclntsh
`if [ -d /u01/app/12.2.0/grid/lib32 ]; then rm -f /u01/app/12.2.0/grid/lib32/libclntsh.so.10.1; rm -f /u01/app/12.2.0/grid/lib/libclntsh.so.10.1; rm -f /u01/app/12.2.0/grid/lib32/libclntsh.so.11.1; rm -f /u01/app/12.2.0/grid/lib/libclntsh.so.11.1; else rm -f /u01/app/12.2.0/grid/lib/libclntsh.so.10.1; rm -f /u01/app/12.2.0/grid/lib/libclntsh.so.11.1; fi `
`if [ -d /u01/app/12.2.0/grid/lib32 ]; then ln -s libclntsh.so /u01/app/12.2.0/grid/lib32/libclntsh.so.10.1; ln -s libclntsh.so /u01/app/12.2.0/grid/lib/libclntsh.so.10.1; ln -s libclntsh.so /u01/app/12.2.0/grid/lib32/libclntsh.so.11.1; ln -s libclntsh.so /u01/app/12.2.0/grid/lib/libclntsh.so.11.1; else ln -s libclntsh.so /u01/app/12.2.0/grid/lib/libclntsh.so.10.1; ln -s libclntsh.so /u01/app/12.2.0/grid/lib/libclntsh.so.11.1; fi `
/u01/app/12.2.0/grid/bin/genoccish
/u01/app/12.2.0/grid/bin/genagtsh /u01/app/12.2.0/grid/lib/libagtsh.so 1.0
make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk ALWAYS=always /u01/app/12.2.0/grid/lib/libasmclntsh12.so
make[1]: Entering directory `/u01/app/12.2.0/grid/install'
rm -f /u01/app/12.2.0/grid/lib/libasmclntsh12.so
/u01/app/12.2.0/grid/bin/linkshlib /u01/app/12.2.0/grid/lib/libasmclntsh12.so /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk so ld_shlib LIBS 
+ PATH=/bin:/usr/bin:/usr/ccs/bin
+ export PATH
+ lib=/u01/app/12.2.0/grid/lib/libasmclntsh12.so
+ makefile=/u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk
+ so_ext=so
+ target=ld_shlib
++ basename /u01/app/12.2.0/grid/lib/libasmclntsh12.so .so
+ libname=libasmclntsh12
++ dirname /u01/app/12.2.0/grid/lib/libasmclntsh12.so
+ sodir=/u01/app/12.2.0/grid/lib
+ ardir=/u01/app/12.2.0/grid/lib/
+ '[' var = ld_shlib ']'
+ suffix=LIBS
+ var=
+ '[' '!' -f /u01/app/12.2.0/grid/lib/libasmclntsh12.a ']'
+ '[' '' '!=' '' ']'
+ make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk ld_shlib _FULL_LIBNAME=/u01/app/12.2.0/grid/lib/libasmclntsh12.so _LIBNAME=libasmclntsh12 _LIBDIR=/u01/app/12.2.0/grid/lib/ '_LIBNAME_LIBS=$(libasmclntsh12LIBS)' '_LIBNAME_EXTRALIBS=$(libasmclntsh12EXTRALIBS)'
make[2]: Entering directory `/u01/app/12.2.0/grid/install'
/u01/app/12.2.0/grid/bin/orald -o /u01/app/12.2.0/grid/lib/libasmclntsh12.so -shared -z noexecstack -Wl,--disable-new-dtags -L/tmp/bootstraplib/ -L/u01/app/12.2.0/grid/lib/ -L/u01/app/12.2.0/grid/rdbms/lib/ -L/u01/app/12.2.0/grid/lib/stubs/ -Wl,--version-script=/u01/app/12.2.0/grid/rdbms/admin/libasmclntsh12.def -Wl,--whole-archive /u01/app/12.2.0/grid/lib/libasmclntsh12.a -Wl,--no-whole-archive    -lirc
make[2]: Leaving directory `/u01/app/12.2.0/grid/install'
make[1]: Leaving directory `/u01/app/12.2.0/grid/install'
make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk ALWAYS=always /u01/app/12.2.0/grid/lib/libasmperl12.so
make[1]: Entering directory `/u01/app/12.2.0/grid/install'
rm -f /u01/app/12.2.0/grid/lib/libasmperl12.so
/u01/app/12.2.0/grid/bin/linkshlib /u01/app/12.2.0/grid/lib/libasmperl12.so /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk so dlopenlib LIBS 
+ PATH=/bin:/usr/bin:/usr/ccs/bin
+ export PATH
+ lib=/u01/app/12.2.0/grid/lib/libasmperl12.so
+ makefile=/u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk
+ so_ext=so
+ target=dlopenlib
++ basename /u01/app/12.2.0/grid/lib/libasmperl12.so .so
+ libname=libasmperl12
++ dirname /u01/app/12.2.0/grid/lib/libasmperl12.so
+ sodir=/u01/app/12.2.0/grid/lib
+ ardir=/u01/app/12.2.0/grid/lib/
+ '[' var = dlopenlib ']'
+ suffix=LIBS
+ var=
+ '[' '!' -f /u01/app/12.2.0/grid/lib/libasmperl12.a ']'
+ '[' '' '!=' '' ']'
+ make -f /u01/app/12.2.0/grid/rdbms/lib/ins_rdbms.mk dlopenlib _FULL_LIBNAME=/u01/app/12.2.0/grid/lib/libasmperl12.so _LIBNAME=libasmperl12 _LIBDIR=/u01/app/12.2.0/grid/lib/ '_LIBNAME_LIBS=$(libasmperl12LIBS)' '_LIBNAME_EXTRALIBS=$(libasmperl12EXTRALIBS)'
make[2]: Entering directory `/u01/app/12.2.0/grid/install'
/u01/app/12.2.0/grid/bin/orald -o /u01/app/12.2.0/grid/lib/libasmperl12.so -shared -z noexecstack -Wl,--disable-new-dtags -L/tmp/bootstraplib/ -L/u01/app/12.2.0/grid/lib/ -L/u01/app/12.2.0/grid/rdbms/lib/ -L/u01/app/12.2.0/grid/lib/stubs/ -Wl,--version-script=/u01/app/12.2.0/grid/rdbms/admin/libasmperl12.def -Wl,--whole-archive /u01/app/12.2.0/grid/lib/libasmperl12.a -Wl,--no-whole-archive -lclntsh -lclntshcore -lasmclntsh12 -lhasgen12 -locr12   -lirc 
make[2]: Leaving directory `/u01/app/12.2.0/grid/install'
make[1]: Leaving directory `/u01/app/12.2.0/grid/install'

重新执行root.sh：

[root@fdcdb01 ~]# /u01/app/12.2.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/fdcdb01/crsconfig/rootcrs_fdcdb01_2021-04-01_05-39-52PM.log
2021/04/01 17:39:55 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2021/04/01 17:39:55 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2021/04/01 17:39:56 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2021/04/01 17:39:56 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2021/04/01 17:39:57 CLSRSC-363: User ignored prerequisites during installation
2021/04/01 17:39:57 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2021/04/01 17:39:59 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2021/04/01 17:40:02 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2021/04/01 17:40:03 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2021/04/01 17:40:12 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2021/04/01 17:40:12 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2021/04/01 17:40:14 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2021/04/01 17:40:15 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2021/04/01 17:40:15 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2021/04/01 17:40:16 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2021/04/01 17:40:16 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2021/04/01 17:40:22 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2021/04/01 17:40:23 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2021/04/01 17:40:28 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.crsd' on 'fdcdb01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'fdcdb01'
CRS-2673: Attempting to stop 'ora.fdcdb01.vip' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.qosmserver' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.OCR.dg' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.cvu' on 'fdcdb01'
CRS-2677: Stop of 'ora.OCR.dg' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'fdcdb01'
CRS-2677: Stop of 'ora.cvu' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'fdcdb01'
CRS-2677: Stop of 'ora.asm' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.ASMNET2LSNR_ASM.lsnr' on 'fdcdb01'
CRS-2677: Stop of 'ora.fdcdb01.vip' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.qosmserver' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.ASMNET2LSNR_ASM.lsnr' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'fdcdb01'
CRS-2677: Stop of 'ora.ons' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'fdcdb01'
CRS-2677: Stop of 'ora.net1.network' on 'fdcdb01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'fdcdb01' has completed
CRS-2677: Stop of 'ora.crsd' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.crf' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'fdcdb01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.crf' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.storage' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'fdcdb01'
CRS-2677: Stop of 'ora.mdnsd' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.asm' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'fdcdb01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'fdcdb01'
CRS-2673: Attempting to stop 'ora.evmd' on 'fdcdb01'
CRS-2677: Stop of 'ora.ctssd' on 'fdcdb01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'fdcdb01'
CRS-2677: Stop of 'ora.cssd' on 'fdcdb01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'fdcdb01'
CRS-2677: Stop of 'ora.gipcd' on 'fdcdb01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2021/04/01 17:40:57 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.evmd' on 'fdcdb01'
CRS-2676: Start of 'ora.mdnsd' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.evmd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'fdcdb01'
CRS-2676: Start of 'ora.gpnpd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'fdcdb01'
CRS-2676: Start of 'ora.gipcd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fdcdb01'
CRS-2676: Start of 'ora.cssdmonitor' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.diskmon' on 'fdcdb01'
CRS-2676: Start of 'ora.diskmon' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.cssd' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.ctssd' on 'fdcdb01'
CRS-2676: Start of 'ora.ctssd' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fdcdb01'
CRS-2676: Start of 'ora.asm' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'fdcdb01'
CRS-2676: Start of 'ora.storage' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'fdcdb01'
CRS-2676: Start of 'ora.crf' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'fdcdb01'
CRS-2676: Start of 'ora.crsd' on 'fdcdb01' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-2664: Resource 'ora.OCR.dg' is already running on 'fdcdb01'
CRS-6017: Processing resource auto-start for servers: fdcdb01
CRS-2672: Attempting to start 'ora.scan1.vip' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.fdcdb01.vip' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.qosmserver' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.cvu' on 'fdcdb01'
CRS-2672: Attempting to start 'ora.ons' on 'fdcdb01'
CRS-2676: Start of 'ora.cvu' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.qosmserver' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.fdcdb01.vip' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'fdcdb01' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'fdcdb01'
CRS-2676: Start of 'ora.ons' on 'fdcdb01' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'fdcdb01' succeeded
CRS-6016: Resource auto-start has completed for server fdcdb01
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2021/04/01 17:42:26 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/04/01 17:42:26 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2021/04/01 17:42:28 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/04/01 17:42:48 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

执行成功。

节点二执行root.sh：

[root@fdcdb02 12.2.0]# /u01/app/12.2.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/12.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/fdcdb02/crsconfig/rootcrs_fdcdb02_2021-04-01_05-45-05PM.log
2021/04/01 17:45:06 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2021/04/01 17:45:06 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2021/04/01 17:45:07 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2021/04/01 17:45:07 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2021/04/01 17:45:08 CLSRSC-363: User ignored prerequisites during installation
2021/04/01 17:45:08 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2021/04/01 17:45:09 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2021/04/01 17:45:09 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2021/04/01 17:45:11 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2021/04/01 17:45:26 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2021/04/01 17:45:26 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2021/04/01 17:45:27 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2021/04/01 17:45:28 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2021/04/01 17:46:14 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2021/04/01 17:46:15 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2021/04/01 17:46:31 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2021/04/01 17:46:57 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2021/04/01 17:46:58 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb02'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb02' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2021/04/01 17:47:32 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2021/04/01 17:47:33 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb02'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb02' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fdcdb02'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'fdcdb02'
CRS-2677: Stop of 'ora.drivers.acfs' on 'fdcdb02' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fdcdb02' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2021/04/01 17:47:40 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'fdcdb02'
CRS-2672: Attempting to start 'ora.evmd' on 'fdcdb02'
CRS-2676: Start of 'ora.mdnsd' on 'fdcdb02' succeeded
CRS-2676: Start of 'ora.evmd' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'fdcdb02'
CRS-2676: Start of 'ora.gpnpd' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'fdcdb02'
CRS-2676: Start of 'ora.gipcd' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fdcdb02'
CRS-2676: Start of 'ora.cssdmonitor' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'fdcdb02'
CRS-2672: Attempting to start 'ora.diskmon' on 'fdcdb02'
CRS-2676: Start of 'ora.diskmon' on 'fdcdb02' succeeded
CRS-2676: Start of 'ora.cssd' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'fdcdb02'
CRS-2672: Attempting to start 'ora.ctssd' on 'fdcdb02'
CRS-2676: Start of 'ora.ctssd' on 'fdcdb02' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fdcdb02'
CRS-2676: Start of 'ora.asm' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'fdcdb02'
CRS-2676: Start of 'ora.storage' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'fdcdb02'
CRS-2676: Start of 'ora.crf' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'fdcdb02'
CRS-2676: Start of 'ora.crsd' on 'fdcdb02' succeeded
CRS-6017: Processing resource auto-start for servers: fdcdb02
CRS-2672: Attempting to start 'ora.net1.network' on 'fdcdb02'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'fdcdb02'
CRS-2672: Attempting to start 'ora.ASMNET2LSNR_ASM.lsnr' on 'fdcdb02'
CRS-2676: Start of 'ora.net1.network' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.ons' on 'fdcdb02'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'fdcdb02' succeeded
CRS-2676: Start of 'ora.ASMNET2LSNR_ASM.lsnr' on 'fdcdb02' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fdcdb02'
CRS-2676: Start of 'ora.ons' on 'fdcdb02' succeeded
CRS-2676: Start of 'ora.asm' on 'fdcdb02' succeeded
CRS-6016: Resource auto-start has completed for server fdcdb02
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2021/04/01 17:49:04 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/04/01 17:49:04 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2021/04/01 17:49:12 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/04/01 17:49:20 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

节点二root.sh执行成功。

[grid@fdcdb01 grid]$ ./gridSetup.sh -applyPSU /soft/32226491/                  
Preparing the home to patch...
Applying the patch /soft/32226491/...
Successfully applied the patch.
The log can be found at: /u01/app/oraInventory/logs/GridSetupActions2021-04-01_04-18-54PM/installerPatchActions_2021-04-01_04-18-54PM.log
Launching Oracle Grid Infrastructure Setup Wizard...

You can find the log of this install session at:
 /u01/app/oraInventory/logs/GridSetupActions2021-04-01_04-18-54PM/gridSetupActions2021-04-01_04-18-54PM.log

GRID软件安装成功。

查看OPatch补丁信息：

[grid@fdcdb02 OPatch]$ opatch lspatches
26839277;DBWLM RELEASE UPDATE 12.2.0.1.0(ID:170913) (26839277)
32231681;ACFS JAN 2021 RELEASE UPDATE 12.2.0.1.210119 (32231681)
32253903;TOMCAT RELEASE UPDATE 12.2.0.1.0(ID:RELEASE) (32253903)
31802727;OCW OCT 2020 RELEASE UPDATE 12.2.0.1.201020 (31802727)
32228578;Database Jan 2021 Release Update : 12.2.0.1.210119 (32228578)

OPatch succeeded.


[root@fdcdb01 32226491]# ll
total 136
drwxr-x--- 4 grid oinstall     48 Jan 10 12:34 26839277
drwxr-x--- 5 grid oinstall     62 Jan 10 12:27 31802727
drwxr-x--- 5 grid oinstall     81 Jan 10 12:25 32228578
drwxr-x--- 5 grid oinstall     62 Jan 10 12:29 32231681
drwxr-x--- 4 grid oinstall     48 Jan 10 12:29 32253903
drwxr-x--- 2 grid oinstall   4096 Jan 10 12:25 automation
-rw-rw-r-- 1 grid oinstall   5828 Jan 10 21:02 bundle.xml
-rw-r--r-- 1 grid oinstall 123977 Jan 10 20:42 README.html
-rw-r--r-- 1 grid oinstall      0 Jan 10 12:25 README.txt

补丁应用成功。

7、检查集群状态

ASM磁盘OCR检查：

5、创建ASM DATA盘

检查ASM磁盘组情况：

所有主机reboot检查集群是否运行正常。

四、安装ORACLE软件

1、安装ORACLE软件

[oracle@fdcdb01 database]$ ./runInstaller  
Starting Oracle Universal Installer...

Checking Temp space: must be greater than 500 MB.   Actual 261237 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 32767 MB    Passed
Checking monitor: must be configured to display at least 256 colors.    Actual 16777216    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2021-04-02_10-03-42AM. Please wait ...[

2、执行ROOT.SH

一节点：

[root@fdcdb01 database]# /u01/app/oracle/product/12.2.0/db/root.sh 
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/oracle/product/12.2.0/db

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

二节点：

[root@fdcdb02 grid]# /u01/app/oracle/product/12.2.0/db/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/oracle/product/12.2.0/db

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

执行成功。

Oracle软件安装成功。

2、安装Oracle RU补丁集（以下操作两节点都需要执行）

更换OPatch包：

##更换OPatch包：
##两个节点需要同时更换：
su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch version"
##OPatch Version: 12.2.0.1.6
 
##You must use the OPatch utility version 12.2.0.1.23 or later to apply this patch.
##这里不能删除原来的OPatch包，而是选择覆盖，否则安装会报错：

##解压补丁包
su - oracle -c "unzip /soft/p6880880_122010_Linux-x86-64.zip -d /u01/app/oracle/product/12.2.0/db/"
##选择A
su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch version"
##OPatch Version: 12.2.0.1.24

Validation of Oracle Inventory：

su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0/db/"


[root@fdcdb01 database]# su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0/db/"
Oracle Interim Patch Installer version 12.2.0.1.24
Copyright (c) 2021, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/12.2.0/db
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.2.0/db//oraInst.loc
OPatch version    : 12.2.0.1.24
OUI version       : 12.2.0.1.4
Log file location : /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatch/opatch2021-04-02_10-36-30AM_1.log

Lsinventory Output file location : /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatch/lsinv/lsinventory2021-04-02_10-36-30AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: fdcdb01
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1): 

Oracle Database 12c                                                  12.2.0.1.0
There are 1 products installed in this Oracle Home.


Installed Products (128): 

Assistant Common Files                                               12.2.0.1.0
BLASLAPACK Component                                                 12.2.0.1.0
Buildtools Common Files                                              12.2.0.1.0
Cluster Verification Utility Common Files                            12.2.0.1.0
Database Configuration and Upgrade Assistants                        12.2.0.1.0
Database Migration Assistant for Unicode                             12.2.0.1.0
Database SQL Scripts                                                 12.2.0.1.0
Database Workspace Manager                                           12.2.0.1.0
DB TOOLS Listener                                                    12.2.0.1.0
Deinstallation Tool                                                  12.2.0.1.0
Enterprise Edition Options                                           12.2.0.1.0
Expat libraries                                                       2.0.1.0.3
Generic Connectivity Common Files                                    12.2.0.1.0
Hadoopcore Component                                                 12.2.0.1.0
HAS Common Files                                                     12.2.0.1.0
HAS Files for DB                                                     12.2.0.1.0
Installation Common Files                                            12.2.0.1.0
Installation Plugin Files                                            12.2.0.1.0
Installer SDK Component                                              12.2.0.1.4
JAccelerator (COMPANION)                                             12.2.0.1.0
Java Development Kit                                                 1.8.0.91.0
LDAP Required Support Files                                          12.2.0.1.0
OLAP SQL Scripts                                                     12.2.0.1.0
Oracle Advanced Analytics                                            12.2.0.1.0
Oracle Advanced Security                                             12.2.0.1.0
Oracle Application Express                                           12.2.0.1.0
Oracle Bali Share                                                    11.1.1.6.0
Oracle Call Interface (OCI)                                          12.2.0.1.0
Oracle Clusterware RDBMS Files                                       12.2.0.1.0
Oracle Configuration Manager                                         12.1.2.0.0
Oracle Configuration Manager Client                                  10.3.2.1.0
Oracle Configuration Manager Deconfiguration                         10.3.1.0.0
Oracle Context Companion                                             12.2.0.1.0
Oracle Core Required Support Files                                   12.2.0.1.0
Oracle Core Required Support Files for Core DB                       12.2.0.1.0
Oracle Database 12c                                                  12.2.0.1.0
Oracle Database 12c                                                  12.2.0.1.0
Oracle Database 12c Multimedia Files                                 12.2.0.1.0
Oracle Database Deconfiguration                                      12.2.0.1.0
Oracle Database Gateway for ODBC                                     12.2.0.1.0
Oracle Database Utilities                                            12.2.0.1.0
Oracle Database Vault option                                         12.2.0.1.0
Oracle DBCA Deconfiguration                                          12.2.0.1.0
Oracle Extended Windowing Toolkit                                    11.1.1.6.0
Oracle Globalization Support                                         12.2.0.1.0
Oracle Globalization Support                                         12.2.0.1.0
Oracle Globalization Support For Core                                12.2.0.1.0
Oracle Help for Java                                                 11.1.1.7.0
Oracle Help Share Library                                            11.1.1.7.0
Oracle Ice Browser                                                   11.1.1.7.0
Oracle Internet Directory Client                                     12.2.0.1.0
Oracle Java Client                                                   12.2.0.1.0
Oracle JDBC Server Support Package                                   12.2.0.1.0
Oracle JDBC/OCI Instant Client                                       12.2.0.1.0
Oracle JDBC/THIN Interfaces                                          12.2.0.1.0
Oracle JFC Extended Windowing Toolkit                                11.1.1.6.0
Oracle JVM                                                           12.2.0.1.0
Oracle JVM For Core                                                  12.2.0.1.0
Oracle Label Security                                                12.2.0.1.0
Oracle LDAP administration                                           12.2.0.1.0
Oracle Locale Builder                                                12.2.0.1.0
Oracle Message Gateway Common Files                                  12.2.0.1.0
Oracle Multimedia                                                    12.2.0.1.0
Oracle Multimedia Client Option                                      12.2.0.1.0
Oracle Multimedia Java Advanced Imaging                              12.2.0.1.0
Oracle Multimedia Locator                                            12.2.0.1.0
Oracle Multimedia Locator Java Required Support Files                12.2.0.1.0
Oracle Multimedia Locator RDBMS Files                                12.2.0.1.0
Oracle Net                                                           12.2.0.1.0
Oracle Net Listener                                                  12.2.0.1.0
Oracle Net Required Support Files                                    12.2.0.1.0
Oracle Net Services                                                  12.2.0.1.0
Oracle Netca Client                                                  12.2.0.1.0
Oracle Notification Service                                          12.2.0.1.0
Oracle Notification Service for Instant Client                       12.2.0.1.0
Oracle ODBC Driver                                                   12.2.0.1.0
Oracle ODBC Driverfor Instant Client                                 12.2.0.1.0
Oracle OLAP                                                          12.2.0.1.0
Oracle OLAP API                                                      12.2.0.1.0
Oracle OLAP RDBMS Files                                              12.2.0.1.0
Oracle One-Off Patch Installer                                       12.2.0.1.6
Oracle Partitioning                                                  12.2.0.1.0
Oracle Programmer                                                    12.2.0.1.0
Oracle R Enterprise Server Files                                     12.2.0.1.0
Oracle RAC Deconfiguration                                           12.2.0.1.0
Oracle RAC Required Support Files-HAS                                12.2.0.1.0
Oracle Real Application Testing                                      12.2.0.1.0
Oracle Recovery Manager                                              12.2.0.1.0
Oracle Scheduler Agent                                               12.2.0.1.0
Oracle Security Developer Tools                                      12.2.0.1.0
Oracle Spatial and Graph                                             12.2.0.1.0
Oracle SQL Developer                                                 12.2.0.1.0
Oracle Starter Database                                              12.2.0.1.0
Oracle Text                                                          12.2.0.1.0
Oracle Text ATG Language Support Files                               12.2.0.1.0
Oracle Text Required Support Files                                   12.2.0.1.0
Oracle Universal Connection Pool                                     12.2.0.1.0
Oracle Universal Installer                                           12.2.0.1.4
Oracle USM Deconfiguration                                           12.2.0.1.0
Oracle Wallet Manager                                                12.2.0.1.0
Oracle XML Development Kit                                           12.2.0.1.0
Oracle XML Query                                                     12.2.0.1.0
oracle.swd.commonlogging                                             13.3.0.0.0
oracle.swd.opatchautodb                                              12.2.0.1.5
oracle.swd.oui.core.min                                              12.2.0.1.4
Parser Generator Required Support Files                              12.2.0.1.0
Perl Interpreter                                                     5.22.0.0.0
Perl Modules                                                         5.22.0.0.0
PL/SQL                                                               12.2.0.1.0
PL/SQL Embedded Gateway                                              12.2.0.1.0
Platform Required Support Files                                      12.2.0.1.0
Precompiler Common Files                                             12.2.0.1.0
Precompiler Common Files for Core                                    12.2.0.1.0
Precompiler Required Support Files                                   12.2.0.1.0
Precompilers                                                         12.2.0.1.0
RDBMS Required Support Files                                         12.2.0.1.0
RDBMS Required Support Files for Instant Client                      12.2.0.1.0
Required Support Files                                               12.2.0.1.0
Secure Socket Layer                                                  12.2.0.1.0
SQL*Plus                                                             12.2.0.1.0
SQL*Plus Files for Instant Client                                    12.2.0.1.0
SQL*Plus Required Support Files                                      12.2.0.1.0
SQLJ Runtime                                                         12.2.0.1.0
SSL Required Support Files for InstantClient                         12.2.0.1.0
Tracle File Analyzer                                                 12.2.0.1.0
XDK Required Support Files                                           12.2.0.1.0
XML Parser for Java                                                  12.2.0.1.0
XML Parser for Oracle JVM                                            12.2.0.1.0
There are 128 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


--------------------------------------------------------------------------------

OPatch succeeded.
[root@fdcdb01 database]#

Run OPatch Conflict Check：

su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/32226491/32228578" 
su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/32226491/31802727"


[root@fdcdb02 soft]# su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/32226491/32228578" 
Oracle Interim Patch Installer version 12.2.0.1.24
Copyright (c) 2021, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/oracle/product/12.2.0/db
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.2.0/db/oraInst.loc
OPatch version    : 12.2.0.1.24
OUI version       : 12.2.0.1.4
Log file location : /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatch/opatch2021-04-02_10-42-26AM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[root@fdcdb02 soft]# su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /soft/32226491/31802727"
Oracle Interim Patch Installer version 12.2.0.1.24
Copyright (c) 2021, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/oracle/product/12.2.0/db
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.2.0/db/oraInst.loc
OPatch version    : 12.2.0.1.24
OUI version       : 12.2.0.1.4
Log file location : /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatch/opatch2021-04-02_10-42-42AM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.

Run OPatch SystemSpace Check：

cat <<EOF >>/tmp/patch_list_dbhome.txt
/soft/32226491/32228578 
/soft/32226491/31802727
EOF
su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt"


[root@fdcdb01 database]# su - oracle -c "/u01/app/oracle/product/12.2.0/db/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt"
Oracle Interim Patch Installer version 12.2.0.1.24
Copyright (c) 2021, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/oracle/product/12.2.0/db
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.2.0/db/oraInst.loc
OPatch version    : 12.2.0.1.24
OUI version       : 12.2.0.1.4
Log file location : /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatch/opatch2021-04-02_10-48-47AM_1.log

Invoking prereq "checksystemspace"

Prereq "checkSystemSpace" passed.

OPatch succeeded.

opatchauto：

##root用户执行，按顺序在节点1，2执行，不可同时执行
/u01/app/oracle/product/12.2.0/db/OPatch/opatchauto apply /soft/32226491 -oh /u01/app/oracle/product/12.2.0/db


##补丁安装过程记录
[root@fdcdb01 database]# /u01/app/oracle/product/12.2.0/db/OPatch/opatchauto apply /soft/32226491 -oh /u01/app/oracle/product/12.2.0/db

OPatchauto session is initiated at Fri Apr  2 10:53:11 2021

System initialization log file is /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatchautodb/systemconfig2021-04-02_10-53-13AM.log.

Session log file is /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatchauto/opatchauto2021-04-02_10-53-47AM.log
The id for this session is GM4I

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0/db
Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/db


Executing patch validation checks on home /u01/app/oracle/product/12.2.0/db
Patch validation checks successfully completed on home /u01/app/oracle/product/12.2.0/db


Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0/db
No step execution required.........
 

Preparing to bring down database service on home /u01/app/oracle/product/12.2.0/db
No step execution required.........
 

Performing prepatch operation on home /u01/app/oracle/product/12.2.0/db
Perpatch operation completed successfully on home /u01/app/oracle/product/12.2.0/db


Start applying binary patch on home /u01/app/oracle/product/12.2.0/db
Binary patch applied successfully on home /u01/app/oracle/product/12.2.0/db


Performing postpatch operation on home /u01/app/oracle/product/12.2.0/db
Postpatch operation completed successfully on home /u01/app/oracle/product/12.2.0/db


Preparing home /u01/app/oracle/product/12.2.0/db after database service restarted
No step execution required.........
 

Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0/db
No step execution required.........
 
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:fdcdb01
RAC Home:/u01/app/oracle/product/12.2.0/db
Version:12.2.0.1.0
Summary:

==Following patches were SKIPPED:

Patch: /soft/32226491/32231681
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /soft/32226491/26839277
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /soft/32226491/32253903
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /soft/32226491/31802727
Log: /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatchauto/core/opatch/opatch2021-04-02_10-54-02AM_1.log

Patch: /soft/32226491/32228578
Log: /u01/app/oracle/product/12.2.0/db/cfgtoollogs/opatchauto/core/opatch/opatch2021-04-02_10-54-02AM_1.log



OPatchauto session completed at Fri Apr  2 10:56:29 2021
Time taken to complete the session 3 minutes, 18 seconds

查看Oracle已安装补丁集：

[oracle@fdcdb01 ~]$ opatch lspatches
32228578;Database Jan 2021 Release Update : 12.2.0.1.210119 (32228578)
31802727;OCW OCT 2020 RELEASE UPDATE 12.2.0.1.201020 (31802727)

OPatch succeeded.

补丁安装成功。

五、建库

dbca

报错：[BasicStep.handleNonIgnorableError:510] ORA-01031: insufficient privileges

##怀疑可能是oracle用户缺少asmadmin组，授予组之后，点击ignore
usermod -a -G asmadmin oracle

CDB实例创建完成。

创建PDB：

SQL> create pluggable database fdcdb admin user admin identified by oracle;

Pluggable database created.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 FDCDB                          MOUNTED
SQL> alter pluggable database all open;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 FDCDB                          READ WRITE NO
SQL> alter session set container=fdcdb;

Session altered.

SQL> show user
USER is "SYS"
SQL> show con_name

CON_NAME
------------------------------
FDCDB

SQL> grant dba to admin;

Grant succeeded.

启动PDB的几种方式：

--1.手动登录每个节点去启动pdb
--节点一启动pdb
sqlplus / as sysdba
show pdbs
--启动单个pdb
alter pluggable database fdcdb open;
--启动所有pdb
alter pluggable database all open;

--此时节点二的pdb仍然是mount状态
--需要再次手动启动
alter pluggable database all open;

--2.创建触发器PDB随CDB启动
--使用SYS用户创建如下触发器即可：
CREATE TRIGGER open_all_pdbs
   AFTER STARTUP
   ON DATABASE
BEGIN
   EXECUTE IMMEDIATE 'alter pluggable database all open';
END open_all_pdbs;
/

添加TNS：

cat <<EOF >> /u01/app/oracle/product/12.2.0/db/network/admin/tnsnames.ora
FDCDB =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = fdcdb01)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = fdcdb)
    )
  )
EOF

测试连接：

[oracle@fdcdb02 ~]$ sqlplus admin/oracle@fdcdb

SQL*Plus: Release 12.2.0.1.0 Production on Fri Apr 2 12:29:54 2021

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> 
SQL> 
SQL> show con_name

CON_NAME
------------------------------
FDCDB
SQL> show user
USER is "ADMIN"

七、检查集群、数据库、监听运行情况

1、集群运行情况检查：

[root@fdcdb01 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

[root@fdcdb01 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.ASMNET2LSNR_ASM.lsnr
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.DATA.dg
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.OCR.dg
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.net1.network
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.ons
               ONLINE  ONLINE       fdcdb01                  STABLE
               ONLINE  ONLINE       fdcdb02                  STABLE
ora.proxy_advm
               OFFLINE OFFLINE      fdcdb01                  STABLE
               OFFLINE OFFLINE      fdcdb02                  STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       fdcdb02                  STABLE
ora.MGMTLSNR
      1        OFFLINE OFFLINE                               STABLE
ora.asm
      1        ONLINE  ONLINE       fdcdb01                  Started,STABLE
      2        ONLINE  ONLINE       fdcdb02                  Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       fdcdb02                  STABLE
ora.fdcdb01.vip
      1        ONLINE  ONLINE       fdcdb01                  STABLE
ora.fdcdb02.vip
      1        ONLINE  ONLINE       fdcdb02                  STABLE
ora.fdcinst.db
      1        ONLINE  ONLINE       fdcdb01                  Open,HOME=/u01/app/o
                                                             racle/product/12.2.0
                                                             /db,STABLE
      2        ONLINE  ONLINE       fdcdb02                  Open,HOME=/u01/app/o
                                                             racle/product/12.2.0
                                                             /db,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       fdcdb02                  STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       fdcdb02                  STABLE
--------------------------------------------------------------------------------

2、数据库运行情况

[oracle@fdcdb01 ~]$ srvctl status database -d fdcinst
Instance fdcinst1 is running on node fdcdb01
Instance fdcinst2 is running on node fdcdb02


SQL> select inst_id,instance_name,status from gv$instance;

   INST_ID INSTANCE_NAME    STATUS
---------- ---------------- ------------
         1 fdcinst1         OPEN
         2 fdcinst2         OPEN

col name for a30
SQL> select con_id,name,open_mode from gv$pdbs;
    CON_ID NAME                           OPEN_MODE
---------- ------------------------------ ----------
         2 PDB$SEED                       READ ONLY
         4 FDCDB                          READ WRITE
         2 PDB$SEED                       READ ONLY
         4 FDCDB                          READ WRITE

3、监听运行情况

[oracle@fdcdb01 ~]$ ps -ef|grep tns           
root       332     2  0 Apr01 ?        00:00:00 [netns]
oracle   20413 16511  0 13:57 pts/2    00:00:00 grep --color=auto tns
grid     23865     1  0 Apr01 ?        00:00:11 /u01/app/12.2.0/grid/bin/tnslsnr ASMNET2LSNR_ASM -no_crs_notify -inherit
grid     23869     1  0 Apr01 ?        00:00:11 /u01/app/12.2.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit
grid     24594     1  0 Apr01 ?        00:00:01 /u01/app/12.2.0/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit

[oracle@fdcdb01 ~]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): fdcdb01,fdcdb02

[oracle@fdcdb01 ~]$ srvctl status listener -l ASMNET1LSNR_ASM
Listener ASMNET1LSNR_ASM is enabled
Listener ASMNET1LSNR_ASM is running on node(s): fdcdb01,fdcdb02

[oracle@fdcdb01 ~]$ srvctl status listener -l ASMNET2LSNR_ASM
Listener ASMNET2LSNR_ASM is enabled
Listener ASMNET2LSNR_ASM is running on node(s): fdcdb01,fdcdb02


[grid@fdcdb02 ~]$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node fdcdb02

八、配置ORACLE参数

1、用户名密码永久生效

--查看用户天数限制
set line222
col profile for a20
col RESOURCE_NAME for a20
col LIMIT for a20
SELECT * FROM dba_profiles WHERE profile ='DEFAULT' AND resource_name = 'PASSWORD_LIFE_TIME';

PROFILE              RESOURCE_NAME        RESOURCE LIMIT                COM INH IMP
-------------------- -------------------- -------- -------------------- --- --- ---
DEFAULT              PASSWORD_LIFE_TIME   PASSWORD 180                  NO  NO  NO

--更改为无限制
ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

SQL> ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

Profile altered.

--已修改为无限制
SQL> SELECT * FROM dba_profiles WHERE profile ='DEFAULT' AND resource_name = 'PASSWORD_LIFE_TIME';

PROFILE              RESOURCE_NAME        RESOURCE LIMIT                COM INH IMP
-------------------- -------------------- -------- -------------------- --- --- ---
DEFAULT              PASSWORD_LIFE_TIME   PASSWORD UNLIMITED            NO  NO  NO

--修改PDB的用户密码天数限制
SQL> ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

Profile altered.

SQL> SELECT * FROM dba_profiles WHERE profile ='DEFAULT' AND resource_name = 'PASSWORD_LIFE_TIME';

PROFILE              RESOURCE_NAME        RESOURCE LIMIT                COM INH IMP
-------------------- -------------------- -------- -------------------- --- --- ---
DEFAULT              PASSWORD_LIFE_TIME   PASSWORD UNLIMITED            NO  NO  NO

2、sqlnet.ora配置允许低版本client连接

cat <<EOF >>$ORACLE_HOME/network/admin/sqlnet.ora
SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8
SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
EOF

3、开启归档、配置删除归档脚本

--设置归档路径
SQL> alter system set log_archive_dest_1='location=+DATA' sid='*';

System altered.

SQL> show parameter log_archive_dest_1

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_1                   string      location=+DATA
log_archive_dest_10                  string
log_archive_dest_11                  string
log_archive_dest_12                  string
log_archive_dest_13                  string
log_archive_dest_14                  string
log_archive_dest_15                  string
log_archive_dest_16                  string
log_archive_dest_17                  string
log_archive_dest_18                  string
log_archive_dest_19                  string

SQL> archive log list
Database log mode              No Archive Mode
Automatic archival             Disabled
Archive destination            +DATA
Oldest online log sequence     8
Current log sequence           9

--查看pdbs启动状态
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 FDCDB                          READ WRITE NO
SQL> 
--关闭一节点所有pdb
SQL> alter pluggable database all close;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 FDCDB                          MOUNTED

--查询节点2的pdbs状态
SQL> select inst_id,con_id,name,open_mode from gv$pdbs;

   INST_ID     CON_ID NAME                                                                                                                             OPEN_MODE
---------- ---------- -------------------------------------------------------------------------------------------------------------------------------- ----------
         1          2 PDB$SEED                                                                                                                         READ ONLY
         1          4 FDCDB                                                                                                                            MOUNTED
         2          2 PDB$SEED                                                                                                                         READ ONLY
         2          4 FDCDB                                                                                                                            READ WRITE


--关闭节点2的pdbs
SQL> alter pluggable database all close;

Pluggable database altered.


--关闭数据库实例fdcinst
[oracle@fdcdb02 ~]$ srvctl stop database -d fdcinst -o immediate

--开启到mount状态
[oracle@fdcdb02 ~]$ srvctl start database -d fdcinst -o mount

--开启归档模式
SQL> select open_mode from v$database;

OPEN_MODE
--------------------
MOUNTED

SQL> archive log list
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            +DATA
Oldest online log sequence     1
Next log sequence to archive   1
Current log sequence           1

--两节点开启到open状态
alter database open;

--布置归档删除脚本
mkdir -p /home/oracle/script/log
vi /home/oracle/script/del_arch.sh
#!/bin/bash
. ~/.bash_profile
DAY_TAG=`date +"%Y-%m-%d"`

rman target / nocatalog msglog /home/oracle/pst/script/log/del_arch_$DAY_TAG.log<<EOF
crosscheck archivelog all;
delete noprompt archivelog until time '(sysdate-7)';
delete noprompt force archivelog until time 'SYSDATE-10';
EOF

--写入crontab
chmod +x /home/oracle/script/del_arch.sh
echo "00 00 * * * /home/oracle/script/del_arch.sh" >> /var/spool/cron/oracle

4、配置ORACLE数据库随主机启动自动启动

--查看当前数据库是否随主机启动
[root@fdcdb01 ~]# crsctl stat res ora.fdcinst.db -p|grep AUTO_START
AUTO_START=restore

--设置随主机启动
[root@fdcdb01 ~]# crsctl modify res ora.fdcinst.db -attr AUTO_START=always -unsupported

[root@fdcdb01 ~]# crsctl stat res ora.fdcinst.db -p|grep AUTO_START
AUTO_START=always

码农公寓