1. 前提条件
1.1 认证操作系统
Certification Information for Oracle Database on Linux x86-64 (Doc ID 1304727.2)
1)HP-UX 11i V3 patch Bundle Sep/ 2008 (B.11.31.0809.326a) or higher (Part Number E10851-01)
1.2 系统硬件和软件环境检查
1) 系统运行级别
# who -r
--Check the daemon (pwgrd) is running
# ps -aef |grep pwgrd
2) 操作系统版本要求(参考文档ID169706.1)
# uname -a
# /bin/getconf KERNEL_BITS
HP-UX 11i V3 patch Bundle Sep/ 2008 (B.11.31.0809.326a) or higher (Part Number E10851-01)
3) 物理内存至少4G
# /usr/contrib/bin/machinfo|grep -i Memory
4)swap大小(大于16G则设置16G,不满足需要扩容内存。物理内存1G-2G,建议设置为物理内存1.5倍,内存2G-16G,建议设置为物理内存一样大小。)
# /usr/sbin/swapinfo -a
5) tmp至少1G,不满足需扩容
# bdf /tmp
6)软件目录大小建议80g
7)系统软件包
For HP-UX 11i V3 (11.31):
PHCO_40381 11.31 Disk Owner Patch
PHCO_41479 11.31 (fixes an 11.2.0.2 ASM disk discovery issue)
PHKL_38038 VM patch - hot patching/Core file creation directory
PHKL_38938 11.31 SCSI cumulative I/O patch
PHKL_39351 Scheduler patch : post wait hang
PHSS_36354 11.31 assembler patch
PHSS_37042 11.31 hppac (packed decimal)
PHSS_37959 Libcl patch for alternate stack issue fix (QXCR1000818011)
PHSS_39094 11.31 linker + fdp cumulative patch
PHSS_39100 11.31 Math Library Cumulative Patch
PHSS_39102 11.31 Integrity Unwind Library
PHSS_38141 11.31 aC++ Runtime
Pro*C/C++, Oracle Call Interface, Oracle C++
Pro*C/C++, Oracle Call Interface, Oracle C++ Call Interface, Oracle XML Developer's Kit (XDK):-
Patch for HP-UX 11i V3 (11.31) on HP-UX Itanium:-
PHSS_39824 - 11.31 HP C/aC++ Compiler (A.06.23) patch
PHKL_40941 - Scheduler patch : post wait hang
-- 查看系统包是否安装
/usr/sbin/swlist -l bundle
/usr/sbin/swlist -l patch|grep PHCO_40381
/usr/sbin/swlist -l patch|grep PHCO_41479
/usr/sbin/swlist -l patch|grep PHKL_38038
/usr/sbin/swlist -l patch|grep PHKL_38938
/usr/sbin/swlist -l patch|grep PHKL_39351
/usr/sbin/swlist -l patch|grep PHSS_36354
/usr/sbin/swlist -l patch|grep PHSS_37042
/usr/sbin/swlist -l patch|grep PHSS_37959
/usr/sbin/swlist -l patch|grep PHSS_39094
/usr/sbin/swlist -l patch|grep PHSS_39100
/usr/sbin/swlist -l patch|grep PHSS_39102
/usr/sbin/swlist -l patch|grep PHSS_38141
/usr/sbin/swlist -l patch|grep PHSS_39824
/usr/sbin/swlist -l patch|grep PHKL_40941
-- 查看特定补丁包的替代包
/usr/sbin/swlist -l patch -a supersedes | grep -in PHCO_40381
/usr/sbin/swlist -l patch -a supersedes | grep -in PHCO_41479
/usr/sbin/swlist -l patch -a supersedes | grep -in PHKL_38038
/usr/sbin/swlist -l patch -a supersedes | grep -in PHKL_38938
/usr/sbin/swlist -l patch -a supersedes | grep -in PHKL_39351
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_36354
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_37042
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_37959
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_39094
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_39100
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_39102
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_38141
/usr/sbin/swlist -l patch -a supersedes | grep -in PHSS_39824
/usr/sbin/swlist -l patch -a supersedes | grep -in PHKL_40941
1.2 Lun分配建议
1) ocr voting:3个1G的lun,ASM对应划分1个normal的dg
2) 控制文件:3个1G的lun,ASM对应划分3个dg
3)redo:2个48G的lun(建议raid 1+0,尽量不用raid5),ASM对应划分2个dg
4) 数据文件:最多500G 1个lun,ASM对应划分1个dg,注意单个dg不超过10t
1.3 网络配置
--查看网卡信息
# lanscan
--网卡速率 lanadmin -s <网卡号>
# lanadmin -s 0
--通过“lanadmin -x <网卡号>”命令确定网卡实际生效的速率和通讯方式
# lanadmin –x 0
参数说明如下:
1) Current Speed:实际生效速率,单位是Mbit/s。
2) Full-Duplex:全双工通讯方式。
3) Half-Duplex:半双工通讯方式。
4) Autonegotiation:自适应工作方式。
1.4 IP需求
每个节点一个public IP,一个virtual IP,一个private IP;
整个集群需要至少一个SCAN IP;
# cat /etc/rc.config.d/netconf |grep -v ^#|perl -pe 's/^\s+$//g'
1.5 配置/etc/hosts文件
--查看IP地址设置
# lanscan
--显示各个接口的ip和状态 (需要保证每台节点上使用的接口具有相同的名字)
# netstat -in
--显示指定接口的当前配置
# ifconfig lanX
--查看路由
# netstat -rn
注1:hostname不允许使用下划线_,建议统一使用小写字母和数字组合,否则安装过程可能会报错。
cp /etc/hosts{,_$(date +%Y%m%d)}
cat >> /etc/hosts <EOF
127.0.0.1 loopback localhost
::1 loopback localhost
# Public Network - (bond0)
192.168.8.145 orcl1
192.168.8.146 orcl2
# Private Interconnect - (bond1)
192.168.168.145 orcl1-priv
192.168.168.146 orcl2-priv
# Public Virtual IP (VIP) addresses - (bond0:X)
192.168.8.147 orcl1-vip
192.168.8.148 orcl2-vip
# SCAN IP - (bond0:X)
192.168.8.149 orcl-scan
EOF
1.7 lib配置
--所有节点设置:
# cd /usr/lib
# ln -s libX11.3 libX11.sl
# ln -s libXIE.2 libXIE.sl
# ln -s libXext.3 libXext.sl
# ln -s libXhp11.3 libXhp11.sl
# ln -s libXi.3 libXi.sl
# ln -s libXm.4 libXm.sl
# ln -s libXp.2 libXp.sl
# ln -s libXt.3 libXt.sl
# ln -s libXtst.2 libXtst.sl
1.8 内核参数配置要求
NPROC 4096
KSI_ALLOC_MAX (NPROC*8)
EXECUTABLE_STACK=0
MAX_THREAD_PROC 1024
MAXDSIZ 1073741824
MAXDSIZ_64BIT 2147483648
MAXTSIZE_64BIT 1073741824
MAXSSIZ 134217728 bytes
MAXSSIZ_64BIT 1073741824
MAXUPRC ((NPROC*9)/10)+1
MSGMAP (MSGTQL+2) *
MSGMNI (NPROC)
MSGSEG 32767 *
MSGTQL (NPROC)
NCSIZE (NINODE+1024)
NFILE (15*NPROC+2048) *
NFLOCKS (NPROC)
NINODE (8*NPROC+2048)
NKTHREAD (((NPROC*7)/4)+16)
SEMMNI (NPROC)
SEMMNS (SEMMNI*2)
SEMMNU (NPROC - 4)
SEMVMX 32767
SHMMAX AvailMem
SHMMNI 4096
SHMSEG 512
VPS_CEILING 64
-- 检查内核配置参数
# /usr/sbin/kctune –d
/usr/sbin/kctune ksi_alloc_max
/usr/sbin/kctune executable_stack
/usr/sbin/kctune max_thread_proc
/usr/sbin/kctune maxdsiz
/usr/sbin/kctune maxdsiz_64bit
/usr/sbin/kctune maxfiles
/usr/sbin/kctune maxfiles_lim
/usr/sbin/kctune maxssiz
/usr/sbin/kctune maxssiz_64bit
/usr/sbin/kctune maxuprc
/usr/sbin/kctune msgmni
/usr/sbin/kctune msgtql
/usr/sbin/kctune ncsize
/usr/sbin/kctune ninode
/usr/sbin/kctune nflocks
/usr/sbin/kctune nkthread
/usr/sbin/kctune nproc
/usr/sbin/kctune semmni
/usr/sbin/kctune semmns
/usr/sbin/kctune semmnu
/usr/sbin/kctune semvmx
/usr/sbin/kctune shmmax
/usr/sbin/kctune shmmni
/usr/sbin/kctune shmseg
/usr/sbin/kctune vps_ceiling
/usr/bin/ndd /dev/tcp tcp_largest_anon_port
/usr/bin/ndd /dev/udp udp_largest_anon_port
--内核参数设置命令:(kctune PARAMETRE=value)
/usr/sbin/kctune -h maxfiles_lim=32767
/usr/sbin/kctune -h maxssiz_64bit=1073741824
/usr/sbin/kctune -h msgmni=4200
/usr/sbin/kctune -h ncsize=36672
/usr/sbin/kctune -h ninode=35648
/usr/sbin/kctune -h nflocks=4200
/usr/sbin/kctune -h nkthread=7366
-- 应用参数设置
/usr/sbin/kcweb -F
-- 列出改变的参数
kmtune
1.9 UDP/TCP参数
--使用ndd命令检查确认UDP/TCP 端口范围是否为9000-65500
/usr/bin/ndd /dev/tcp tcp_smallest_anon_port tcp_largest_anon_port
/usr/bin/ndd /dev/udp udp_smallest_anon_port udp_largest_anon_port
--编辑/etc/rc.config.d/nddconf文件更新配置,如下:
TRANSPORT_NAME[0]=tcp
NDD_NAME[0]=tcp_smallest_anon_port
NDD_VALUE[0]=9000
TRANSPORT_NAME[1]=tcp
NDD_NAME[1]=tcp_largest_anon_port
NDD_VALUE[1]=65500
TRANSPORT_NAME[2]=udp
NDD_NAME[2]=udp_smallest_anon_port
NDD_VALUE[2]=9000
TRANSPORT_NAME[3]=udp
NDD_NAME[3]=udp_largest_anon_port
NDD_VALUE[3]=65500
1.10 开启异步IO
# ls -l /dev/async (默认是开启的)
crw-rw-rw- 1 bin bin 101 0x000000 Apr 29 2013 /dev/async
--若没有开启异步I/O,则可以通过一下方法开启
-- 创建/dev/async character device
/sbin/mknod /dev/async c 101 0x0
chown oracle:dba /dev/async
chmod 660 /dev/async
1.11 创建用户和组并赋予权限
-- 创建用户组:
# /usr/sbin/groupadd -g 1000 oinstall
# /usr/sbin/groupadd -g 1020 asmadmin
# /usr/sbin/groupadd -g 1021 asmdba
# /usr/sbin/groupadd -g 1022 asmoper
# /usr/sbin/groupadd -g 1031 dba
# /usr/sbin/groupadd -g 1032 oper
-- 创建用户:
/usr/sbin/useradd –u 1101 -g oinstall -G dba,asmdba,oper -m oracle
/usr/sbin/useradd –u 1100 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba –m grid
-- 修改用户密码
passwd oracle
passwd grid
-- su测试一下密码是否修改成功
-- 检查确认
id oracle
uid=1101(oracle) gid=1000(oinstall) groups=1000(oinstall),1031(dba),1021(asmdba),1032(oper)
id grid
uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1031(dba),1020(asmadmin),1022(asmoper),1021(asmdba)
-- 检查/etc/privgroup 是否存在,
-- 否则,创建并编辑文件
vi /etc/privgroup
oinstall RTPRIO MLOCK RTSCHED
-- 若存在,则授权
# /usr/sbin/setprivgrp oinstall RTPRIO MLOCK RTSCHED
if [ -f "/etc/privgroup" ]; then
/usr/sbin/setprivgrp oinstall RTPRIO MLOCK RTSCHED
else
cat > /etc/privgroup <<EOF
oinstall RTPRIO MLOCK RTSCHED
EOF
fi
--检查确认oinstall Oracle Inventory group授权
# /usr/bin/getprivgrp oinstall
oinstall: RTPRIO MLOCK RTSCHED
1.12 创建目录
-- 创建目录
mkdir -p /oracle/app/grid
mkdir -p /oracle/app/11.2.0/grid
chown -R grid:oinstall /oracle
mkdir -p /oracle/app/oracle
mkdir -p /oracle/app/oracle/product/11.2.0/db_1
chown oracle:oinstall /oracle/app/oracle
chown oracle:oinstall /oracle/app/oracle/product/11.2.0/db_1
chmod -R 775 /oracle
-- 检查目录属性,确认修改是否正确
ls -ltrd /oracle/app/grid /oracle/app/11.2.0/grid /oracle/app/oracle /oracle /oracle/app/oracle/product/11.2.0/db_1
1.13 配置grid、oracle用户环境变量
su - grid
vi $HOME/.profile
export ORACLE_SID=+ASM1
export ORACLE_BASE=/oracle/app/grid
export ORACLE_HOME=/oracle/app/11.2.0/grid/product/db_1
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH
su - oracle
$HOME/.bash_profile
export ORACLE_SID=orcl1
export ORACLE_BASE=/oracle/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORA_CRS_HOME=/oracle/app/11.2.0/grid/product/db_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:${ORA_CRS_HOME}/bin:$PATH:$HOME/dba
umask 022
export TNS_ADMIN=${ORA_CRS_HOME}/network/admin
其它节点参照配置
1.14 共享磁盘配置
从存储划分共享存储至两台主机,通过多路径软件识别。
# 注意:在使用这些lun之前,先进行格式化,如下操作
dd if=/dev/zero of=/dev/rdisk/disk3 bs=8192 count=1280
1)磁盘聚合路径信息
# ioscan –fnNkCdisk
2)查看磁盘大小
# diskinfo /dev/rdisk/disk3
3)找出对应盘符的唯一ID,并找出其WWN(盘符应该是字符设备文件)
# scsimgr lun_map -D /dev/rdisk/disk3 |grep WWID
4)检查确认磁盘未被VG使用
# vgdisplay –v
5)根据一一对应的WWN值,将相应WWN对应所有节点的盘符设置成一致:
for i in `ls -l /dev/rdisk/disk* |awk '{print $NF}'|grep -v _p`
do
echo $i
scsimgr lun_map -D $i |grep WWID
diskinfo $i |grep size
done
-- 举例:
ls -l /dev/rdisk/disk*
crw-r----- 1 bin sys 13 0x000006 Aug 1 22:16 /dev/rdisk/disk4
crw-r----- 1 bin sys 13 0x000007 Aug 1 22:16 /dev/rdisk/disk5
crw-r----- 1 bin sys 13 0x000008 Aug 1 22:16 /dev/rdisk/disk6
crw-r----- 1 bin sys 13 0x000008 Aug 1 22:16 /dev/rdisk/disk7
-- node1
/dev/rdisk/disk4
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c000000a8
size: 209715840 Kbytes
/dev/rdisk/disk5
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c00000f25
size: 1049280 Kbytes
/dev/rdisk/disk6
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c00000f24
size: 1049280 Kbytes
/dev/rdisk/disk7
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c00000f26
size: 1049280 Kbytes
-- node2
/dev/rdisk/disk14
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c000000a8
size: 209715840 Kbytes
/dev/rdisk/disk15
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c00000f24
size: 1049280 Kbytes
/dev/rdisk/disk16
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c00000f26
size: 1049280 Kbytes
/dev/rdisk/disk17
World Wide Identifier(WWID) = 0x60060e8006d69c000000d69c00000f25
size: 1049280 Kbytes
-- 绑定盘符
-- node1
mkdir -p /dev/oracle
mknod /dev/oracle/ocr1 c 13 0x000007
mknod /dev/oracle/ocr2 c 13 0x000008
mknod /dev/oracle/ocr3 c 13 0x000009
mknod /dev/oracle/data1 c 13 0x000006
-- node2
crw-r----- 1 bin sys 13 0x000005 Aug 2 04:33 /dev/rdisk/disk14
crw-r----- 1 bin sys 13 0x000006 Aug 2 04:33 /dev/rdisk/disk15
crw-r----- 1 bin sys 13 0x000007 Aug 2 04:33 /dev/rdisk/disk16
crw-r----- 1 bin sys 13 0x000008 Aug 2 04:33 /dev/rdisk/disk17
mkdir -p /dev/oracle
mknod /dev/oracle/ocr1 c 13 0x000008
mknod /dev/oracle/ocr2 c 13 0x000006
mknod /dev/oracle/ocr3 c 13 0x000007
mknod /dev/oracle/data1 c 13 0x000005
--修改用户,用户组及权限
chown -R grid:asmadmin /dev/oracle
chmod 660 /dev/oracle/*
-- 清空磁盘头数据
dd if=/dev/zero of=/dev/oracle/ocr1 bs=8192 count=1280
dd if=/dev/zero of=/dev/oracle/ocr2 bs=8192 count=1280
dd if=/dev/zero of=/dev/oracle/ocr2 bs=8192 count=1280
dd if=/dev/zero of=/dev/oracle/data1 bs=8192 count=1280
1.15配置ntp服务
-- 查看当前时区
echo $TZ
or
cat /etc/TIMEZONE
or
cat /etc/default/tz
-- 设置时区
# set_parms timezone
____________________________________________________________________________
The following procedure enables you to set the time zone.
Select your location from the following list:
1) North America or Hawaii
2) Central America
3) South America
4) Europe
5) Africa
6) Asia
7) Australia, New Zealand
8) Russia / CIS
____________________________________________________________________________
Enter the number for your location (1-7) then press [Enter] #
-- 添加ntp服务器,根据实际环境添加或修改NTP服务端IP地址
vi /etc/ntp.conf
server 127.127.0.1
-- 更改两个参数的值
vi /etc/rc.config.d/netdaemons
export NTPDATE_SERVER=127.127.0.1
export XNTPD=1 (1表示启动xntpd;0表示不启动xntpd)
export XNTPD_ARGS=”-x”
-- 配置ntpd的参数,我们主要强调的是要配置成"微调的模式" 。HP默认是微调模式。
-- 启动ntp服务
/sbin/init.d/xntpd start
-- 检查是否生效
ntpq -p
ps -ef |grep ntp|grep -v grep
-- 安装完成后需要使用如下命令确认
$ crsctl check ctss
1.16 配置SSH互信
1)grid用户和oracle用户分别配置
# su - grid
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
节点一:
-- 将生成的rsa和dsa密钥复制到authorized_keys
$ cat ~/.ssh/*.pub >> ~/.ssh/authorized_keys
-- 将节点2的生成的密钥追加到节点1的authorized_keys中,若有多个节点,依次执行
$ ssh node2 "cat ~/.ssh/*.pub" >> ~/.ssh/authorized_keys
-- 将最终的authorized_keys文件传输到其它所有节点
$ scp ~/.ssh/authorized_keys node2:~/.ssh/authorized_keys
-- 修改文件权限
$ chmod 600 ~/.ssh/authorized_keys
-- oracle用户参照上面步骤配置
-- 互信验证(oracle,grid)所有节点操作
su - grid
export SSH='ssh -o ConnectTimeout=3 -o ConnectionAttempts=5 -o PasswordAuthentication=no -o StrictHostKeyChecking=no'
$ ${SSH} rac1 date
$ ${SSH} rac1-priv date
$ ${SSH} rac2 date
$ ${SSH} rac2-priv date
2) oralce11g可以在安装时进行自动配置ssh,第一次设置,点击“Setup”,设置成功后点击“Test”,成功后需两边节点测试验证互信。
2. 图形界面安装
2.1 预检查
su - grid
./runcluvfy.sh stage -pre crsinst -n vpar4,vpar5 -fixup -verbose
2.2 图形界面安装GI软件
以grid用户登录,进入到安装文件解压目录
./runInstaller
……
-- root用户,分别在两个节点执行以下脚本
/oracle/app/oraInventory/orainstRoot.sh
/oracle/app/11.2.0/grid/root.sh
2.3 检查集群状态
注意:gsd为OFFLINE状态正常,配置ntp服务则ora.ctssd为OBSERVER的状态
su - grid
crsctl stat res -t -init
crsctl stat res -t
2.4
创建ASM磁盘组
su – grid
asmca
2.5 图形界面安装RDBMS 软件
注意事项:两个DB的安装包需解压在同个目录下。
su - oracle
./runInstaller
……
--节点root用户运行脚本
/oracle/app/oracle/product/11.2.0/dbhome_1/root.sh
2.6 dbca建库
su – oracle
dbca