Oracle rac asm 扩容

Oracle rac asm 扩容

转载自 这货不是DBA:https://blog.csdn.net/qq_28766111/article/details/89242157

Oracle rac asm 扩容-使用udev方式绑定multipath多路径设备

 

环境介绍:
故事发生在Oracle 11.2.0.4 RAC/ Linux 6.8 的数据库集群服务器上。
DATA/ ARCH/ 磁盘组空间不足,进行扩容。

首先,如果要在线使用udev方式扩容asm,建议先备份两个节点所有在用网卡信息,避免bug误删除网卡信息。备份至/tmp目录下。(包括the ifcfg-eth0 (public), ifcfg-eth1 (private) and ifcfg-eth2 (backup)),注意11.2.0.3及以后在linux上的bug,见如下官网解释:
Network interface going down when dynamically adding disks to storage using udev in RHEL 6 (文档 ID 1569028.1)
Applies to: 
Oracle Database - Enterprise Edition - Version 11.2.0.3 and later
Oracle Net Services - Version 11.2.0.4 to 11.2.0.4 [Release 11.2]
Linux x86-64
Symptoms
Dynamically added new disks to storage using udev rules(as per the docs note 414897.1, 1528148.1, 371814.1 etc).

During start_udev, udev has deleted the public network interface and this caused the listener to crash, and clusterware moved all services, scan listeners and the VIP on node 1 to node 2.

Changes
Trying to add disk to storage using udev
Cause
udev has deleted the public network interface.

Solution
As per RHEL you are advised to set HOTPLUG="no" for the network configuration scripts.

Add HOTPLUG="no" to the ifcfg-eth0 (public), ifcfg-eth1 (private) and ifcfg-eth2 (backup) network config files in /etc/sysconfig/network-scripts directory.

 

建议先做数据库备份

1.停止数据库及集群服务

su - grid
srvctl stop listener
srvctl stop  database -d jyrac

rac1/rac2
su - root
/u01/11.2.0/grid/bin/crsctl stop crs

####################### rac1/rac2 begin ##############################

2.网卡信息修改

注意:如果是使用网卡绑定,比如绑定后的网卡为bond0,则要在bond0的配置文件里添加hotplug配置信息,在eth0里添加不起作用。(实战中所有在用网卡均添加)
备份所有网卡信息至/tmp 目录

rac1/rac2
cp ifcfg-eth0 /tmp/ifcfg-eth0bak
cp ifcfg-bond0 /tmp/ifcfg-bond0bak
cp ifcfg-eth1 /tmp/ifcfg-eth1bak
...

实际:
[root@jyrac1 network-scripts]# cp * /tmp/wkbak
[root@jyrac1 network-scripts]# 
[root@jyrac2 network-scripts]# cp * /tmp/wkbak/
[root@jyrac2 network-scripts]# 

修改网卡信息

HOTPLUG="no"

重启网络服务

service network restart
[root@jyrac1 network-scripts]# service network restart
Shutting down interface bond0:                             [  OK  ]
Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface bond0:  Determining if ip address 192.168.20.11 is already in use for device bond0...
                                                           [  OK  ]
Bringing up interface eth0:  Determining if ip address 187.187.2.61 is already in use for device eth0...
                                                           [  OK  ]
[root@jyrac1 network-scripts]#

[root@jyrac2 network-scripts]# service network restart
Shutting down interface bond0:                             [  OK  ]
Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface bond0:  Determining if ip address 192.168.20.12 is already in use for device bond0...
                                                           [  OK  ]
Bringing up interface eth0:  Determining if ip address 187.187.2.62 is already in use for device eth0...
                                                           [  OK  ]
[root@jyrac2 network-scripts]#

3.修改udev权限文件

备份udev文件至/tmp 目录

[root@jyrac1 network-scripts]# cp /etc/udev/rules.d/12-dm-permissions.rules /tmp/wkbak/
vi /etc/udev/rules.d/12-dm-permissions.rules
增加:

ENV{DM_NAME}=="arch4", OWNER:="grid", GROUP:="asmadmin", MODE:="660"
ENV{DM_NAME}=="data3", OWNER:="grid", GROUP:="asmadmin", MODE:="660"

4.加载配置文件

[root@rac-db1 /]# /sbin/udevadm control reload-rules

5.添加udev磁盘

(生产库千万别重启udev,否则vip会漂移,且网卡信息可能会被删除,甚至宕库,重启监听可解决)

udevadm trigger --subsystem-match=block --action=add
[root@jyrac1 network-scripts]# udevadm control reload-rules
unrecognized command
udevadm[27284]: unrecognized command
[root@jyrac1 network-scripts]# 
[root@jyrac1 network-scripts]# 
[root@jyrac1 network-scripts]# 
[root@jyrac1 network-scripts]# /sbin/udevadm control reload-rules
unrecognized command
udevadm[27682]: unrecognized command
[root@jyrac1 network-scripts]# find / -name udevadm
/sbin/udevadm
^C
[root@jyrac1 network-scripts]# 
[root@jyrac1 network-scripts]# 
[root@jyrac1 network-scripts]# 
[root@jyrac1 network-scripts]# cd
[root@jyrac1 ~]# /sbin/udevadm control reload-rules
unrecognized command
udevadm[27987]: unrecognized command
[root@jyrac1 ~]# ls -la /sbin/udevadm 
-rwxr-xr-x. 1 root root 137256 Oct 17  2013 /sbin/udevadm
[root@jyrac1 ~]# /sbin/udevadm control reload-rules
unrecognized command
udevadm[28151]: unrecognized command
[root@jyrac1 ~]# /sbin/start_udev
Starting udev:                                             [  OK  ]
[root@jyrac1 ~]# 

6.检查磁盘属性

ll -l /dev/dm*

如果有问题,可以考虑重启multipath 或者udev试试。

/sbin/start_udev
or
service multipathd restart

再次检查属性,确认没问题之后考虑启动集群服务。

[root@jyrac1 ~]# ll -l /dev/dm*
brw-rw----. 1 root disk     253,  0 Apr 11 18:23 /dev/dm-0
brw-rw----. 1 root disk     253,  1 Apr 11 18:23 /dev/dm-1
brw-rw----. 1 grid asmadmin 253, 10 Apr 11 18:23 /dev/dm-10
brw-rw----. 1 root disk     253, 11 Apr 11 18:23 /dev/dm-11
brw-rw----. 1 root disk     253, 12 Apr 11 18:23 /dev/dm-12
brw-rw----. 1 grid asmadmin 253, 13 Apr 11 18:23 /dev/dm-13
brw-rw----. 1 grid asmadmin 253, 14 Apr 11 18:23 /dev/dm-14
brw-rw----. 1 root disk     253,  2 Apr 11 18:23 /dev/dm-2
brw-rw----. 1 grid asmadmin 253,  3 Apr 11 18:23 /dev/dm-3
brw-rw----. 1 grid asmadmin 253,  4 Apr 11 18:23 /dev/dm-4
brw-rw----. 1 grid asmadmin 253,  5 Apr 11 18:23 /dev/dm-5
brw-rw----. 1 grid asmadmin 253,  6 Apr 11 18:23 /dev/dm-6
brw-rw----. 1 grid asmadmin 253,  7 Apr 11 18:23 /dev/dm-7
brw-rw----. 1 grid asmadmin 253,  8 Apr 11 18:23 /dev/dm-8
brw-rw----. 1 grid asmadmin 253,  9 Apr 11 18:23 /dev/dm-9
[root@jyrac1 ~]# 

[root@jyrac2 ~]# ll -l /dev/dm*
brw-rw---- 1 root disk     253,  0 Apr 11 18:25 /dev/dm-0
brw-rw---- 1 root disk     253,  1 Apr 11 18:25 /dev/dm-1
brw-rw---- 1 grid asmadmin 253, 10 Apr 11 18:25 /dev/dm-10
brw-rw---- 1 root disk     253, 11 Apr 11 18:25 /dev/dm-11
brw-rw---- 1 root disk     253, 12 Apr 11 18:25 /dev/dm-12
brw-rw---- 1 grid asmadmin 253, 13 Apr 11 18:25 /dev/dm-13
brw-rw---- 1 grid asmadmin 253, 14 Apr 11 18:25 /dev/dm-14
brw-rw---- 1 root disk     253,  2 Apr 11 18:25 /dev/dm-2
brw-rw---- 1 grid asmadmin 253,  3 Apr 11 18:25 /dev/dm-3
brw-rw---- 1 grid asmadmin 253,  4 Apr 11 18:25 /dev/dm-4
brw-rw---- 1 grid asmadmin 253,  5 Apr 11 18:25 /dev/dm-5
brw-rw---- 1 grid asmadmin 253,  6 Apr 11 18:25 /dev/dm-6
brw-rw---- 1 grid asmadmin 253,  7 Apr 11 18:25 /dev/dm-7
brw-rw---- 1 grid asmadmin 253,  8 Apr 11 18:25 /dev/dm-8
brw-rw---- 1 grid asmadmin 253,  9 Apr 11 18:25 /dev/dm-9
[root@jyrac2 ~]# 

7.启动crs集群服务

rac1/rac2
su - root
/u01/11.2.0/grid/bin/crsctl start crs

######################## rac1/rac2 end############################

8.检查crs服务

su - grid
crs_stat -t
crsctl stat res -t

9.启动数据库

srvctl start  database -d jyrac

10.检查数据库状态

crsctl stat res -t

11.检查asm是否识别到未添加的磁盘

set linesize 300
col name for a20
select group_number,name,state,type,total_mb,free_mb,usable_file_mb,allocation_unit_size/1024/1024 unit_mb from v$asm_diskgroup order by 1;

GROUP_NUMBER NAME                 STATE                  TYPE           TOTAL_MB    FREE_MB USABLE_FILE_MB    UNIT_MB
------------ -------------------- ---------------------- ------------ ---------- ---------- -------------- ----------
           1 ARCH                 MOUNTED                EXTERN           614400     120548         120548          1
           2 DATA                 MOUNTED                EXTERN           409600      83773          83773          1
           3 OCR                  MOUNTED                NORMAL            15360      14434           4657          1
set linesize 300
col path for a50
select group_number,name,path,mount_status,state,redundancy,total_mb,free_mb,failgroup,create_date from v$asm_disk order by 1,2;

GROUP_NUMBER NAME                 PATH                                               MOUNT_STATUS   STATE            REDUNDANCY       TOTAL_MB    FREE_MB FAILGROUP                                                CREATE_DATE
------------ -------------------- -------------------------------------------------- -------------- ---------------- -------------- ---------- ---------- ------------------------------------------------------------ -----------------------
           0                      /dev/mapper/arch4                                  CLOSED         NORMAL           UNKNOWN                 0          0
           0                      /dev/mapper/data3                                  CLOSED         NORMAL           UNKNOWN                 0          0
           1 ARCH_0000            /dev/mapper/arch1                                  CACHED         NORMAL           UNKNOWN            204800      40180 ARCH_0000                                                21-MAY-2015 11:00:32
           1 ARCH_0001            /dev/mapper/arch2                                  CACHED         NORMAL           UNKNOWN            204800      40183 ARCH_0001                                                21-MAY-2015 11:00:32
           1 ARCH_0002            /dev/mapper/arch3                                  CACHED         NORMAL           UNKNOWN            204800      40185 ARCH_0002                                                21-MAY-2015 11:00:32
           2 DATA_0000            /dev/mapper/data1                                  CACHED         NORMAL           UNKNOWN            204800      41879 DATA_0000                                                21-MAY-2015 10:59:33
           2 DATA_0001            /dev/mapper/data2                                  CACHED         NORMAL           UNKNOWN            204800      41894 DATA_0001                                                21-MAY-2015 10:59:33
           3 OCR_0000             /dev/mapper/crs1                                   CACHED         NORMAL           UNKNOWN              5120       4812 OCR_0000                                                 21-MAY-2015 10:16:24
           3 OCR_0001             /dev/mapper/crs2                                   CACHED         NORMAL           UNKNOWN              5120       4810 OCR_0001                                                 21-MAY-2015 10:16:24
           3 OCR_0002             /dev/mapper/crs3                                   CACHED         NORMAL           UNKNOWN              5120       4812 OCR_0002                                                 21-MAY-2015 10:16:24

10 rows selected.

12.ASM扩容


alter diskgroup DATA add disk '/dev/mapper/data3' rebalance power 8;

alter diskgroup ARCH add disk '/dev/mapper/arch4' rebalance power 8;

13.检查进度

SQL> select * from v$asm_operation;

GROUP_NUMBER OPERATION  STATE         POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE
------------ ---------- -------- ---------- ---------- ---------- ---------- ---------- ----------- ----------------------------------------------------------------------------------------
           1 REBAL      WAIT              8
           2 REBAL      RUN               8          8      33914     108615       5606          13

SQL> 
set linesize 300
col path for a50
select group_number,name,path,mount_status,state,redundancy,total_mb,free_mb,failgroup,create_date from v$asm_disk order by 1,2;

转自

邮箱:286397146@qq.com
昵称:A这货不是DBA

上一篇:iscc wp


下一篇:达梦DCP认证是什么?值得考吗?