搭建共享存储

4.1 创建共享磁盘

该步骤可以用cmd命令也可以用图形界面，本文采用命令行进行添加。
在cmd 中进入 WMware Workstation安装目录，执行命令创建磁盘：

cd  E:\software\VMware Workstation

vmware-vdiskmanager.exe -c -s 6g -a lsilogic -t 2 "E:\sharedisk\ocr_vote.vmdk"

vmware-vdiskmanager.exe -c -s 10g -a lsilogic -t 2 "E:\sharedisk\data.vmdk"

vmware-vdiskmanager.exe -c -s 10g -a lsilogic -t 2 "E:\sharedisk\fra.vmdk"

----------------------------------------------------------------------------------------------------------

E:\software\VMware Workstation>vmware-vdiskmanager.exe -c -s 10240Mb -a lsilogic -t 2 E:\sharedisk\ocr_vote.vmdk
Creating disk 'E:\sharedisk\ocr_vote.vmdk'
  Create: 100% done.
Virtual disk creation successful.

E:\software\VMware Workstation>vmware-vdiskmanager.exe -c -s 10G -a lsilogic -t 2 E:\sharedisk\fra.vmdk
Creating disk 'E:\sharedisk\fra.vmdk'
  Create: 100% done.
Virtual disk creation successful.

4.2 修改虚拟机配置文件

关闭两台虚拟机，用文本编辑器打开 虚拟机名字.vmx，即打开配置文件，2个节点都需要修改。
添加以下内容，其中，scsix:y 表示第x个总线上的第y个设备:

disk.enableuuid="true"
disk.locking = "false"
scsi1.shared = "true"
disklib.datacachemaxsize = "0"
disklib.datacachemaxreadaheadsize = "0"
disklib.datacacheminreadaheadsize = "0"
disklib.datacachepagesize= "4096"
disklib.maxunsyncedwrites = "0"
scsi1.present = "true"
scsi1.virtualdev = "lsilogic"
scsil.sharedbus = "virtual"
scsi1:0.present = "true"
scsi1:0.mode = "independent-persistent"
scsi1:0.filename = "e:\sharedisk\ocr_vote.vmdk"
scsi1:0.devicetype = "disk"
scsi1:0.redo = ""
scsi1:1.present = "true"
scsi1:1.mode = "independent-persistent"
scsi1:1.filename = "e:\sharedisk\data.vmdk"
scsi1:1.devicetype = "disk"
scsi1:1.redo = ""
scsi1:2.present = "true"
scsi1:2.mode = "independent-persistent"
scsi1:2.filename = "e:\sharedisk\fra.vmdk"
scsi1:2.devicetype = "disk"
scsi1:2.redo = ""

说明：如果报有的参数不存在的错误，那么请将虚拟机的兼容性设置到Workstation 9.0。

4.3 格式化磁盘

关闭 VMware Workstation 软件重新打开，此时看到共享磁盘正确加载则配置正确，这里尤其注意第二个节点，若不一样请检查配置。

然后开启2台虚拟机：

#查看一下3块磁盘sdb\sdc\sdc是否已经存在
rac1:~ # fdisk -l | grep /dev/sd
Disk /dev/sdd: 10 GiB, 10737418240 bytes, 20971520 sectors
Disk /dev/sdc: 10 GiB, 10737418240 bytes, 20971520 sectors
Disk /dev/sda: 500 GiB, 536870912000 bytes, 1048576000 sectors
/dev/sda1           2048    4208639   4206592    2G 82 Linux swap / Solaris
/dev/sda2  *     4208640   88100863  83892224   40G 83 Linux
/dev/sda3       88100864 1048575999 960475136  458G 83 Linux
Disk /dev/sdb: 10 GiB, 10737418240 bytes, 20971520 sectors

格式化磁盘和给磁盘分区

#3块磁盘都需要格式化，因为是共享盘只需在一台主机操作即可
rac1:~ # mkfs.ext4 /dev/sdd
rac1:~ # fdisk /dev/sdd

Welcome to fdisk (util-linux 2.33.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

The old ext4 signature will be removed by a write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x46caf446.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 

Using default response p.
Partition number (1-4, default 1): 
First sector (2048-20971519, default 2048): 
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-20971519, default 20971519): 

Created a new partition 1 of type 'Linux' and of size 10 GiB.

Command (m for help): 


Command (m for help): w

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

4.4 设置共享磁盘（UDEV）

4.4.1 配置udev绑定的scsi_id

● 首先切换到root用户下
● 2个节点上获取的uuid应该是一样的，不一样的话说明之前的配置有问题
1、不同的操作系统，scsi_id命令的位置不同。

rac1:~ # cat /etc/issue

Welcome to SUSE Linux Enterprise Server 12 SP5  (x86_64) - Kernel \r (\l)

2、编辑 /etc/scsi_id.config 文件，如果该文件不存在，则创建该文件并添加如下行：

linux-yph6:~ # echo "options=--whitelisted --replace-whitespace" > /etc/scsi_id.config
linux-yph6:~ # more /etc/scsi_id.config
options=--whitelisted --replace-whitespace
linux-yph6:~ # 

3、获取uuid

rac1:~ # /usr/lib/udev/scsi_id -g -u -d /dev/sdb
36000c29c90671cf41e2632f3b234ee4b
rac1:~ # /usr/lib/udev/scsi_id -g -u -d /dev/sdc
36000c29dca498808b4c6e3ea65946f53
rac1:~ # /usr/lib/udev/scsi_id -g -u -d /dev/sdd
36000c29ac0aea3c6d620c99442635f27

4.4.2 创建并配置udev rules文件

#没有/etc/udev/rules.d/99-oracle-asmdevices.rules文件的话就新建一个
rac1:~ # cd /etc/udev/rules.d/
rac1:~ # for i in b c d 
do
echo "KERNEL==\"sd*\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d /dev/\$parent\", RESULT==\"`/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", SYMLINK+=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"" >> 99-oracle-asmdevices.rules
done

4.4.3 用udevadm进行测试

注意udevadm命令不接受/dev/sdc这样的挂载设备名，必须是使用/sys/block/sdc这样的原始设备名。
udevadm test /sys/block/sdc在显示中，有类似如下输出，表示测试正确，/dev/sdc设备在udev启动以后将会绑定为/dev/asm-disk1：

rac1:/etc/udev/rules.d # udevadm test /sys/block/sdc

udevadm_test: UDEV_LOG=6
udevadm_test: DEVPATH=/devices/pci0000:00/0000:00:0d.0/host4/target4:0:0/4:0:0:0/block/sdc
udevadm_test: MAJOR=8 udevadm_test: MINOR=32
udevadm_test: DEVNAME=/dev/asm-disk1
udevadm_test: DEVTYPE=disk
udevadm_test: ACTION=add
udevadm_test: SUBSYSTEM=block

4.4.4 使配置生效

rac1:~ # /sbin/udevadm trigger --type=devices --action=change
rac1:~ # udevadm control --reload-rules
rac1:~ # ll /dev/as*
lrwxrwxrwx 1 root root 4 Aug 12 22:08 /dev/asm-diskb -> sdb1
lrwxrwxrwx 1 root root 4 Aug 12 22:08 /dev/asm-diskc -> sdc1
lrwxrwxrwx 1 root root 4 Aug 12 22:08 /dev/asm-diskd -> sdd1