Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

一,简介

 

 Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

二,配置虚拟机
1,创建虚拟机
(1)添加三块儿网卡:
 
主节点 二节点
eth0:    公网  192.168.1.20/24   NAT eth0:    公网  192.168.1.21/24   NAT
eth1:私网  192.168.2.20/24     host-only eth1:      私网  192.168.2.21/24   host-only
eth2:    DHCP    Bridged eth2:     DHCP    Bridged
主机名: testdb11a 主机名:  testdb11b
 
(2)添加五块儿磁盘:其中一块儿分配20G的空间供系统使用,其余四块儿为ASM磁盘,作为共享
 
创建4块儿独立磁盘,大小分配1GB,选择SCSI,独立永久
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
 
现在分配
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
共享磁盘放到单独的一个目录中,在复制二节点的时候方便寻找,取名字分别为:asm0.vmdk  asm1.vmdk  asm2.vmdk   asm3.vmdk(一定要有后缀.vmdk)
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
Finsh~~
下面的三块执行同样的操作!
接下来,四块共享磁盘分别分配到和系统硬盘完全不同的总线下面
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
总体如下:
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
三,安装操作系统
安装的时候在这里需要注意:
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
 
只选择sda,其余的盘对勾全部去掉,继续~
全部默认,软件只要有图形界面即可,主机名,IP地址,和需要的软件我们在安装完操作系统后按需安装
 
安装11g的时候如果不适用VNS或者X-manager等远程图形界面工具,需要安装vm-tools,否则在开启图形界面的时候会特别的闪
 
四,配置  ----主节点
(1)关闭iptables和selinux
 查看状态
[root@localhost ~]# /etc/init.d/iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    RH-Firewall-1-INPUT  all  --  0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    RH-Firewall-1-INPUT  all  --  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain RH-Firewall-1-INPUT (2 references)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           icmp type 255 
3    ACCEPT     esp  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     ah   --  0.0.0.0/0            0.0.0.0/0           
5    ACCEPT     udp  --  0.0.0.0/0            224.0.0.251         udp dpt:5353 
6    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:631 
7    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:631 
8    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
9    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
10   REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

关闭防火墙

[root@localhost ~]# service iptables stop
Flushing firewall rules: [  OK  ]
Setting chains to policy ACCEPT: filter [  OK  ]
Unloading iptables modules: [  OK  ]

[root@localhost ~]# chkconfig iptables off

查看
[root@localhost ~]# chkconfig --list | grep iptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off

[root@localhost ~]# service iptables status

Firewall is stopped.

禁用seLinux 

查看selinux状态
[root@localhost ~]# getenforce 
Enforcing

[root@localhost ~]# cat /etc/selinux/config 
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - SELinux is fully disabled.
SELINUX=enforcing
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.
SELINUXTYPE=targeted

修改SELINUX=enforcing为

SELINUX=disabled

注:这里需要重启服务器,这里我们等所有的配置都完成后再重启

(2)网络配置
a.配置IP地址
 修改配置文件
 第一块网卡
[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0 

# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth0
BOOTPROTO=static
IPADDR=192.168.1.20
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
HWADDR=00:0C:29:0B:FD:9E
ONBOOT=yes

第二块儿网卡

[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1

# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth1
BOOTPROTO=static
IPADDR=192.168.2.20
NETMASK=255.255.255.0
GATEWAY=192.168.2.1
HWADDR=00:0C:29:0B:FD:A8
ONBOOT=yes

 第三块网卡(保持默认)

[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth2

# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth2
BOOTPROTO=dhcp
HWADDR=00:0C:29:0B:FD:B2
ONBOOT=no

 b.修改主机名

[root@localhost ~]# vi /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=testdb11a

[root@localhost ~]# hostname testdb11a

c.配置hosts文件

说明:如果不适用DNS,在/etc/hosts文件中必须包含虚假的SCAN IP信息

# Public
192.168.1.20   testdb11a
192.168.1.21   testdb11b
# Private
192.168.2.20   testdb11a-priv
192.168.2.21   testdb11b-priv
# Virtual
192.168.1.22   testdb11a-vip
192.168.1.23   testdb11b-vip
# SCAN
192.168.1.24   testdb11-scan
192.168.1.25   testdb11-scan
192.168.1.26   testdb11-scan

 说明:SCAN 地址不应该在hosts文件中定义,而是在DNS中定义和public IP地址在同一个网段内的3个循环地址。在这里安装使用hosts文件避免了SCAN 地址,这种方式只实用于11.2.0.2以前的版本,11。2.0.2以后的版本,必须配置DNS,我们安装的是11.2.0.3.0版本,所以DNS是必须配置的,下面使用dnsmasq配置一个简单的DNS

 

在上面的/etc/hosts文件中加入3个和public网段相同的没有使用的iP地址,编辑下面的文件(nameserver后面可以是127.0.0.1,也可以是本地具体的IP地址)为了方便我们就选其中的一个节点做DNS服务器
[root@testdb11a ~]# more /etc/resolv.conf 
search localdomain
nameserver 192.168.1.20
启动dnsmasq服务
[root@testdb11a e22489]# /etc/init.d/dnsmasq start
Starting dnsmasq: [  OK  ]
查看
[root@testdb11a e22489]# nslookup testdb11-scan
Server:         192.168.1.20
Address:        192.168.1.20#53

Name:   testdb11-scan
Address: 192.168.1.26
Name:   testdb11-scan
Address: 192.168.1.24
Name:   testdb11-scan
Address: 192.168.1.25

 

 

 

 (3)关于时间同步

节点之间的时间同步对于oracle RAC来说是非常重要的,在oracle11g r2之前需要通过操作系统搭建NTP服务完成节点之间时间的同步。在oracle 11g 开始新增加了一个CTSS(Cluster Time Synchronization Services,集群时间同步服务),来完成节点间时间的同步,在安装Grid Infrastructure的时候,如果没有发现有NTP服务,则会自动安装ctss服务。这只对于RAC节点间时间的同步,对于其他系统来说是不起作用的。要使用oracle自己的CTSS,除了要停止、禁止自动重启NTP服务外,/etc/ntp.conf配置文件也不允许存在.(When the installer finds that the NTP protocol is not active, the Cluster Time Synchronization Service is automatically installed in active mode and synchronizes the time across the nodes. If NTP is found configured, then the Cluster Time Synchronization Service is started in observer mode, and no active time synchronization is performed by Oracle Clusterware within the cluster. )

[root@localhost ~]# service ntpd stop
Shutting down ntpd: [  OK  ]
[root@localhost ~]# chkconfig ntpd off
[root@localhost ~]# chkconfig --list | grep ntpd
ntpd            0:off   1:off   2:off   3:off   4:off   5:off   6:off
[root@localhost ~]# mv /etc/ntp.conf /etc/ntp.conf.
[root@localhost ~]# rm /var/run/ntpd.pid        ###这个文件是维护nptd进程pid的

如果想要使用NTP服务进行时间的同步,必须在"/etc/sysconfig/ntpd"中加入“-x” 标记

OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

然后重启ntpd服务

#service ntpd restart

 (4)内核参数及安全设置

修改/etc/sysctl.conf 添加下面内容

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 1054504960
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048586

 生效

sysctl -p

 修改/etc/security/limits.conf 

加入

oracle                                soft    nproc   2047
oracle               hard    nproc   16384
oracle               soft    nofile  1024
oracle               hard    nofile  65536

grid               soft    nproc   2047
grid               hard    nproc   16384
grid               soft    nofile  1024
grid               hard    nofile  65536

 

在/etc/pam.d/login 中加入
session    required     pam_limits.so

 (5)创建用户和组,以及所需要的目录

创建用户和组

 groupadd -g 501 oinstall
 groupadd -g 502 dba
 groupadd -g 503 oper
 groupadd -g 504 asmadmin
 groupadd -g 505 asmoper
 groupadd -g 506 asmdba
 useradd -g oinstall -G dba,asmdba,oper oracle
 useradd -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid 
 设置密码
passwd oracle
passwd grid

 创建所需目录并给于相应的权限

mkdir -p /u01/app/grid
mkdir -p /u01/app/11.2.0/grid
chown -R grid:oinstall /u01
mkdir -p /u01/app/oracle
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01

(6)安装所需要的rpm包

RHEL 5和Oracle Linux 5所需的包如下

binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-2.5-24 (32 bit)
glibc-common-2.5
glibc-devel-2.5
glibc-devel-2.5 (32 bit)
glibc-headers-2.5
ksh-20060214
libaio-0.3.106
libaio-0.3.106 (32 bit)
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
libgcc-4.1.2
libgcc-4.1.2 (32 bit)
libstdc++-4.1.2
libstdc++-4.1.2 (32 bit)
libstdc++-devel 4.1.2
make-3.81
sysstat-7.0.2
  • unixODBC-2.2.11 (32-bit) or later

  • unixODBC-devel-2.2.11 (64-bit) or later

  • unixODBC-2.2.11 (64-bit) or later

 

检查

rpm -q --qf %{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n  binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc-4.1.2 libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel

安装显示 not installed的包

使用yum安装,避免依赖关系的检查

配置yum源

[root@testdb11a ~]# mount /dev/cdrom /mnt
mount: block device /dev/cdrom is write-protected, mounting read-only
[root@testdb11a ~]# 
[root@testdb11a ~]# 
[root@testdb11a ~]# cat /etc/yum.repos.d/dvd.repo 
[dvd]
name=install dvd
baseurl=file:///mnt/Server
enabled=1
gpgcheck=0
[root@testdb11a ~]# yum clean all
Loaded plugins: katello, product-id, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Cleaning up Everything
[root@testdb11a ~]#yum list

安装

[root@testdb11a ~]# rpm -q --qf %{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n  binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc-4.1.2 libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel
binutils-2.17.50.0.6-20.el5 (x86_64)
compat-libstdc++-33-3.2.3-61 (x86_64)
compat-libstdc++-33-3.2.3-61 (i386)
elfutils-libelf-0.137-3.el5 (x86_64)
elfutils-libelf-0.137-3.el5 (i386)
elfutils-libelf-devel-0.137-3.el5 (i386)
elfutils-libelf-devel-0.137-3.el5 (x86_64)
package gcc is not installed
package gcc-c++ is not installed
glibc-2.5-81 (x86_64)
glibc-2.5-81 (i686)
glibc-common-2.5-81 (x86_64)
package glibc-devel is not installed
package glibc-headers is not installed
ksh-20100621-5.el5 (x86_64)
libaio-0.3.106-5 (x86_64)
libaio-0.3.106-5 (i386)
package libaio-devel is not installed
libgcc-4.1.2-52.el5 (x86_64)
libgcc-4.1.2-52.el5 (i386)
libstdc++-4.1.2-52.el5 (x86_64)
libstdc++-4.1.2-52.el5 (i386)
package libstdc++-devel is not installed
make-3.81-3.el5 (x86_64)
package sysstat is not installed
package unixODBC is not installed
package unixODBC-devel is not installed
[root@testdb11a ~]# yum install -y gcc gcc-c++ glibc-develglibc-headerslibaio-devel libstdc++-devel sysstat unixODBC    ---把显示not install的包都跟在后面
Loaded plugins: katello, product-id, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Install Process
No package glibc-develglibc-headerslibaio-devel available.
Resolving Dependencies
--> Running transaction check
---> Package gcc.x86_64 0:4.1.2-52.el5 set to be updated
--> Processing Dependency: glibc-devel >= 2.2.90-12 for package: gcc
---> Package gcc-c++.x86_64 0:4.1.2-52.el5 set to be updated
---> Package libstdc++-devel.i386 0:4.1.2-52.el5 set to be updated
dvd/filelists                                                                                                                 | 2.8 MB     00:00     
---> Package libstdc++-devel.x86_64 0:4.1.2-52.el5 set to be updated
---> Package sysstat.x86_64 0:7.0.2-11.el5 set to be updated
---> Package unixODBC.i386 0:2.2.11-10.el5 set to be updated
--> Processing Dependency: unixODBC-libs = 2.2.11-10.el5 for package: unixODBC
--> Processing Dependency: libodbcinst.so.1 for package: unixODBC
--> Processing Dependency: libodbc.so.1 for package: unixODBC
---> Package unixODBC.x86_64 0:2.2.11-10.el5 set to be updated
--> Running transaction check
---> Package glibc-devel.x86_64 0:2.5-81 set to be updated
--> Processing Dependency: glibc-headers = 2.5-81 for package: glibc-devel
--> Processing Dependency: glibc-headers for package: glibc-devel
---> Package unixODBC-libs.i386 0:2.2.11-10.el5 set to be updated
---> Package unixODBC-libs.x86_64 0:2.2.11-10.el5 set to be updated
--> Running transaction check
---> Package glibc-headers.x86_64 0:2.5-81 set to be updated
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers
--> Processing Dependency: kernel-headers for package: glibc-headers
--> Running transaction check
---> Package kernel-headers.x86_64 0:2.6.18-308.el5 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================================================================================
 Package                                  Arch                            Version                                 Repository                    Size
=====================================================================================================================================================
Installing:
 gcc                                      x86_64                          4.1.2-52.el5                            dvd                          5.3 M
 gcc-c++                                  x86_64                          4.1.2-52.el5                            dvd                          3.8 M
 libstdc++-devel                          i386                            4.1.2-52.el5                            dvd                          2.8 M
 libstdc++-devel                          x86_64                          4.1.2-52.el5                            dvd                          2.8 M
 sysstat                                  x86_64                          7.0.2-11.el5                            dvd                          187 k
 unixODBC                                 i386                            2.2.11-10.el5                           dvd                          290 k
 unixODBC                                 x86_64                          2.2.11-10.el5                           dvd                          291 k
Installing for dependencies:
 glibc-devel                              x86_64                          2.5-81                                  dvd                          2.4 M
 glibc-headers                            x86_64                          2.5-81                                  dvd                          596 k
 kernel-headers                           x86_64                          2.6.18-308.el5                          dvd                          1.4 M
 unixODBC-libs                            i386                            2.2.11-10.el5                           dvd                          551 k
 unixODBC-libs                            x86_64                          2.2.11-10.el5                           dvd                          554 k

Transaction Summary
=====================================================================================================================================================
Install      12 Package(s)
Upgrade       0 Package(s)

Total download size: 21 M
Downloading Packages:
-----------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                465 MB/s |  21 MB     00:00     
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : unixODBC-libs                                                                                                                1/12 
  Installing     : kernel-headers                                                                                                               2/12 
  Installing     : glibc-headers                                                                                                                3/12 
  Installing     : glibc-devel                                                                                                                  4/12 
  Installing     : libstdc++-devel                                                                                                              5/12 
  Installing     : unixODBC                                                                                                                     6/12 
  Installing     : sysstat                                                                                                                      7/12 
  Installing     : libstdc++-devel                                                                                                              8/12 
  Installing     : unixODBC-libs                                                                                                                9/12 
  Installing     : gcc                                                                                                                         10/12 
  Installing     : gcc-c++                                                                                                                     11/12 
  Installing     : unixODBC                                                                                                                    12/12 
Installed products updated.

Installed:
  gcc.x86_64 0:4.1.2-52.el5        gcc-c++.x86_64 0:4.1.2-52.el5    libstdc++-devel.i386 0:4.1.2-52.el5    libstdc++-devel.x86_64 0:4.1.2-52.el5   
  sysstat.x86_64 0:7.0.2-11.el5    unixODBC.i386 0:2.2.11-10.el5    unixODBC.x86_64 0:2.2.11-10.el5       

Dependency Installed:
  glibc-devel.x86_64 0:2.5-81           glibc-headers.x86_64 0:2.5-81  kernel-headers.x86_64 0:2.6.18-308.el5  unixODBC-libs.i386 0:2.2.11-10.el5 
  unixODBC-libs.x86_64 0:2.2.11-10.el5 

Complete!

最后检查

[root@testdb11a ~]# rpm -q --qf %{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n  binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc-4.1.2 libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel
binutils-2.17.50.0.6-20.el5 (x86_64)
compat-libstdc++-33-3.2.3-61 (x86_64)
compat-libstdc++-33-3.2.3-61 (i386)
elfutils-libelf-0.137-3.el5 (x86_64)
elfutils-libelf-0.137-3.el5 (i386)
elfutils-libelf-devel-0.137-3.el5 (i386)
elfutils-libelf-devel-0.137-3.el5 (x86_64)
gcc-4.1.2-52.el5 (x86_64)
gcc-c++-4.1.2-52.el5 (x86_64)
glibc-2.5-81 (x86_64)
glibc-2.5-81 (i686)
glibc-common-2.5-81 (x86_64)
glibc-devel-2.5-81 (x86_64)
glibc-headers-2.5-81 (x86_64)
ksh-20100621-5.el5 (x86_64)
libaio-0.3.106-5 (x86_64)
libaio-0.3.106-5 (i386)
libaio-devel 
libgcc-4.1.2-52.el5 (x86_64)
libgcc-4.1.2-52.el5 (i386)
libstdc++-4.1.2-52.el5 (x86_64)
libstdc++-4.1.2-52.el5 (i386)
libstdc++-devel-4.1.2-52.el5 (i386)
libstdc++-devel-4.1.2-52.el5 (x86_64)
make-3.81-3.el5 (x86_64)
sysstat-7.0.2-11.el5 (x86_64)
unixODBC-2.2.11-10.el5 (x86_64)
unixODBC-2.2.11-10.el5 (i386)
unixODBC-devel-2.2.11-10.el5 (x86_64)
unixODBC-devel-2.2.11-10.el5 (i386)

已经全部安装~~~

(7)设置grid和oracle的环境变量

grid用户:

[root@localhost ~]# su - grid
[grid@testdb11a ~]$ vi .bash_profile 

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

#export PATH
##################################################
# User specific environment and startup programs #
##################################################
export ORACLE_BASE=/u01/app/grid
####export ORACLE_HOME=$ORACLE_BASE/grid/product/11.2.0
export ORACLE_HOME=/u01/app/11.2.0/grid      ####这里一定不能使用上面的安装目录,否则在安装grid Infrastructure的时候包INS-32026
export ORACLE_PATH=$ORACLE_BASE/common/oracle/sql:.:$ORACLE_HOME/rdbms/admin
export ORACLE_SID=+ASM1
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin:$ORACLE_HOME/bin
export ORACLE_TERM=xterm
#export TNS_ADMIN=$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=$ORACLE_HOME/JRE
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
##################################################
# set NLS_LANG to resolve messy code in SQLPLUS  #
##################################################
export NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1
##################################################
# Shell setting.                                 #
##################################################
umask 022
set -o vi
export PS1="\${ORACLE_SID}@`hostname`  \${PWD}$ "

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

oracle用户:

[root@localhost ~]# su - oracle
dbrac1@testdb11a  /home/oracle$ cat .bash_profile 
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

#export PATH
##################################################  
# User specific environment  and startup programs#  
##################################################  
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0
export ORACLE_PATH=$ORACLE_BASE/common/oracle/sql:.:$ORACLE_HOME/rdbms/admin
export ORACLE_SID=dbrac1
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin:$ORACLE_HOME/bin
export ORACLE_TERM=xterm  
#export TNS_ADMIN=$ORACLE_HOME/network/admin  
export LD_LIBRARY_PATH=$ORACLE_HOME/lib  
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib  
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib  
export CLASSPATH=$ORACLE_HOME/JRE  
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib  
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib  
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib  
export THREADS_FLAG=native  
export TEMP=/tmp  
export TMPDIR=/tmp  
##################################################  
# set NLS_LANG to resolve messy code in SQLPLUS  
##################################################  
export NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1  
##################################################  
# Shell setting.  
##################################################  
umask 022  
set -o vi  
export PS1="\${ORACLE_SID}@`hostname`  \${PWD}$ "  

 (8)创建共享磁盘

[root@testdb11a dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde

分区

[root@testdb11a dev]# fdisk /dev/sdb

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-130, default 130): 
Using default value 130

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

后面的sdc,sdd,sde执行相同的操作

[root@testdb11a dev]# ls sd*
sda  sda1  sda2  sdb  sdb1  sdc  sdc1  sdd  sdd1  sde  sde1
 绑定裸设备
编辑文件/etc/sysconfig/rawdevice
[root@testdb11a ~]# vi /etc/sysconfig/rawdevices 

# raw device bindings
# format:  <rawdev> <major> <minor>
#          <rawdev> <blockdev>
# example: /dev/raw/raw1 /dev/sda1
#          /dev/raw/raw2 8 5
/dev/raw/raw1 /dev/sdb1
/dev/raw/raw2 /dev/sdc1
/dev/raw/raw3 /dev/sdd1
/dev/raw/raw4 /dev/sde1

 启动rawdevice

[root@testdb11a ~]# /etc/init.d/rawdevices start
Assigning devices: 
           /dev/raw/raw1  -->   /dev/sdb1
/dev/raw/raw1:  bound to major 8, minor 17
           /dev/raw/raw2  -->   /dev/sdc1
/dev/raw/raw2:  bound to major 8, minor 33
           /dev/raw/raw3  -->   /dev/sdd1
/dev/raw/raw3:  bound to major 8, minor 49
           /dev/raw/raw4  -->   /dev/sde1
/dev/raw/raw4:  bound to major 8, minor 65
done

 查看

[root@testdb11a ~]# raw -qa 
/dev/raw/raw1:  bound to major 8, minor 17
/dev/raw/raw2:  bound to major 8, minor 33
/dev/raw/raw3:  bound to major 8, minor 49
/dev/raw/raw4:  bound to major 8, minor 65

修改设备权限

[root@testdb11a raw]# ll
total 0
crw------- 1 root root 162, 1 Sep 15 23:07 raw1
crw------- 1 root root 162, 2 Sep 15 23:07 raw2
crw------- 1 root root 162, 3 Sep 15 23:07 raw3
crw------- 1 root root 162, 4 Sep 15 23:07 raw4
[root@testdb11a raw]# chown grid:asmdba /dev/raw/raw[1-4]
[root@testdb11a raw]# chmod 660 /dev/raw/raw[1-4]
[root@testdb11a raw]# ll
total 0
crw-rw---- 1 grid asmdba 162, 1 Sep 15 23:07 raw1
crw-rw---- 1 grid asmdba 162, 2 Sep 15 23:07 raw2
crw-rw---- 1 grid asmdba 162, 3 Sep 15 23:07 raw3
crw-rw---- 1 grid asmdba 162, 4 Sep 15 23:07 raw4

 设置重启自动修改设备权限

编辑/etc/rc.d/rc.local 

加入
chown grid:asmdba /dev/raw/raw[1-4]
chmod 660 /dev/raw/raw[1-4]

PS:需要安装vm-tools

五,配置副节点

(1)关闭系统

[root@testdb11a ~]# shutdown -h 0

Broadcast message from root (pts/2) (Mon Sep 15 23:59:39 2014):

The system is going down for system halt NOW!
(2)克隆主节点
把克隆的节点放到对应的目录下
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0 
Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0
 
 (3)在主节点二节点的虚拟机.vmx参数文件中加入
disk.locking="false"  
diskLib.dataCacheMaxSize = "0" 
diskLib.dataCacheMaxReadAheadSize = "0"  
diskLib.DataCacheMinReadAheadSize = "0"  
diskLib.dataCachePageSize = "4096" 
diskLib.maxUnsyncedWrites = "0"  
scsi1:0.deviceType = "disk" 
scsi1:1.deviceType = "disk" 
scsi1:2.deviceType = "disk" 
scsi1:3.deviceType = "disk"
disk.EnableUUID ="true"  
启动rac1和rac2两个虚拟机
(4)修改二节点中的与一节点中不同的配置
root用户登录系统,修改网络参数
  • hostname: testdb11b
  • IP Address eth0: 192.168.1.21 (public address)
  • Default Gateway eth0: 192.168.1.1 (public address)
  • IP Address eth1: 192.168.2.21 (private address)
  • Default Gateway eth1: none
 修改主机名
[root@testdb11a ~]# vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=testdb11b
 修改IP地址(ifcfg-eth0和ifcfg-eth1两个文件)
[root@testdb11a ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth0
BOOTPROTO=static
IPADDR=192.168.1.21
NETMASK=255.255.255.0
GATAWAY=192.168.1.1
ONBOOT=yes
HWADDR=00:0c:29:b6:22:54
[root@testdb11a ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth1
BOOTPROTO=static
IPADDR=192.168.2.21
NETMASK=255.255.255.0
ONBOOT=yes
HWADDR=00:0c:29:b6:22:5e

 修改环境变量

grid用户
.bash_profile 中的 export ORACLE_SID=+ASM2

  oracle用户

.bash_profile文件的export ORACLE_SID=dbrac2

在rac2上查看共享磁盘

[root@testdb11a raw]# raw -qa
/dev/raw/raw1:  bound to major 8, minor 17
/dev/raw/raw2:  bound to major 8, minor 33
/dev/raw/raw3:  bound to major 8, minor 49
/dev/raw/raw4:  bound to major 8, minor 65
[root@testdb11a raw]# cd /dev/raw
[root@testdb11a raw]# ll
total 0
crw-rw---- 1 grid asmdba 162, 1 Sep 16 00:11 raw1
crw-rw---- 1 grid asmdba 162, 2 Sep 16 00:11 raw2
crw-rw---- 1 grid asmdba 162, 3 Sep 16 00:11 raw3
crw-rw---- 1 grid asmdba 162, 4 Sep 16 00:11 raw4

重启二节点虚拟机

执行

ping -c 3 testdb11a
ping -c 3 testdb11b
ping -c 3 testdb11a-priv
ping -c 3 testdb11b-priv

保证两台虚拟机之间网络互通

在安装之前,软件解压包的目录下有一个节点间校验系统要求的工具,可以使用下面的命令进行校验预安装要求,但是使用这个工具必须配置ssh互信,所有这里就不做校验

 

 ./runcluvfy.sh stage -pre crsinst -n testdb11a,testdb11b -verbose
 六,安装grid Infrastructure
(1)通过ftp或者其他方式上传软件,并解压
创建一个放置oracle软件的目录
mkdir -p /oracle/media
chown -R oracle:oinstall /oracle/media
chmod -R 775 /oracle/media
[root@testdb11a media]# ll
total 3401628
-rw-r--r-- 1 oracle oinstall 1358454646 Nov 25  2011 p10404530_112030_Linux-x86-64_1of7.zip
-rw-r--r-- 1 oracle oinstall 1142195302 Nov 27  2011 p10404530_112030_Linux-x86-64_2of7.zip     ----上面两个为数据库软件
-rw-r--r-- 1 grid   oinstall  979195792 Sep 15 18:11 p10404530_112030_Linux-x86-64_3of7.zip     ----grid 软件
[root@testdb11a media]# 

解压

[root@testdb11a app]# cd /oracle/media/
[root@testdb11a media]# ll
total 8
drwxr-xr-x 8 oracle oinstall 4096 Sep 22  2011 database
drwxr-xr-x 8 grid   oinstall 4096 Sep 22  2011 grid
[root@testdb11a media]# 

在/oracle/media/grid/rpm这个目录下有个cvuqdisk-1.0.9-1.rpm包,在两个节点上安装

scp cvuqdisk-1.0.9-1.rpm testdb11b:/home/grid
[root@testdb11b grid]# rpm -ivh cvuqdisk-1.0.9-1.rpm 
Preparing...                ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]

 

(2)启动图形界面

执行./runInstaller 

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

跳过软件更新

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

选择“install and Configure Grid Infrastructure for a Cluster ”

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

选择“Typical Installation”

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

输入正确的SCAN名,点击Add把另一个节点加入

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

填入二节点的正确的Hostname和vip name

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

点击“SSH Connectivity”,输入grid的密码,点击“Setup”配置互信(备注:下面的这幅截图SCAN Name写错了,应该是testdb11-scan,和hosts文件中的相同

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

出现上面的提示,则互信配置成功,下面点击“Test”进行测试

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

点击“Identify network interfaces..”检查网络配置是否正确,确认无误后,点击“OK”

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

输入grid的安装目录,选择“Oracle Automatic Storage Management ”,输入ASM的密码,下一步

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

使用简单密码,继续

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

选择“External”,选择所有的磁盘,如果磁盘不显示,点击“Change Discovery Path”,输入“/dev/raw/raw*”

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

默认

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

下面的这几个警告可以忽略,勾选"Ignore All",继续(说明,下图中第三个警告不能跳过)

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

安装

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

提示需要执行两个脚本

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

执行结果如下

节点一,节点二

在这里二节点执行root.sh的时候报错

CRS-5005: IP Address: 192.168.1.24 is already in use in the network
. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/testdb11b/agent/crsd/orarootagent_root/orarootagent_root.log".

出现上面的错误的原因是:我没有配置dns服务,只在hosts文件中加入了相应的scan-ip,希望能骗过oracle,但是这种做法在11.2.0.2开始这种做法已经不可取,所以只能配置dns去解决。下面的安装软件过程中检测出来的错误,在11.2.0.2之前是不检查/etc/resolv.conf这个文件的(两个节点都需要配置)

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

重新执行脚本,

显示成功后,点击OK继续安装

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

查看日志

INFO: Checking Single Client Access Name (SCAN)...
INFO: ERROR: 
INFO: PRVF-5056 : Scan Listener "LISTENER_SCAN1" not running
INFO: Checking name resolution setup for "testdb11-scan"...
INFO: ERROR: 
INFO: PRVF-4657 : Name resolution setup check for "testdb11-scan" (IP address: 192.168.1.25) failed
INFO: ERROR: 
INFO: PRVF-4657 : Name resolution setup check for "testdb11-scan" (IP address: 192.168.1.26) failed
INFO: ERROR: 
INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "testdb11-scan"
INFO: Verification of SCAN VIP and Listener setup failed

 

可以放心跳过

七,安装数据库软件

进入oracle用户,database目录,执行

./runInstaller

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

点击下一步

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

点击“SSH Connectivity”配置oracle用户的节点间互信。然后点击“Setup”,并“Test”

选择语言,然后下一步

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

选择企业版

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

选择安装路径

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

选择管理组

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

先决条件检查,选择忽略

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

我分配的内存为1GB,内存绝对不能小于1GB推荐最小1.5g。卡在56%半天才过去

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

执行脚本

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

执行完成后,点击OK,database soft安装完成

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

八,使用DBCA进行建库

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

 九,遇到的问题

问题1:二节点执行root.sh脚本报错

问题2:只能看到一个节点

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

解决:oracle 11g RAC安装节点二执行结果错误CRS-5005: IP Address: 192.168.1.24 is already in use in the network

查看服务:

主节点:

+ASM1@testdb11a  /oracle/media/grid$ crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host        
----------------------------------------------------------------------
ora.DATA.dg    ora....up.type 0/5    0/     ONLINE    ONLINE    testdb11a   
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    testdb11a   
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    OFFLINE               
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    testdb11a   
ora.cvu        ora.cvu.type   0/5    0/0    ONLINE    ONLINE    testdb11a   
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE               
ora....network ora....rk.type 2/5    0/     ONLINE    ONLINE    testdb11a   
ora.oc4j       ora.oc4j.type  0/1    0/2    ONLINE    ONLINE    testdb11a   
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    testdb11a   
ora....ry.acfs ora....fs.type 0/5    0/     ONLINE    ONLINE    testdb11a   
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    OFFLINE               
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    testdb11a   
ora....1A.lsnr application    0/5    0/0    ONLINE    ONLINE    testdb11a   
ora....11a.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....11a.ons application    0/3    0/0    ONLINE    ONLINE    testdb11a   
ora....11a.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    testdb11a  

 

二节点

+ASM2@testdb11b  /home/grid$ crs_stat -t -v
Name           Type           R/RA   F/FT   Target    State     Host        
----------------------------------------------------------------------
ora.DATA.dg    ora....up.type 0/5    0/     ONLINE    ONLINE    testdb11b   
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    testdb11b   
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    testdb11b   
ora.cvu        ora.cvu.type   0/5    0/0    ONLINE    ONLINE    testdb11b   
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE               
ora....network ora....rk.type 2/5    0/     ONLINE    ONLINE    testdb11b   
ora.oc4j       ora.oc4j.type  0/1    0/2    ONLINE    ONLINE    testdb11b   
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    testdb11b   
ora....ry.acfs ora....fs.type 0/5    0/     ONLINE    ONLINE    testdb11b   
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    testdb11b   
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    testdb11b   
ora....11b.gsd application    0/5    0/0    OFFLINE   OFFLINE               
ora....11b.ons application    0/3    0/0    ONLINE    ONLINE    testdb11b   
ora....11b.vip ora....t1.type 0/0    0/0    ONLINE    ONLINE    testdb11b 

 

 

 

 

 

 

 

 

 

 

 

 

 
 
 
 

Oracle Database 11g Release 2(11.2.0.3.0) RAC On Redhat Linux 5.8 Using Vmware Workstation 9.0

上一篇:黄聪:NaviCat通过Http方式连接服务器的MySQL数据库(转)


下一篇:sqlalchemy