1、Oracle 11G R2 RAC安装手册一、 基础知识1. RAC是什么RAC，全称real application clusters，译为“实时应用集群”，是Oracle新版数据库中采用的一项新技术，是高可用性的一种，也是Oracle数据库支持网格计算环境的核心技术。2. RAC的优缺点l 优点Oracle RAC支持Oracle9i、10g、11g版本，可以支持24 x 7 有效的数据库应用系统，在低成本服务器上构建高可用性数据库系统，并且自由部署应用，无需修改代码。在Oracle RAC环境下，Oracle集成提供了集群软件和存储管理软件，为用户降低了应用成本。当应用规模需要扩充时，用
2、户可以按需扩展系统，以保证系统的性能。(1) 多节点负载均衡; (2) 提供高可用：故障容错和无缝切换功能，将硬件和软件错误造成的影响最小化; (3) 通过并行执行技术提高事务响应时间-通常用于数据分析系统; (4) 通过横向扩展提高每秒交易数和连接数-通常对于联机事务系统; (5) 节约硬件成本，可以用多个廉价PC服务器代替昂贵的小型机或大型机，同时节约相应维护成本; (6) 可扩展性好，可以方便添加删除节点，扩展硬件资源。l 缺点(1) 相对单机，管理更复杂，要求更高; (2) 可能会增加软件成本3. Oracle RAC原理在一个应用环境当中，所有的服务器使用和管理同一个数据库，目的是为
3、了分散每一台服务器的工作量，硬件上至少需要两台以上的服务器，而且还需要一个共享存储设备。同时所有服务器上的OS都应该是同一类OS,根据负载均衡的配置策略，当一个客户端发送请求到某一台服务的listener后，这台服务器根据我们的负载均衡策略，会把请求发送给本机的RAC组件处理也可能会发送给另外一台服务器的RAC组件处理，处理完请求后，RAC会通过集群软件来访问共享存储设备.逻辑结构上看：l 每一个参加集群的节点有一个独立的instance，这些instance访问同一个数据库。l 每一个节点的linstance都有自己的SGA。l 每一个节点的linstance都有自己的background
4、process。l 每一个节点的linstance都有自己的redo logs。l 每一个节点的instance都有自己的undo表空间。l 所有节点都共享一份datafiles和controlfiles。三类Resource：l VIP - 虚拟IP地址(Virtual IP) l OCR - Oracle Cluster Registry(集群注册文件),记录每个节点的相关信息l Voting Disk,仲裁机制用于仲裁多个节点向共享节点同时写的行为，这样 做是为了避免发生冲突。二、 RAC环境设计1. 服务器配置操作系统主机名内存网卡Redhat Enterprise Linux 5.4
5、rac12Geth0（Public）eth1（Prive）Redhat Enterprise Linux 5.4rac22Geth0（Public）eth1（Prive）2. 用户组规划GroupNameGroupID说明HOSToinstall501Oracle清单和软件所有者rac1，rac2asmadmin504Oracle自动存储管理组rac1，rac2asmdba506ASM数据库管理员组rac1，rac2asmopen507ASM操作员组rac1，rac2dba502数据库管理员rac1，rac23. 用户规划UserIDUserNameGroup说明HOST口令501gridoi
6、nstalldbaasmadminasmdbaasmoperOracle Cluster用户rac1rac2grid123502oracleoinstalldbaasmdbaOracle数据库管理员rac1rac2oracle1234. IP规划主机名公用IP私有IP虚拟IPrac1172.16.1.21110.10.10.211172.16.1.111rac2172.16.1.21210.10.10.212172.16.1.112ScanIP172.16.1.1005. 存储规划存储组件Block DeviceASMlib NameSizeOCR/dev/sdbOCR_VOL11G/dev/
7、sdcOCR_VOL21G/dev/sddOCR_VOL31G数据文件/dev/sdeDATA_VOL110G文件备份/dev/sdfBACK_VOL15G三、 Oracle RAC搭建1. 配置共享存储（ASM依赖）l 方法一：vmware workstation 9虚拟共享存储(1) 在VM安装目录VMware Workstation下执行命令：#vmware-vdiskmanager.exe -c -s 1GB -a lsilogic -t 2 D:racDiskdb01.vmdk#vmware-vdiskmanager.exe -c -s 1GB -a lsilogic -t 2 D:
8、racDiskdb02.vmdk#vmware-vdiskmanager.exe -c -s 1GB -a lsilogic -t 2 D:racDiskdb03.vmdk#vmware-vdiskmanager.exe -c -s 10GB -a lsilogic -t 2 D:racDiskdb04.vmdk#vmware-vdiskmanager.exe -c -s 5GB -a lsilogic -t 2 D:racDiskdb05.vmdk(2) 修改rac1和rac2下的.vmx文件，在最后一行添加如下内容：diskLib.dataCacheMaxSize=0 diskLib.da
9、taCacheMaxReadAheadSize=0diskLib.dataCacheMinReadAheadSize=0diskLib.dataCachePageSize=4096diskLib.maxUnsyncedWrites = 0disk.locking = FALSEscsi1.sharedBus = virtual scsi1.present = TRUEscsi1.virtualDev = lsilogicscsi1:0.mode = independent-persistentscsi1:0.deviceType = diskscsi1:0.present = TRUEscsi
10、1:0.fileName = D:racDiskdb01.vmdkscsi1:1.mode = independent-persistentscsi1:1.deviceType = diskscsi1:1.present = TRUEscsi1:1.fileName = D:racDiskdb02.vmdk scsi1:2.mode = independent-persistentscsi1:2.deviceType = diskscsi1:2.present = TRUEscsi1:2.fileName = D:racDiskdb03.vmdk scsi1:3.mode = independ
11、ent-persistentscsi1:3.deviceType = diskscsi1:3.present = TRUEscsi1:3.fileName = D:racDiskdb04.vmdkscsi1:4.mode = independent-persistentscsi1:4.deviceType = diskscsi1:4.present = TRUEscsi1:4.fileName = D:racDiskdb05.vmdk(3) 重新打开虚拟机，并启动。(4) 磁盘分区，在rac1执行#fdisk /dev/sdbDevice contains neither a valid DO
12、S partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content wont be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)Command (m
13、 for help): pDisk /dev/sdb: 1048 MB, 1048576000 bytes 64 heads, 32 sectors/track, 1000 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (
14、1-1000, default 1):Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-1000, default 1000): Using default value 1000 Command (m for help): wThe partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.#fdisk /dev/sdc 重复上面操作#fdisk /dev/sdd 重复上面操作#fdi
15、sk /dev/sde 重复上面操作#fdisk /dev/sdf 重复上面操作l 方法二：Linux mount挂载（待解决）2. 安装依赖包#yum install -y expect* iscsi* binutils* compat-libstdc+* expat* elfutils-libelf* gcc* glibc* ksh* libgcc*#yum install -y pdksh* libgomp* compat-db* libXp* libaio* cpp* setarch* sysstat* libstdc+* make-3*#yum install -y unixODBC
16、#yum install -y unixODBC-devel3. 查看依赖包安装状况#rpm -q -qf %NAME-%VERSION-%RELEASE(%ARCH)n binutils elfutils-libelf elfutils-libelf expat gcc gcc-c+ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc libstdc+ libstdc+-devel make pdksh sysstat unixODBC unixODBC-devel执行结果binutils-2
17、.17.50.0.6-26.el5(x86_64)elfutils-libelf-0.137-3.el5(x86_64)elfutils-libelf-0.137-3.el5(i386)elfutils-libelf-0.137-3.el5(x86_64)elfutils-libelf-0.137-3.el5(i386)expat-1.95.8-11.el5_8(x86_64)expat-1.95.8-11.el5_8(i386)gcc-4.1.2-54.el5(x86_64)gcc-c+-4.1.2-54.el5(x86_64)glibc-2.5-118.el5_10.2(i686)glib
18、c-2.5-118.el5_10.2(x86_64)glibc-common-2.5-118.el5_10.2(x86_64)glibc-devel-2.5-118.el5_10.2(x86_64)glibc-devel-2.5-118.el5_10.2(i386)glibc-headers-2.5-118.el5_10.2(x86_64)ksh-20100621-18.el5(x86_64)libaio-0.3.106-5(x86_64)libaio-0.3.106-5(i386)libaio-devel-0.3.106-5(i386)libaio-devel-0.3.106-5(x86_6
19、4)libgcc-4.1.2-54.el5(x86_64)libgcc-4.1.2-54.el5(i386)libstdc+-4.1.2-54.el5(x86_64)libstdc+-4.1.2-54.el5(i386)libstdc+-devel-4.1.2-54.el5(x86_64)libstdc+-devel-4.1.2-54.el5(i386)make-3.81-3.el5(x86_64)pdksh-5.2.14-37.el5_8.1(x86_64)sysstat-7.0.2-12.el5(x86_64)unixODBC-2.2.11-10.el5(i386)unixODBC-2.2
20、.11-10.el5(x86_64)unixODBC-devel-2.2.11-10.el5(x86_64)unixODBC-devel-2.2.11-10.el5(i386)4. 网络配置(1) 修改rac1和rac2的network#vi /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=rac1 #此处为主机名GATEWAY=172.16.1.254#vi /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=rac2GATEWAY=172.
21、16.1.254(2) 修改rac1和rac2的hosts#vi /etc/hosts#eth0-Public IP#172.16.1.211 rac1172.16.1.212 rac2#eth1PRIVATE IP#10.10.10.211 rac1-priv10.10.10.212 rac2-priv#VIP#172.16.1.111 rac1-vip172.16.1.112 rac2-vip#SCAN#172.16.1.100 rac-cluster-scan(3) 重启rac1和rac2网络服务#service network restart(4) 配置虚拟IP#ifconfig et
22、h0:0 172.16.1.111 netmask 255.255.0.0 up#ifconfig eth0:0 172.16.1.112 netmask 255.255.0.0 up*注：虚拟IP在重启网络服务会被卸载，需要重新设置。5. 配置NTPD服务NTPD服务为Grid的检查项，提供时间同步服务。(1) 修改NTPD参数文件（rac1，rac2）#vi /etc/sysconfig/ntpdSYNC_HWCLOCK=yesOPTIONS=-x -u ntp:ntp -p /var/run/ntpd.pid(2) 修改rac1的ntpd.conf文件，删除原有内容录入以下内容#cp /
23、etc/ntp.conf /etc/ntp.conf.bak#vi /etc/ntp.confrestrict 0.0.0.0 mask 0.0.0.0 nomodifyserver127.127.1.0fudge127.127.1.0 stratum 10restrict 127.127.1.0driftfile /var/lib/ntp/driftbroadcastdelay 0.008authenticate nokeys /etc/ntp/keys(3) 清空rac1的ntpservers原有内容#cp /etc/ntp/ntpservers /etc/ntp/ntpservers.b
24、ak#vi /etc/ntp/ntpservers(4) 编辑rac1的step-tickers，添加一行#cp /etc/ntp/step-tickers /etc/ntp/step-tickers.bak#vi /etc/ntp/step-tickers127.127.1.0(5) 修改rac2的ntpd.conf文件，删除原有内容录入以下内容#cp /etc/ntp.conf /etc/ntp.conf.bak#vi /etc/ntp.confrestrict 0.0.0.0 mask 0.0.0.0 nomodifyserver127.127.1.0server172.16.1.211
25、fudge127.127.1.0 stratum 10driftfile /var/lib/ntp/driftbroadcastdelay 0.008authenticate nokeys /etc/ntp/keys(6) 清空rac2的ntpservers原有内容#cp /etc/ntp/ntpservers /etc/ntp/ntpservers.bak#vi /etc/ntp/ntpservers(7) 编辑rac2的step-tickers，添加一行#cp /etc/ntp/step-tickers /etc/ntp/step-tickers.bak#vi /etc/ntp/step-
26、tickers172.16.1.211#rac1的ip地址(8) rac1节点重启NTPD服务#service ntpd restart(9) rac2节点停止NTPD服务，rac1重启NTPD服务后5、6分钟再执行#service ntpd stop#ntpdate rac1#service ntpd start(10) NTPD启用自动加载（rac1，rac2）#chkconfig ntpd on6. 关闭服务关闭rac1和rac2无用的sendmail和cups服务，加快开机速度。#chkconfig sendmail off#chkconfig cups off7. 用户、用户组(1)
27、 创建用户和用户组#groupadd -g 501 oinstall#groupadd -g 502 dba#groupadd -g 504 asmadmin#groupadd -g 506 asmdba#groupadd -g 507 asmoper#useradd -u 501 -g oinstall -G dba,asmadmin,asmdba,asmoper grid#useradd -u 502 -g oinstall -G dba,asmdba oracle#id oracle#id grid(2) 修改用户密码#passwd oracle#passwd grid8. 文件夹及权限
28、#mkdir -p /u01/app/grid/#mkdir -p /u01/app/11.2.0/grid/product/db_1#mkdir -p /u01/app/oraInventory#chown -R grid:oinstall /u01/app#mkdir -p /u01/app/oracle/product/11.2.0/db_1#chown -R oracle:oinstall /u01/app/oracle#chmod -R 775 /u019. 修改系统参数(1) 修改rac1和rac2的limits.conf#vi /etc/security/limits.conf#
29、ORACLE SETTINGgrid soft nproc 2047grid hard nproc 16384grid soft nofile 1024grid hard nofile 65536oracle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536(2) 修改rac1和rac2的/etc/pam.d/login#vi /etc/pam.d/login#ORACLE SETTINGsession required pam_limits.so(3) 修改rac1和ra
30、c2的/etc/sysctl.conf#vi /etc/sysctl.conf#ORACLE SETTINGfs.aio-max-nr = 1048576fs.file-max = 6815744kernel.shmall = 2097152kernel.shmmax = 536870912kernel.shmmni = 4096kernel.sem = 250 32000 100 128net.ipv4.ip_local_port_range = 9000 65500net.core.rmem_default = 262144net.core.rmem_max = 4194304net.co
31、re.wmem_default = 262144net.core.wmem_max = 1048586(4) 执行命令(rac1、rac2)#sysctl -p(5) 修改rac1和rac2的profile，在末尾添加#vi /etc/profileif $USER = oracle | $USER = grid ; thenif $SHELL = /bin/ksh ; thenulimit -p 16384ulimit -n 65536elseulimit -u 16384 -n 65536fiumask 022 fi(6) 执行命令(rac1、rac2)#source /etc/profi
32、le(7) 修改rac1和rac2的/etc/csh.login#vi /etc/csh.loginif ( $USER = oracle | $USER = grid ) thenlimit maxproc 16384limit descriptors 65536endifEOFCSH(8) 修改grid用户的环境变量#su - grid$cd /home/grid/$vi .bash_profileTMP=/tmp; export TMPTMPDIR=$TMP; export TMPDIRORACLE_SID=+ASM1; export ORACLE_SIDORACLE_BASE=/u01
33、/app/grid; export ORACLE_BASEORACLE_HOME=/u01/app/11.2.0/grid/product/db_1; export ORACLE_HOMENLS_DATE_FORMAT=yyyy-mm-dd HH24:MI:SS; export NLS_DATE_FORMATTHREADS_FLAG=native; export THREADS_FLAGPATH=$ORACLE_HOME/bin:$PATH; export PATHTHREADS_FLAG=native; export THREADS_FLAGPATH=$ORACLE_HOME/bin:$PA
34、TH; export PATHif $USER = oracle | $USER = grid ; thenif $SHELL = /bin/ksh ; thenulimit -p 16384ulimit -n 65536elseulimit -u 16384 -n 65536fi umask 022fi#su - grid$cd /home/grid/$vi .bash_profileTMP=/tmp; export TMPTMPDIR=$TMP; export TMPDIRORACLE_SID=+ASM2; export ORACLE_SIDORACLE_BASE=/u01/app/gri
35、d; export ORACLE_BASEORACLE_HOME=/u01/app/11.2.0/grid/product/db_1; export ORACLE_HOMENLS_DATE_FORMAT=yyyy-mm-dd HH24:MI:SS; export NLS_DATE_FORMATTHREADS_FLAG=native; export THREADS_FLAGPATH=$ORACLE_HOME/bin:$PATH; export PATHTHREADS_FLAG=native; export THREADS_FLAGPATH=$ORACLE_HOME/bin:$PATH; expo
36、rt PATHif $USER = oracle | $USER = grid ; thenif $SHELL = /bin/ksh ; thenulimit -p 16384ulimit -n 65536elseulimit -u 16384 -n 65536fi umask 022fi(9) 执行命令(rac1、rac2)$source .bash_profile(10) 修改oracle用户的环境变量#su - oracle$cd /home/oracle/$vi .bash_profile# Oracle Settings oracleTMP=/tmp; export TMPTMPDI
37、R=$TMP; export TMPDIRORACLE_BASE=/u01/app/oracle; export ORACLE_BASEORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOMEORACLE_SID=orcl1; export ORACLE_SIDORACLE_TERM=xterm; export ORACLE_TERMPATH=/usr/sbin:$PATH; export PATHPATH=$ORACLE_HOME/bin:$PATH; export PATHLD_LIBRARY_PATH=$ORACLE
38、_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATHCLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATHNLS_DATE_FORMAT=yyyy-mm-dd HH24:MI:SS; export NLS_DATE_FORMATNLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANGif $USER = oracle | $USER = grid ; thenif $SHELL = /bin/
39、ksh ; thenulimit -p 16384ulimit -n 65536elseulimit -u 16384 -n 65536fiumask 022fi#su - grid$cd /home/grid/$vi .bash_profile# Oracle Settings oracleTMP=/tmp; export TMPTMPDIR=$TMP; export TMPDIRORACLE_BASE=/u01/app/oracle; export ORACLE_BASEORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_
40、HOMEORACLE_SID=orcl2; export ORACLE_SIDORACLE_TERM=xterm; export ORACLE_TERMPATH=/usr/sbin:$PATH; export PATHPATH=$ORACLE_HOME/bin:$PATH; export PATHLD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATHCLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLAS
41、SPATHNLS_DATE_FORMAT=yyyy-mm-dd HH24:MI:SS; export NLS_DATE_FORMATNLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANGif $USER = oracle | $USER = grid ; thenif $SHELL = /bin/ksh ; thenulimit -p 16384ulimit -n 65536elseulimit -u 16384 -n 65536fiumask 022fi(11) 执行命令(rac1、rac2)$source .bash_profile10. 配置
42、用户等效性(1) 使用Oracle用户，在所有节点执行#su - oracle$ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/home/oracle/.ssh/id_rsa): Created directory /home/oracle/.ssh.Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been sa
43、ved in /home/oracle/.ssh/id_rsa.Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.The key fingerprint is:be:8b:34:fd:c9:34:a7:2f:d2:f7:d9:30:1f:2e:48:d6 oraclerac1$ssh-keygen -t dsaGenerating public/private dsa key pair.Enter file in which to save the key (/home/oracle/.ssh/id_dsa): Ent
44、er passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_dsa.Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.The key fingerprint is:4f:bc:4b:9a:a6:cd:1c:15:e7:67:92:94:a2:cd:50:14 oraclerac1(2) 使用Oracle用户，在节点rac1执行$cat /home/oracle/.ssh/*.pub /home/oracle/.ssh/authorized_keys$ssh rac2 cat /home/oracle/.ssh/*.pub /home/oracle/.ssh/authorized_keys $scp /home/oracle
