oracle 12c r2 RAC配置手册.doc

1、ORACLE 12C R2 Real Application Cluster Installation Guide朱海清StarTimes Software Technology Co., LtdASM 磁盘空间最低要求12C R2 相比前一版本， OCR 的磁盘占用需求有了明显增长。为了方便操作，设置如下：External: 1 个卷 x40GNormal: 3 个卷 x30GHight: 5 个卷 x25GFlex: 3 个卷 x30GOCR+VOLTING+MGMT 存储通常放到一个磁盘组，且 选择 Normal 的冗余方式，也即最少 3 块 asm 磁盘 80G 空间。操作系统安装操作

2、系统安装时把“Server with GUI“ 和”Compatibility Libraries”勾上，其他都不用 选择。版本采用 CentOS 7、RHEL 7 或者 Oracle Linux 7安装 oracle 预安装包yum install -y oracle-rdbms-server-12cR1-preinstall创建用户和组oracle 用户和 dba、oinstall 组已经在上一步创建完毕。rac 所有节点的 oracle 用户和 grid 用户的 uid 和 gid 必须一致，所以创建的时候最好制定 uid 和 gid。groupadd -gid 54323 asmdba

3、groupadd -gid 54324 asmopergroupadd -gid 54325 asmadmingroupadd -gid 54326 opergroupadd -gid 54327 backupdbagroupadd -gid 54328 dgdbagroupadd -gid 54329 kmdbausermod -uid 54321 -gid oinstall -groups dba,oper,asmdba,asmoper,backupdba,dgdba,kmdba oracleuseradd -uid 54322 -gid oinstall -groups dba,asma

4、dmin,asmdba,asmoper grid安装目录mkdir -p /u01/app/12.2.0/gridmkdir -p /u01/app/gridmkdir -p /u01/app/oraclechown -R grid:oinstall /u01chown oracle:oinstall /u01/app/oraclechmod -R 775 /u01/用户环境变量grid 环境变量cat /home/grid/.bash_profileORACLE_SID=+ASM1ORACLE_HOME=/u01/12.2.0/gridPATH=$ORACLE_HOME/bin:$PATHL

5、D_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/libCLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlibexport ORACLE_SID CLASSPATH ORACLE_HOME LD_LIBRARY_PATH PATHEOF在节点 2，ORACLE_SID=+ASM2oracle 环境变量cat /home/oracle/.bash_profileORACLE_SID=starboss1ORACLE_HOME=/u01/app/oracle/product/12.2.

6、0/db_1ORACLE_HOSTNAME=rac01PATH=$ORACLE_HOME/bin:$PATHLD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/libCLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlibexport ORACLE_SID ORACLE_HOME ORACLE_HOSTNAME PATH LD_LIBRARY_PATH CLASSPATHEOF在节点 2，ORACLE_SID=starboss2，ORACLE_HOSTNAME=rac02修改 lo

7、gind.conf# vi /etc/systemd/logind.conf RemoveIPC=no # systemctl daemon-reload # systemctl restart systemcd-logind加载 pam_limits.so 模块echo “session required pam_limits.so“ /etc/pam.d/login禁用 selinuxsetenforce 0vi /etc/sysconfig/selinux禁用防火墙# systemctl stop firewalld thenif $SHELL = “/bin/ksh“ ; thenul

8、imit -u 16384ulimit -n 65536elseulimit -u 16384 -n 65536fifiif $USER = “grid“ ; thenif $SHELL = “/bin/ksh“ ; thenulimit -u 16384ulimit -n 65536elseulimit -u 16384 -n 65536fifiEOF修改共享内存分区大小将如下参数添加到/etc/fstab，具体大小数值根据实际情况调整，因为这个数值和物理内存以及 MEMORY_TARGET 有关。echo “shm /dev/shm tmpfs size=12g 0 0” /etc/fst

9、ab修改后，只需重新对 shm 进行挂载即可：mount -o remount /dev/shm多路径# yum install device-mapper-multipath# cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/获取 scsi id# /usr/lib/udev/scsi_id -whitelisted -replace-whitespace -device=/dev/sda# vi /etc/multipath.confmultipaths multipath wwid 36000d310

10、012522000000000000000006alias vol01multipath wwid 36000d310012522000000000000000005alias vol02# systemctl start multipathd.service# multipath -ll配置磁盘ASMlib 方式安装 ASMLibOracle Linux 7yum install -y kmod-oracleasmCentOS 7yum install -y http:/mirror.centos.org/centos/7/os/x86_64/Packages/kmod-oracleasm-

11、2.0.8-17.el7.centos.x86_64.rpmyum install -y http:/ install -y http:/public- 磁盘配置12C R2 中对磁盘组空间要求比 12C R1 更大。rootrac01 # /etc/init.d/oracleasm configure -iConfiguring the Oracle ASM library driver.This will configure the on-boot properties of the Oracle ASM librarydriver. The following questions wil

12、l determine whether the driver isloaded on boot and what permissions it will have. The current valueswill be shown in brackets (). Hitting without typing ananswer will keep that current value. Ctrl-C will abort.Default user to own the driver interface : gridDefault group to own the driver interface

13、: asmadminStart Oracle ASM library driver on boot (y/n) n: yScan for Oracle ASM disks on boot (y/n) y: yWriting Oracle ASM library driver configuration: doneInitializing the Oracle ASMLib driver: OK Scanning the system for Oracle ASMLib disks: OK rootrac01 # reboot用 fdisk 在共享磁盘上创建主分区：rootrac01 # fdi

14、sk /dev/sddWelcome to fdisk (util-linux 2.23.2).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition tableBuilding a new DOS disklabel with disk identifier 0x86f899a0.Command (m for help): nPartiti

15、on type:p primary (0 primary, 0 extended, 4 free)e extendedSelect (default p): pPartition number (1-4, default 1): First sector (2048-39976959, default 2048): Using default value 2048Last sector, +sectors or +sizeK,M,G (2048-39976959, default 39976959): Using default value 39976959Partition 1 of typ

16、e Linux and of size 19.1 GiB is setCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.在集群的任意节点创建 asm 磁盘：rootrac01 # /etc/init.d/oracleasm createdisk OCR01 /dev/sdd1Marking disk “OCR01“ as an ASM disk: OK rootrac01 # /etc/init.d/oracle

17、asm createdisk OCR02 /dev/sde1Marking disk “OCR02“ as an ASM disk: OK rootrac01 # /etc/init.d/oracleasm createdisk OCR03 /dev/sdf1Marking disk “OCR03“ as an ASM disk: OK rootrac01 # /etc/init.d/oracleasm createdisk DATA01 /dev/sdb1Marking disk “DATA01“ as an ASM disk: OK rootrac01 # /etc/init.d/orac

18、leasm createdisk DATA02 /dev/sdc1Marking disk “DATA02“ as an ASM disk: OK 分别两个节点执行：rootrac01 # /etc/init.d/oracleasm scandisksrootrac01 # /etc/init.d/oracleasm listdisks注：如果需要清空磁盘，重新部署 asm，需要使用 dd 命令，如：dd if=/dev/zero of=/dev/sdb1 bs=8192 count=128000UDEV 方式参考：https:/oracle- RAC 节点上已 经安装了必要的 UDEV 包r

19、ootrh2 # rpm -qa|grep udev udev-095-14.21.el5CentOS 6/Oracle Linux 6/RHEL 6 1.通过 scsi_id 获取设备的块设备的唯一标识名，假设系统上已有 LUN sdc-sdpfor i in c d e f g h i j k l m n o p ; do echo “sd$i“ “scsi_id -g -u -s /block/sd$i “; done sdc 1IET_00010001 sdd 1IET_00010002 sde 1IET_00010003 sdf 1IET_00010004 以上列出于块设备名对应的唯

20、一标识名 2.创建必要的 UDEV 配置文件， 首先切换到配置文件目录 rootrh2 # cd /etc/udev/rules.d 定义必要的规则配置文件 rootrh2 rules.d# touch 99-oracle-asmdevices.rules rootrh2 rules.d# cat 99-oracle-asmdevices.rules KERNEL=“sd*“, BUS=“scsi“, PROGRAM=“/sbin/scsi_id -g -u -s %p“, RESULT=“1IET_00010001“, NAME=“ocr1“, OWNER=“grid“, GROUP=“as

21、madmin“, MODE=“0660“ KERNEL=“sd*“, BUS=“scsi“, PROGRAM=“/sbin/scsi_id -g -u -s %p“, RESULT=“1IET_00010002“, NAME=“ocr2“, OWNER=“grid“, GROUP=“asmadmin“, MODE=“0660“ KERNEL=“sd*“, BUS=“scsi“, PROGRAM=“/sbin/scsi_id -g -u -s %p“, RESULT=“1IET_00010003“, NAME=“asm-disk1“, OWNER=“grid“, GROUP=“asmadmin“

22、, MODE=“0660“ KERNEL=“sd*“, BUS=“scsi“, PROGRAM=“/sbin/scsi_id -g -u -s %p“, RESULT=“1IET_00010004“, NAME=“asm-disk2“, OWNER=“grid“, GROUP=“asmadmin“, MODE=“0660“ Result 为/sbin/scsi_id -g -u -s %p 的输出， 按顺序填入刚才获取的唯一标识名即可 OWNER 一般 为 grid，GROUP 为 asmadmin，MODE 即为磁盘读写权限，采用 0660 即可 NAME 为 UDEV 映射后的设备名， 建

23、议为 OCR 和 VOTE DISK 创建独立的 DISKGROUP，为了容易区分将该 DISKGROUP 专用的设备 命名为 ocr1ocrn 的形式 其余磁盘可以根据其实际用途或磁盘组名来命名 3.将该规则文件拷贝到其他节点上 rootrh2 rules.d# scp 99-oracle-asmdevices.rules Other_node:/etc/udev/rules.d 4.在所有节点上启动 udev 服务，或者重启服 务器即可 rootrh2 rules.d# /sbin/udevcontrol reload_rules rootrh2 rules.d# /sbin/start_

24、udev Starting udev: OK 5.检查设备是否到位 rootrh2 rules.d# cd /dev rootrh2 dev# ls -l ocr* brw-rw- 1 grid asmadmin 8, 32 Jul 10 17:31 ocr1 brw-rw- 1 grid asmadmin 8, 48 Jul 10 17:31 ocr2 rootrh2 dev# ls -l asm-disk* brw-rw- 1 grid asmadmin 8, 64 Jul 10 17:31 asm-disk1 brw-rw- 1 grid asmadmin 8, 80 Jul 10 17

25、:31 asm-disk2 brw-rw- 1 grid asmadmin 8, 96 Jul 10 17:31 asm-disk3 brw-rw- 1 grid asmadmin 8, 112 Jul 10 17:31 asm-disk4 CentOS 7/Oracle Linux 7/RHEL 7获取块设备 id# /usr/lib/udev/scsi_id -g -u -d /dev/sdb114f504e46494c45526a75744363422d796357662d4b436a65# /usr/lib/udev/scsi_id -g -u -d /dev/sdc114f504e4

26、6494c455254535a7a414d2d62494b6f2d5a6f6a42# /usr/lib/udev/scsi_id -g -u -d /dev/sdd114f504e46494c45526566324e626c2d4770654c2d6b443064# /usr/lib/udev/scsi_id -g -u -d /dev/sde114f504e46494c455266326e7547552d384953442d6135576a# /usr/lib/udev/scsi_id -g -u -d /dev/sdf114f504e46494c4552774263526f742d534a

27、75392d36374f69创建参数文件touch /etc/scsi_id.configoptions=-g# cat /etc/udev/rules.d/99-oracle-asmdevices.rulesKERNEL=“sd?1“, SUBSYSTEM=“block“, PROGRAM=“/usr/lib/udev/scsi_id -g -u -d /dev/$parent“, RESULT=“14f504e46494c45526a75744363422d796357662d4b436a65“, SYMLINK+=“asm-disk1“, OWNER=“grid“, GROUP=“asm

28、admin“, MODE=“0660“KERNEL=“sd?1“, SUBSYSTEM=“block“, PROGRAM=“/usr/lib/udev/scsi_id -g -u -d /dev/$parent“, RESULT=“14f504e46494c455254535a7a414d2d62494b6f2d5a6f6a42“, SYMLINK+=“asm-disk2“, OWNER=“grid“, GROUP=“asmadmin“, MODE=“0660“KERNEL=“sd?1“, SUBSYSTEM=“block“, PROGRAM=“/usr/lib/udev/scsi_id -g

29、 -u -d /dev/$parent“, RESULT=“14f504e46494c45526566324e626c2d4770654c2d6b443064“, SYMLINK+=“asm-disk3“, OWNER=“grid“, GROUP=“asmadmin“, MODE=“0660“KERNEL=“sd?1“, SUBSYSTEM=“block“, PROGRAM=“/usr/lib/udev/scsi_id -g -u -d /dev/$parent“, RESULT=“14f504e46494c455266326e7547552d384953442d6135576a“, SYML

30、INK+=“asm-disk4“, OWNER=“grid“, GROUP=“asmadmin“, MODE=“0660“KERNEL=“sd?1“, SUBSYSTEM=“block“, PROGRAM=“/usr/lib/udev/scsi_id -g -u -d /dev/$parent“, RESULT=“14f504e46494c4552774263526f742d534a75392d36374f69“, SYMLINK+=“asm-disk5“, OWNER=“grid“, GROUP=“asmadmin“, MODE=“0660“加载并刷新块设备分区表# /sbin/partpr

31、obe /dev/sdb1# /sbin/partprobe /dev/sdc1# /sbin/partprobe /dev/sdd1# /sbin/partprobe /dev/sde1# /sbin/partprobe /dev/sdf1udev 测试# /sbin/udevadm test /block/sdb/sdb1# /sbin/udevadm test /block/sdc/sdc1# /sbin/udevadm test /block/sdd/sdd1# /sbin/udevadm test /block/sde/sde1# /sbin/udevadm test /block/

32、sdf/sdf1启动服务# /sbin/udevadm control -reload-rules检查连接生成rootudev # ls -l /dev/asm-disk*lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk1 - sdb1lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk2 - sdc1lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk3 - sdd1lrwxrwxrwx 1 root root 4 Aug 22 13:19

33、 /dev/asm-disk4 - sde1lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk5 - sdf1禁用 ntp/sbin/service ntpd stopchkconfig ntpd offmv /etc/ntp.conf /etc/ntp.conf.orgrm /var/run/ntpd.pid停止 avahi-daemon 服务systemctl stop avahi-dnsconfdsystemctl stop avahi-daemonsystemctl disable avahi-dnsconfdsystemctl di

34、sable avahi-daemonIP 配置如果不安装 DNS 服务，通过 hosts 文件来解析， 则只能配置一个 SCAN IP，只能 连接 rac 某一个节点，无法实现负载均衡。#public，接 业务交换机，bond192.168.245.134 rac01192.168.245.140 rac02#private，直连心跳，bond10.0.1.1 rac01-priv10.0.1.2 rac02-priv#virtual192.168.245.136 rac01-vip192.168.245.142 rac02-vip#scan-ip，oracle rac service192.

35、168.245.135 rac-cluster-scan安装 cvuqdisk 软件包安装包在数据库安装软件压缩包的 rpm 文件夹下。rpm -ivh cvuqdisk-1.0.10-1.rpm安装 GI从 Oracle Grid Infrastructure 12c Release 2 (12.2)开始， GI 安装方式变成了 image-based 方式，Oracle 提供的 Grid 安装文件是直接已经安装好的 ORACLE_HOME,因此我们需要把 GRID 的安装文件直接解压到预先创建好的 GIRD ORACLE_HOME 中，然后运行 gridSetup.sh 启动图形界面，开始

36、配置 GRID。# su - grid$cd /u01/12.2.0/grid$unzip /oracle_soft/grid_12201.zip$ ./gridSetup.sh选择“Configure Oracle Grid Infrastructure for a New Cluster”,点击 Next第 4 布，初始界面可能只看到本地节点，点 击 Add 添加对方节点即可，输入 hosts 文件中配置的节点名称和 vip 的名称。在“SSH connectivity” 中， 输入 grid 用户的密码，点 击 setup，安装程序会自动设置 grid 用户的 ssh 免秘登 陆认证。按

37、照事先的规划，选择网卡用途第八步是创建 OCR 和 VOLTING 的存储磁盘，如果界面中没有磁盘，点击“Change Discovery Path ”重新 发现一下磁盘即可。如果我们没有在环境中配置 DNS 和 GNS 服务， 检查 就会报 DNS 和 resolve.conf 的错误，因为我们采用的是 hosts 文件解析，这里略过即可。点击 install，GI 会开始安装，并复制文件到其它 节点机型同步安装。这里提示用 root 用户执行脚本，必须先在本地节点挨个启动执行，成功之后，才能再在其他节点并行。rootrac01 # sh /u01/app/oraInventory/orai

38、nstRoot.sh Changing permissions of /u01/app/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world.Changing groupname of /u01/app/oraInventory to oinstall.The execution of the script is complete.rootrac01 # sh /u01/app/12.2.0/grid/root.shPerforming roo

39、t user operation.The following environment variables are set as: ORACLE_OWNER= gridORACLE_HOME= /u01/app/12.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: Copying dbhome to /usr/local/bin .Copying oraenv to /usr/local/bin . Copying coraenv to /usr/local/bin .Creating /et

40、c/oratab file.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Relinking oracle with rac_on optionUsing configuration parameter file: /

41、u01/app/12.2.0/grid/crs/install/crsconfig_paramsThe log of current session can be found at:/u01/app/grid/crsdata/rac01/crsconfig/rootcrs_rac01_2017-08-16_02-48-07PM.log2017/08/16 14:48:16 CLSRSC-594: Executing installation step 1 of 19: SetupTFA.2017/08/16 14:48:16 CLSRSC-4001: Installing Oracle Tra

42、ce File Analyzer (TFA) Collector.2017/08/16 14:48:59 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.2017/08/16 14:48:59 CLSRSC-594: Executing installation step 2 of 19: ValidateEnv.2017/08/16 14:49:04 CLSRSC-363: User ignored prerequisites during installation2017/08/1

43、6 14:49:04 CLSRSC-594: Executing installation step 3 of 19: CheckFirstNode.2017/08/16 14:49:05 CLSRSC-594: Executing installation step 4 of 19: GenSiteGUIDs.2017/08/16 14:49:06 CLSRSC-594: Executing installation step 5 of 19: SaveParamFile.2017/08/16 14:49:12 CLSRSC-594: Executing installation step

44、6 of 19: SetupOSD.2017/08/16 14:49:13 CLSRSC-594: Executing installation step 7 of 19: CheckCRSConfig.2017/08/16 14:49:13 CLSRSC-594: Executing installation step 8 of 19: SetupLocalGPNP.2017/08/16 14:49:44 CLSRSC-594: Executing installation step 9 of 19: ConfigOLR.2017/08/16 14:49:51 CLSRSC-594: Exe

45、cuting installation step 10 of 19: ConfigCHMOS.2017/08/16 14:49:52 CLSRSC-594: Executing installation step 11 of 19: CreateOHASD.2017/08/16 14:49:57 CLSRSC-594: Executing installation step 12 of 19: ConfigOHASD.2017/08/16 14:50:12 CLSRSC-330: Adding Clusterware entries to file oracle-ohasd.service20

46、17/08/16 14:50:35 CLSRSC-594: Executing installation step 13 of 19: InstallAFD.2017/08/16 14:50:40 CLSRSC-594: Executing installation step 14 of 19: InstallACFS.CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on rac01CRS-2793: Shutdown of Oracle High Availability S

47、ervices-managed resources on rac01 has completedCRS-4133: Oracle High Availability Services has been stopped.CRS-4123: Oracle High Availability Services has been started.2017/08/16 14:51:20 CLSRSC-594: Executing installation step 15 of 19: InstallKA.2017/08/16 14:51:25 CLSRSC-594: Executing installation step 16 of 19: InitConfig.CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on rac01CRS-2793: Shutdown of Oracle High Availability Services-ma