Pavan DBA's Blog



ByRajani Kumar Katam, Oracle RAC DBA.Satyam Computer Services private Ltd.Step by step installation Oracle 11g (11.1.0.6.0) RAC on Red hat Enterprise LINUX AS 4 with screenshots.The following are the sequence of steps that are to be executed on the nodes.Install the Linux Operating SystemInstall Required Linux Packages for Oracle RAC (refer oracle doc for the required packages Packages varies depending on the version of the operating system).Network Configuration Using the Network Configuration application, you need to configure both NIC devices as well as the /etc/hosts file. Both of these tasks can be completed using the Network Configuration GUI. Notice that the /etc/hosts settings are the same for both nodes.For example we need to specify the entries in the /etc/hosts file as below in both the nodes.etc/hosts127.0.0.1 localhost.localdomain localhost# Public Network - (eth0)192.168.1.100 linux1192.168.1.101 linux2# Private Interconnect - (eth1)192.168.2.100 linux1-priv192.168.2.101 linux2-priv# Public Virtual IP (VIP) addresses - (eth0)192.168.1.200 linux1-vip192.168.1.201 linux2-vipCreate "oracle" User and Directoriesgroupadd -g 501 oinstall groupadd -g 502 dba groupadd -g 504 asmuseradd -m -u 501 -g oinstall -G dba, asm -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle# mkdir -p /u01/app/oracle# chown -R oracle:oinstall /u01/app# chmod -R 775 /u01/appCreating directory for oracle clusterware.# mkdir -p /u01/app/crs# chown -R oracle:oinstall /u01/app/crs# chmod -R 775 /u01/app/crsCreate Mount Point for OCFS2 / Clusterware Let's now create the mount point for the Oracle Cluster File System, Release 2 (OCFS2) that will be used to store the two Oracle Clusterware shared files (OCR file and voting disk file)# mkdir -p /u02/oradata/orcl# chown -R oracle:oinstall /u02/oradata/orcl# chmod -R 775 /u02/oradata/orclConfigure the Linux Servers for OracleEdit the .bash_profile file and set the required environment variables in both the nodes.PATH=$PATH:$HOME/binexport ORACLE_SID=hrms1export ORACLE_HOME=/u02/app/oracle/db_homeexport ORA_CRS_HOME=/u02/app/oracle/crs_homeexport PATH=$PATH:$ORACLE_HOME/bin:$ORACLE_HOME/libunset USERNAMESwap Space Considerations Installing Oracle Database 11g Release 1 requires a minimum of 1GB of memory# cat /proc/meminfo | grep MemTotal# cat /proc/meminfo | grep SwapTotalConfiguring Kernel Parameters and Shell Limits In both the nodessysctl -pnet.ipv4.ip_forward = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1kernel.msgmnb = 65536kernel.msgmax = 65536net.core.rmem_default = 4194304net.core.rmem_max = 4194304net.core.wmem_default = 262144net.core.wmem_max = 262144kernel.shmmax = 1073741823kernel.sem = 250 32000 100 128fs.file-max = 65536net.ipv4.ip_local_port_range = 1024 65000Setting Shell Limits for the oracle User cat >> /etc/security/limits.conf <<EOForacle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536EOFConfigure RAC Nodes for Remote Access using SSH(Refer any UNIX doc for configuring SSH or oracle RAC installation doc)Enabling SSH User Equivalency for the Current Shell Session $ exec /usr/bin/ssh-agent $SHELL$ /usr/bin/ssh-add Enter passphrase for /home/oracle/.ssh/id_rsa: xxxxx Identity added: /home/oracle/.ssh/id_rsa (/home/oracle/.ssh/id_rsa)Reboot both nodes after configuring kernel level Install & Configure Oracle Cluster File System (OCFS2) parameters# rpm -Uvh ocfs2-tools-1.2.6-1.el5.i386.rpm# rpm -Uvh ocfs2-2.6.18-8.el5-1.2.6-1.el5.i686.rpm# rpm -Uvh ocfs2console-1.2.6-1.el5.i386.rpmConfigure OCFS2 $ su -# ocfs2console &Select [Cluster] -> [Configure Nodes...]. This will start the OCFS2 ClusterOn the "Node Configuration" dialog, click the [Add] button. This will bring up the "Add Node" dialog. In the "Add Node" dialog, enter the Host name and IP address for the first node in the cluster. Leave the IP Port set to its default value of 7777. In my example, I added both nodes using linux1 / 192.168.1.100 for the first node and linux2 / 192.168.1.101 for the second nodeClick [Apply] on the "Node Configuration" dialog - All nodes should now be "Active". After verifying all values are correct, exit the application using [File] -> [Quit].This needs to be performed on both Oracle RAC nodes in the cluster. Configure O2CB to Start on Boot# /etc/init.d/o2cb offline ocfs2# /etc/init.d/o2cb unload# /etc/init.d/o2cb configureFormat the OCFS2 Filesystem Create a partition on the SAN or shared storage for storing ocrfile and voting disk files that are created at the time of cluster ware installation.(use fdisk command as root user and create a partition)NOTE:Always recommended to create 4 partitions so that we can maintain redundant copies of voting disk file and OCR file$ su -# mkfs.ocfs2 -b 4K -C 32K -N 4 –L ocfs2 /dev/sde2Mount the OCFS2 File system$ su -# mount -t ocfs2 -o datavolume,nointr –L “ocfs2” /u02/oradata/orcl (the name of the file system where u want to mount tat partition).Configure OCFS2 to Mount Automatically at StartupWe can do that by adding the following line to the /etc/fstab file on both Oracle RAC nodes in the cluster: LABEL=ocfs2 /u02/oradata/orcl ocfs2 _netdev,datavolume,nointr 0 010.Install & Configure Automatic Storage Management libraries (ASMLib 2.0)# rpm -Uvh oracleasm-support-2.0.4-1.el5.i386.rpm# rpm -Uvh oracleasm-2.6.18-8.el5-2.0.4-1.el5.i686.rpm# rpm -Uvh oracleasmlib-2.0.3-1.el5.i386.rpmConfiguring and Loading the ASMLib Packages $ su -# /etc/init.d/oracleasm configureCreate ASM Disks for Oracle $ su -# /etc/init.d/oracleasm createdisk vol1 /dev/sde1# /etc/init.d/oracleasm createdisk VOL2 /dev/sde2NOTE:Create the number of disks depending on your requirement.# /etc/init.d/oracleasm scandisks# /etc/init.d/oracleasm listdisksVOL1VOL2VOL3VOL410 .Pre-Installation Tasks for Oracle Clusterware 11gVerifying the Hardware and Operating System Setup with CVU$ ./runcluvfy.sh stage -post hwos -n hcslinux1, hcslinux2 –verbose11.Installing Oracle clusterware softwareNote: Before installing clusterware please verify remote host access and user equivalence using ssh.$ sh runInstaller[root@hcslnx01 crs_home]# sh root.shChecking to see if Oracle CRS stack is already configuredSetting the permissions on OCR backup directorySetting up Network socket directoriesOracle Cluster Registry configuration upgraded successfullySuccessfully accumulated necessary OCR keys.Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node <nodenumber>: <nodename> <private interconnect name> <hostname>node 1: hcslnx01 hcslnx01-priv hcslnx01node 2: hcslnx02 hcslnx02-priv hcslnx02Creating OCR keys for user 'root', privgrp 'root'..Operation successful.Now formatting voting device: /ocfs2/voting_fileFormat of 1 voting devices complete.Startup will be queued to init within 30 seconds.Adding daemons to inittabExpecting the CRS daemons to be up within 600 seconds.Cluster Synchronization Services is active on these nodes. hcslnx01Cluster Synchronization Services is inactive on these nodes. hcslnx02Local node checking complete. Run root.sh on remaining nodes to start CRS daemons.[root@hcslnx01 crs_home]#[root@hcslnx02 crs_home]# sh root.shWARNING: directory '/' is not owned by rootChecking to see if Oracle CRS stack is already configuredSetting the permissions on OCR backup directorySetting up Network socket directoriesOracle Cluster Registry configuration upgraded successfullyThe directory '/' is not owned by root. Changing owner to rootclscfg: EXISTING configuration version 4 detected.clscfg: version 4 is 11 Release 1.Successfully accumulated necessary OCR keys.Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node <nodenumber>: <nodename> <private interconnect name> <hostname>node 1: hcslnx01 hcslnx01-priv hcslnx01node 2: hcslnx02 hcslnx02-priv hcslnx02clscfg: Arguments check out successfully.NO KEYS WERE WRITTEN. Supply -force parameter to override.-force is destructive and will destroy any previous clusterconfiguration.Oracle Cluster Registry for cluster has already been initializedStartup will be queued to init within 30 seconds.Adding daemons to inittabExpecting the CRS daemons to be up within 600 seconds.Cluster Synchronization Services is active on these nodes. hcslnx01 hcslnx02Cluster Synchronization Services is active on all the nodes.Waiting for the Oracle CRSD and EVMD to startOracle CRS stack installed and running under init(1M)Running vipca(silent) for configuring nodeappsCreating VIP application resource on (2) nodes...Creating GSD application resource on (2) nodes...Creating ONS application resource on (2) nodes...Starting VIP application resource on (2) nodes...Starting GSD application resource on (2) nodes...Starting ONS application resource on (2) nodes...Done.You have new mail in /var/spool/mail/root[root@hcslnx02 crs_home]#12. Verify Oracle Clusterware Installation Check Cluster Nodes$ $ORA_CRS_HOME/bin/olsnodes -nlinux1 1linux2 2Confirm Oracle Clusterware Function$ $ORA_CRS_HOME/bin/crs_stat -t -vName Type R/RA F/FT Target State Host----------------------------------------------------------------------ora.linux1.gsd application 0/5 0/0 ONLINE ONLINE linux1ora.linux1.ons application 0/3 0/0 ONLINE ONLINE linux1ora. application 0/0 0/0 ONLINE ONLINE linux1ora.linux2.gsd application 0/5 0/0 ONLINE ONLINE linux2ora.linux2.ons application 0/3 0/0 ONLINE ONLINE linux2ora. application 0/0 0/0 ONLINE ONLINE linux2Check CRS Status $ $ORA_CRS_HOME/bin/crsctl check crsCluster Synchronization Services appears healthyCluster Ready Services appears healthyEvent Manager appears healthyCheck Oracle Clusterware Auto-Start Scripts$ ls -l /etc/init.d/init.*-rwxr-xr-x 1 root root 2236 Oct 12 22:08 /etc/init.d/init.crs-rwxr-xr-x 1 root root 5290 Oct 12 22:08 /etc/init.d/init.crsd-rwxr-xr-x 1 root root 49416 Oct 12 22:08 /etc/init.d/init.cssd-rwxr-xr-x 1 root root 3859 Oct 12 22:08 /etc/init.d/init.evmd?13. Install Oracle Database 11g Software$ sh runInstallerInstalling Oracle 11g RAC software…NOTE:The above command has to be executed manullay on the failed node by connecting as Oracle user.Creating 11g RAC database by invoking DBCA utility.NOTE:There are five node-level tasks defined for SRVCTL: Adding and deleting node-level applications Setting and unsetting the environment for node-level applications Administering node applications Administering ASM instances Starting and stopping a group of programs that includes virtual IP addresses, listeners, Oracle Notification Services, and Oracle Enterprise Manager agents (for maintenance purposes). Status of all instances and services$ srvctl status database -d orclInstance orcl1 is running on node linux1Instance orcl2 is running on node linux2Status of a single instance$ srvctl status instance -d orcl -i orcl2Instance orcl2 is running on node linux2Status of node applications on a particular node$ srvctl status nodeapps -n linux1VIP is running on node: linux1GSD is running on node: linux1Listener is running on node: linux1ONS daemon is running on node: linux1Status of an ASM instance$ srvctl status asm -n linux1ASM instance +ASM1 is running on node linux1.List all configured databases$ srvctl config databaseorclDisplay configuration for our RAC database$ srvctl config database -d orcllinux1 orcl1 /u01/app/oracle/product/11.1.0/db_1linux2 orcl2 /u01/app/oracle/product/11.1.0/db_1Display the configuration for node applications - (VIP, GSD, ONS, Listener)$ srvctl config nodeapps -n linux1 -a -g -s -lVIP exists.: /linux1-vip/192.168.1.200/255.255.255.0/eth0GSD exists.ONS daemon exists.Listener exists.Display the configuration for the ASM instance(s)$ srvctl config asm -n linux1+ASM1 /u01/app/oracle/product/11.1.0/db_1BYRajani Kumar KatamSatyam Computer Services private Ltd. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download