Oracle RAC Installation Step 1 - Oracle Clusterware Installation and Configuration

NOTE: This Installation is on RHEL5/OEL5/CentOS5

Complete preinstallation tasks

System Hardware requirements

System Requirement
------------------------
At least 1 GB of physical memory is needed.
# grep MemTotal /proc/meminfo
A minimum of 1 GB of swap space is required.
# grep SwapTotal /proc/meminfo
The /tmp directory should be at least 400 MB.
# df -k /tmp
The Oracle Database 10g software requires up to 4 GB of disk space.

Network Hardware Requirement
----------------------------------
Each node must have at least two network adapters.
Each public network adapter must support TCP/IP.
The interconnect adapter must support User Datagram Protocol (UDP).
The host name and IP address associated with the public interface must be registered in the domain name service (DNS) or the /etc/hosts file.


Software requirements

Network Software Requirement
---------------------------------
Supported interconnect software protocols are required:
-TCP/IP
-UDP
-Reliable Data Gram
Token Ring (not supported on AIX platforms)

Package Requirements
-----------------------
Package versions are checked by the cluvfy utility.

RPMs needed for RHEL 4.0 64-bit

Hangcheck-timer Module Configuration
----------------------------------------
The hangcheck-timer module monitors the Linux kernel for hangs.
Make sure that the hangcheck-timer module is running on all nodes:
# /sbin/lsmod | grep -i hang
Add entry to start the hangcheck-timer module on all nodes, if necessary:
# vi /etc/rc.local
/sbin/insmod hangcheck-timer hangcheck_tick=30 hangcheck_margin=180

Required UNIX Groups and Users
-----------------------------------
Create on each node.
user - oracle
group - dba, oinstall
# groupadd -g 500 oinstall
# groupadd -g 501 dba
# useradd -u 500 -d /home/oracle -g "oinstall" -G "dba" -m -s /bin/bash oracle

Verify the existence of the nobody nonprivileged user:
# grep nobody /etc/passwd
Nobody:x:99:99:Nobody:/:/sbin/nobody


Environment configuration

The oracle User Environment
------------------------------
Set umask to 022.
Set the DISPLAY environment variable.
Set the ORACLE_BASE environment variable.
Set the TMP and TMPDIR variables, if needed.

$ cd
$ vi .bash_profile
umask 022
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
TMP=/u01/mytmp; export TMP
TMPDIR=$TMP; export TMPDIR

User Shell Limits
-------------------
Add the following lines to the
/etc/security/limits.conf file:

* soft nproc 2047
* hard nproc 16384
* soft nofile 1024
* hard nofile 65536

Add the following line to the /etc/pam.d/login file:

session required /lib/security/pam_limits.so

Configuring for Remote Installation
------------------------------------
The OUI supports User Equivalence or Secure Shell (ssh)
for remote cluster installations. To configure user equivalence:
Edit the /etc/hosts.equiv file.
Insert both private and public node names for each node in your cluster.

#vi /etc/hosts.equiv
node1
node2

Test the configuration using rsh as the oracle user.
$ rsh node1 uname -r
$ rsh node2 uname -r

To configure Secure Shell:
--------------------------
Create the public and private keys on all nodes:
[node1]$ /usr/bin/ssh-keygen -t dsa
[node2]$ /usr/bin/ssh-keygen -t dsa

Concatenate id_dsa.pub from all nodes into the authorized_keys file on the first node:
[node1]$ ssh node1 "cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
[node2]$ ssh node2 "cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

Copy the authorized_keys file to the other nodes:
[node1]$ scp ~/.ssh/authorized_keys node2:/home/oracle/.ssh/


Linux Operating Systems Parameters
--------------------------------------
Parameter Value File
semmsl 250 /proc/sys/kernel/sem
semmns 32000 /proc/sys/kernel/sem
semopm 100 /proc/sys/kernel/sem
semmni 128 /proc/sys/kernel/sem
shmall 2097152 /proc/sys/kernel/shmall
shmmax ½ physical memory /proc/sys/kernel/shmmax
shmmni 4096 /proc/sys/kernel/shmmni
file-max 65536 /proc/sys/fs/file-max
rmem_max 262144 /proc/sys/net/core/rmem_max
rmem_default 262144 /proc/sys/net/core/rmem_default
wmem_max 262144 /proc/sys/net/core/wmem_max
wmem_default 262144 /proc/sys/net/core/wmem_default


# ipcs -l

------ Shared Memory Limits --------
max number of segments = 4096 // SHMMNI
max seg size (kbytes) = 32768 // SHMMAX
max total shared memory (kbytes) = 8388608 // SHMALL
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 1024 // SEMMNI
max semaphores per array = 250 // SEMMSL
max semaphores system wide = 256000 // SEMMNS
max ops per semop call = 32 // SEMOPM
semaphore max value = 32767

------ Messages: Limits --------
max queues system wide = 1024 // MSGMNI
max size of message (bytes) = 65536 // MSGMAX
default max size of queue (bytes) = 65536 // MSGMNB



Cluster Setup Tasks
---------------------
View the Certifications by Product section at http://metalink.oracle.com/.
Verify your high-speed interconnects.
Determine the shared storage (disk) option for your system:

-OCFS or other shared file system solution
-Raw devices
-ASM

ASM cannot be used for the OCR and Voting Disk files!
Install the necessary operating system patches.


Obtaining OCFS (Optional)
----------------------------
To get OCFS for Linux, visit the Web site at http://oss.oracle.com/projects/ocfs/files.

Download the following Red Hat Package Manager (RPM) packages:
ocfs-support-1.0-n.i386.rpm
ocfs-tools-1.0-n.i386.rpm

Download the following RPM kernel module:
ocfs-2.4.21-EL-typeversion.rpm, where typeversion is the Linux version.

Using Raw Partitions
----------------------
Install shared disk.
Identify the shared disks to use.
Partition the device

# fdisk -l

Disk /dev/sda: 9173 MB, 917311480 bytes 255 heads, 63 sectors/track, 1115 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes.

Disk /dev/sdb: 9173 MB, 917311480 bytes 255 heads, 63 sectors/track, 1115 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes.

# fdisk /dev/sda

Number of Partitions Partition Size(MB) Purpose
1 500 SYSTEM tablespace
1 300+250 per Inst. SYSAUX tablespace
1 per Inst. 500 UNDOTBSn tablespace
1 160 EXAMPLE tablespace
1 120 USERS tablespace
2 per Inst. 120 2 online Redo Logs per Inst.
2 120 First and Second controlfile
1 250 TEMP tablespace
1 5 Server parameter file (SPFILE)
1 5 Password file
1 100 Volume for OCR
1 20 Clusterware voting disk

Binding the Partitions
Identify the devices that are already bound:
# /usr/bin/raw -qa

Edit the /etc/sysconfig/rawdevices file:
# cat /etc/sysconfig/rawdevices file
# raw device bindings
dev/raw/raw1 /dev/sda1
...

Adjust the ownership and permissions of the OCR file to root:dba and 640 respectively

Adjust the ownership and permissions of all other raw files to oracle:dba and 660 respectively

Execute the rawdevices command


Raw Device Mapping file
-------------------------
Create a database directory and set proper permissions:
# mkdir -p $ORACLE_BASE/oradata/dbname
# chown oracle:oinstall $ORACLE_BASE/oradata
# chmod 775 $ORACLE_BASE/oradata

Edit the $ORACLE_BASE/oradata/dbname/dbname_raw.conf file:
# cd $ORACLE_BASE/oradata/dbname
# vi dbname_raw.conf

Set the DBCA_RAW_CONFIG enviroment variable to specify the full path to this file.

Verifying Cluster Setup with cluvfy

Install the cvuqdisk rpm required for cluvfy:

# su root
# cd /stage/10201-production/clusterware/rpm
# export CVUQDISK_GRP=dba
rpm -iv cvuqdisk-1.0.1-1.rpm

Run the cluvfy utility as oracle as shown below:

#cd /u01/stage/10gR2/clusterware/cluvfy

. /runcluvfy.sh stage -post hwos -n all -verbose