Check Cluster Commands

Ref:http://youroracledba.wordpress.com/2012/11/23/rac-commands-srvctl-in-detail/

display the registered databases	srvctl config database
status	srvctl status database -d <database srvctl status instance -d <database> -i <instance> srvctl status nodeapps -n <node> srvctl status service -d <database> srvctl status asm -n <node>
stopping/starting	srvctl stop database -d <database> srvctl stop instance -d <database> -i <instance>,<instance> srvctl stop service -d <database> [-s <service><service>] [-i <instance>,<instance>] srvctl stop nodeapps -n <node> srvctl stop asm -n <node> srvctl start database -d <database> srvctl start instance -d <database> -i <instance>,<instance> srvctl start service -d <database> -s <service><service> -i <instance>,<instance> srvctl start nodeapps -n <node> srvctl start asm -n <node>
adding/removing	srvctl add database -d <database> -o <oracle_home> srvctl add instance -d <database> -i <instance> -n <node> srvctl add service -d <database> -s <service> -r <preferred_list> srvctl add nodeapps -n <node> -o <oracle_home> -A <name|ip>/network srvctl add asm -n <node> -i <asm_instance> -o <oracle_home> srvctl remove database -d <database> -o <oracle_home> srvctl remove instance -d <database> -i <instance> -n <node> srvctl remove service -d <database> -s <service> -r <preferred_list> srvctl remove nodeapps -n <node> -o <oracle_home> -A <name|ip>/network srvctl asm remove -n <node>

Use the crsctl command to determine if the cluster is working on a specific node

$GRID_HOME/bin/crsctl check crs

use the crsctl command to check if CSS is running across all nodes of the cluster:

$GRID_HOME/bin/crsctl check cluster

crsctl command will provide a report on all the cluster resources and if they are running.

$GRID_HOME/bin/crsctl stat res –t

Starting and Stopping the Cluster

Use the crsctl command to start and stop the cluster on a specific node. This example starts the cluster and the following example stops the cluster.

$GRID_HOME/bin/crsctl start crs

$GRID_HOME/bin/crsctl stop crs

Enable or Disable Oracle Clusterware Daemons

$GRID_HOME/bin/crsctl disable crs

$GRID_HOME/bin/crsctl enable crs

Managing the Voting Disks

crsctl command is also useful for managing the voting disks. This command locates all the voting disks that you have allocated:

$GRID_HOME/bin/crsctl query css votedisk

Adding or Removing a Database to the Oracle Restart Configuration using SRVCTL

If you add a RAC database manually you will need to add it to the Oracle Restart configuration so Oracle Restart will manage it. In the following example we add a database called newdb:

Srvctl add database –d newdb –o $ORACLE_HOME

This command will remove that same database from the Oracle Restart configuration. Note that this does not remove the database, shut it down or clean up database datafiles.

Srvctl remove database –d newdb

Start and Stop a RAC Database with SRVCTL

The following commands will start and stop a database instance on the cluster.

Srvctl stop database –d orcl –i orcl1

If we wanted to stop the database on the entire cluster we would issue this command instead:

Srvctl stop database –d orcl

The following command will restart the orcl1 instance on the orcl database:

Srvctl start database –d orcl –i orcl1

This command will restart all instances of the orcl database:

Srvctl start database –d orcl

Adding or Removing a Database to the Oracle Restart Configuration using SRVCTL

If you add a RAC database manually you will need to add it to the Oracle Restart configuration so Oracle Restart will manage it. In the following example we add a database called newdb:

Srvctl add database –d newdb –o $ORACLE_HOME

This command will remove that same database from the Oracle Restart configuration. Note that this does not remove the database, shut it down or clean up database datafiles.

Srvctl remove database –d newdb

Start and Stop a RAC Database with SRVCTL

The following commands will start and stop a database instance on the cluster.

Srvctl stop database –d orcl –i orcl1

If we wanted to stop the database on the entire cluster we would issue this command instead:

Srvctl stop database –d orcl

The following command will restart the orcl1 instance on the orcl database:

Srvctl start database –d orcl –i orcl1

This command will restart all instances of the orcl database:

Srvctl start database –d orcl

Checking a Oracle Database Restart Configuration using SRVCTL

The following example checks the restart configuration for an Oracle Database using the srvctl command:

Srvctl config database –d orcl

Moving a datafile from a ASM disk group to another disk group and move database from one Disk group to another disk group

Moving a datafile from a ASM disk group to another disk group and move database from one Disk group to another disk group

This is for full database move. This will move all datafiles under ASM to a new disk group

First place the database in mount mode
SQL>startup mount;
Then switch to RMAN ...
RMAN> backup as copy database format '+NEW_DISC_GROUP' ;
RMAN> switch database to copy ;
SQL> alter database open;
move the database from one disk group to another disk group
1. Backup the spfile and identify the spfile location,
2. start the database in mount mode

SQL> select name from v$database;

NAME
---------
TEST

SQL> select open_mode from V$database;

OPEN_MODE
----------
MOUNTED

connect RMAN 

RMAN> connect target /

connected to target database: TEST (DBID=1868766554, not open)

RMAN> backup as copy database format '+FRA';

Starting backup at 07-APR-14
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=1088 devtype=DISK
channel ORA_DISK_1: starting datafile copy
input datafile fno=00008 name=+DATA/test/datafile/demo12.290.740943031
output filename=+FRA/test/datafile/demo12.3605.844266055 tag=TAG20140407T142054 recid=27 stamp=844266074
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:25
channel ORA_DISK_1: starting datafile copy
input datafile fno=00011 name=+DATA/test/datafile/hello1_data.362.740943031
output filename=+FRA/test/datafile/hello1_data.3609.844266081 tag=TAG20140407T142054 recid=28 stamp=844266108
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile fno=00001 name=+DATA/test/datafile/system.361.740943031
output filename=+FRA/test/datafile/system.3624.844266115 tag=TAG20140407T142054 recid=29 stamp=844266144
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile fno=00003 name=+DATA/test/datafile/sysaux.360.740943031
output filename=+FRA/test/datafile/sysaux.3512.844266151 tag=TAG20140407T142054 recid=30 stamp=844266184
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile fno=00004 name=+DATA/test/datafile/Rajdba.359.740943031
output filename=+FRA/test/datafile/Rajdba.3546.844266187 tag=TAG20140407T142054 recid=31 stamp=844266192
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:08
channel ORA_DISK_1: starting datafile copy
input datafile fno=00005 name=+DATA/test/datafile/boisjmsdb.358.740943031
output filename=+FRA/test/datafile/boisjmsdb.3470.844266193 tag=TAG20140407T142054 recid=32 stamp=844266198
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting datafile copy
input datafile fno=00006 name=+DATA/test/datafile/xxxxxxxxxxx.357.740943031
output filename=+FRA/test/datafile/xxxxxxxxxxx.3489.844266201 tag=TAG20140407T142054 recid=33 stamp=844266205
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting datafile copy
input datafile fno=00007 name=+DATA/test/datafile/users.356.844256443
output filename=+FRA/test/datafile/users.3477.844266207 tag=TAG20140407T142054 recid=34 stamp=844266213
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting datafile copy
input datafile fno=00002 name=+DATA/test/datafile/undotbs1.355.740943057
output filename=+FRA/test/datafile/undotbs1.3471.844266215 tag=TAG20140407T142054 recid=35 stamp=844266216
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting datafile copy
input datafile fno=00009 name=+DATA/test/datafile/demo1.354.740943057
output filename=+FRA/test/datafile/demo1.3750.844266217 tag=TAG20140407T142054 recid=36 stamp=844266218
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
channel ORA_DISK_1: starting datafile copy
input datafile fno=00012 name=+DATA/test/datafile/hello.353.740943059
output filename=+FRA/test/datafile/hello.3450.844266219 tag=TAG20140407T142054 recid=37 stamp=844266219
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
channel ORA_DISK_1: starting datafile copy
input datafile fno=00010 name=+DATA/test/datafile/demo.302.740943059
output filename=+FRA/test/datafile/demo.3447.844266221 tag=TAG20140407T142054 recid=38 stamp=844266220
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
channel ORA_DISK_1: starting datafile copy
copying current control file
output filename=+FRA/test/controlfile/backup.3445.844266221 tag=TAG20140407T142054 recid=39 stamp=844266221
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 07-APR-14

RMAN>


RMAN> switch database to copy ;

datafile 1 switched to datafile copy "+FRA/test/datafile/system.3624.844266115"
datafile 2 switched to datafile copy "+FRA/test/datafile/undotbs1.3471.844266215"
datafile 3 switched to datafile copy "+FRA/test/datafile/sysaux.3512.844266151"
datafile 4 switched to datafile copy "+FRA/test/datafile/Rajdba.3546.844266187"
datafile 5 switched to datafile copy "+FRA/test/datafile/boisjmsdb.3470.844266193"
datafile 6 switched to datafile copy "+FRA/test/datafile/xxxxxxxxxxx.3489.844266201"
datafile 7 switched to datafile copy "+FRA/test/datafile/users.3477.844266207"
datafile 8 switched to datafile copy "+FRA/test/datafile/demo12.3605.844266055"
datafile 9 switched to datafile copy "+FRA/test/datafile/demo1.3750.844266217"
datafile 10 switched to datafile copy "+FRA/test/datafile/demo.3447.844266221"
datafile 11 switched to datafile copy "+FRA/test/datafile/hello1_data.3609.844266081"
datafile 12 switched to datafile copy "+FRA/test/datafile/hello.3450.844266219"


RMAN> sql 'alter database open';

sql statement: alter database open

RMAN>

SQL> select FILE_NAME,TABLESPACE_NAME from dba_data_files;

FILE_NAME                                                              TABLESPACE_NAME
---------------------------------------------------------------------- ------------------------------
+FRA/test/datafile/system.3624.844266115                               SYSTEM
+FRA/test/datafile/undotbs1.3471.844266215                             UNDOTBS1
+FRA/test/datafile/sysaux.3512.844266151                               SYSAUX
+FRA/test/datafile/Rajdba.3546.844266187                          Rajdba
+FRA/test/datafile/boisjmsdb.3470.844266193                           xxxxxyyyy
+FRA/test/datafile/xxxxxxxxxxx.3489.844266201                         xxxxxxxxxxx
+FRA/test/datafile/users.3477.844266207                                USERS
+FRA/test/datafile/demo12.3605.844266055                           demo12
+FRA/test/datafile/demo1.3750.844266217                          demo1
+FRA/test/datafile/demo.3447.844266221                           demo
+FRA/test/datafile/hello1_data.3609.844266081                    hello1_DATA
+FRA/test/datafile/hello.3450.844266219                           hello

I'll explain two ways of moving a datafile from a ASM disk group to another disk group.

Here is the first way:

1.       Find the name of the datafile that you're going to migrate:

SQL> SELECT FILE_NAME FROM DBA_DATA_FILES WHERE tablespace_name='USERS';

FILE_NAME

--------------------------------------------------------------------------------

+DATA/test/datafile/users.356.740943031

2. Take the tablespace offline:

SQL> alter tablespace USERS offline;

Tablespace altered.

3. Copy the file using RMAN:

RMAN> COPY datafile '+DATA/test/datafile/users.356.740943031' to '+FRA';

4. Change datafile's name to the name displayed as output of the command above:    

SQL> ALTER DATABASE RENAME FILE '<old_name>' TO '<output file name written at rman output>';

SQL> ALTER DATABASE RENAME FILE '+DATA/test/datafile/users.356.740943031' to '+FRA/test/datafile/users.3605.844255353';

Database altered.

SQL> alter tablespace USERS online;

Tablespace altered.

SQL> select FILE_NAME,TABLESPACE_NAME ,ONLINE_STATUS from DBA_DATA_FILES where TABLESPACE_NAME='USERS';

FILE_NAME

--------------------------------------------------------------------------------

TABLESPACE_NAME                ONLINE_

------------------------------ -------

+DATA/test/datafile/users.356.844256443

USERS                          ONLINE

And here is the second way:

1. Find the name of the datafile that you're going to migrate just like firt method:

SQL> SELECT FILE_NAME FROM DBA_DATA_FILES WHERE tablespace_name='USERS';

FILE_NAME

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

+FRA/test/datafile/users.3605.844255353

2. Take the datafile offline instead this time:

SQL> ALTER DATABASE DATAFILE '+FRA/test/datafile/users.3605.844255353' OFFLINE;

Database altered.

3. Copy the file using RMAN:

RMAN> connect target /

connected to target database: TEST (DBID=1868766554)

RMAN> COPY datafile '+FRA/test/datafile/users.3605.844255353' to '+DATA';

Starting backup at 07-APR-14

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: sid=1074 devtype=DISK

channel ORA_DISK_1: starting datafile copy

input datafile fno=00007 name=+FRA/test/datafile/users.3605.844255353

output filename=+DATA/test/datafile/users.356.844256443 tag=TAG20140407T114042 recid=26 stamp=844256446

channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07

Finished backup at 07-APR-14

4. Change datafile's name to the new name displayed as output of the command above to update data dictionary: 

SQL> ALTER DATABASE RENAME FILE '+FRA/test/datafile/users.3605.844255353' to '+DATA/test/datafile/users.356.844256443';

Database altered.

5. Rename the datafile using RMAN to update ASM. This command is equivalent of "alter database rename file":  

RMAN> SWITCH DATAFILE '<new_name>' TO COPY;

RMAN> SWITCH DATAFILE '+DATA/test/datafile/users.356.844256443' to copy;

datafile 7 switched to datafile copy "+DATA/test/datafile/users.356.844256443"

RMAN>

RMAN> sql 'alter tablespace USERS online';

sql statement: alter tablespace USERS online

RMAN>

6. Recover  the new datafile:

RMAN> RECOVER DATAFILE '<new_name>';

RMAN> RECOVER DATAFILE '+DATA/test/datafile/users.356.844256443';

Starting recover at 07-APR-14

allocated channel: ORA_DISK_1

channel ORA_DISK_1: sid=1081 devtype=DISK

starting media recovery

media recovery complete, elapsed time: 00:00:02

Finished recover at 07-APR-14

RMAN>

7. Bring it online

SQL> ALTER DATABASE DATAFILE '<new_name>' ONLINE;

RMAN> sql 'alter tablespace USERS online';

sql statement: alter tablespace USERS online

SQL> select FILE_NAME,TABLESPACE_NAME ,ONLINE_STATUS from DBA_DATA_FILES where TABLESPACE_NAME='USERS';

FILE_NAME

--------------------------------------------------------------------------------

TABLESPACE_NAME                ONLINE_

------------------------------ -------

+DATA/test/datafile/users.356.844256443

USERS                          ONLINE

Oracle Blogs

http://vijaydbacorner.blogspot.com.au/

http://select-star-from.blogspot.com.au/

Patching steps

Ref:http://select-star-from.blogspot.com.au/search/label/PATCHING


Patch 13621679 - 11.1.0.7.11 Patch Set Update
Patchset/PSU    Patch Number    Description
11.1.0.7.11    13621679    DATABASE PATCH SET UPDATE 11.1.0.7.11 (INCLUDES CPU APR2012)

Patch 13621679 - 11.1.0.7.11 Patch Set Update
=============================================
http://bbs.dbsupport.cn/thread-160-1-1.html

Patch InformationPatch Set Update (PSU) patches are cumulative. That is, the content of all previous PSUs is included in the latest PSU patch.

PSU 11.1.0.7.11 includes all fixes previously included in PSU 11.1.0.7.10 and those listed in .
To install the PSU 11.1.0.7.11 patch, the Oracle home must have the 11.1.0.7.0 Database installed. Subsequent PSU patches can be installed on Oracle Database 11.1.0.7.0 or any PSU with a lower 5th numeral version than the one being installed.

OPatch Utility
==============
You must use the OPatch utility version 11.1.0.8.2 or later to apply this patch. Oracle recommends that you use the latest released OPatch 11.1, which is available for download from My Oracle Support patch6880880 by selecting the 11.1.0.0.0 release.
For information about OPatch documentation, including any known issues, see My Oracle Support Note 293369.1 OPatch documentation list.

PSU Patching 11G Steps on PRIMARY database
==========================================

check DR sysnc status
=====================
http://select-star-from.blogspot.in/2013/09/data-guard-sync-status.html

------------------------------------------------------------------------------------------------------------------

Start OEM black out
===================

Confirmation 
Blackout "Blackout-Apr 24 1998 9:33:42 PM" created successfully, Blackout status will be propagated to the target(s) shortly. 

------------------------------------------------------------------------------------------------------------------

Node 1: (AS ORACLE) 
===================

$ hostname
$ uname
$ bash
$ date
$ df -kg or df -h
$ cat /etc/oratab or cat/var/opt/oracle/oratab  ------> Note ASM and Database Home Paths
$ ps -ef| grep pmon
$ ps -ef| grep tns
$ ps -ef| grep emagent
$ ps -ef|grep oracle
$ ps -ef| grep d.bin
$ ps -ef|grep crs
 root  344170       1   0   Apr 24      -  0:00 /oradb/crs/product/11.1.0/crs_1/bin/oclskd.bin

$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012
$ opatch version
$ /oradb/crs/product/11.1.0/crs_1/OPatch/opatch version
$ /oradb/asm/product/11.1.0/asm_1/OPatch/opatch version
$ /oradb/app/oracle/product/11.1.0/db_1/OPatch/opatch version
$ which opatch
/oradb/app/oracle/product/11.1.0/db_1/OPatch/opatch
. oraenv  --- asm instance
$ opatch lsinventory
. oraenv  --- database 1 instance
$ opatch lsinventory
. oraenv  --- database 2 instance
$ opatch lsinventory

$ ps -ef| grep pmon
$ srvctl status database -d <database_1>                       or srvctl status instance -d <database_1> -i <database_1_instance> 
$ srvctl status database -d <database_2>                       or srvctl status instance -d <database_2> -i <database_2_instance>

$ srvctl status asm -n <Node 1>
$ srvctl status asm -n <Node 2>

$ srvctl status listener -n <Node 1>
$ srvctl status listener -n <Node 2>

$ srvctl status nodeapps -n <Node 1>
$ srvctl status nodeapps -n <Node 2>

$ ps -ef|grep oracle or  ps -ef|grep emagent
./emctl status agent or  /oradb/app/oracle/product/agent1/agent11g/bin/emctl status agent

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ ./crsctl check cluster

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ ./crsctl check crs

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------

$ srvctl stop database -d <database_1>              or srvctl stop insatnce -d <database_1> -i <database_1_instance>
$ srvctl stop database -d <database_2> -o immediate or srvctl stop insatnce -d <database_2> -i <database_2_instance>

$ srvctl stop asm -n <Node 1>
$ srvctl stop asm -n <Node 2>

$ srvctl stop listener -n <Node 1>
$ srvctl stop listener -n <Node 2>

$ srvctl stop nodeapps -n <Node 1>
$ srvctl stop nodeapps -n <Node 2>

$ ps -ef|grep oracle or  ps -ef|grep emagent
./emctl stop agent   or  /oradb/app/oracle/product/agent1/agent11g/bin/emctl stop agent

$ ps -ef| grep oracle
$ lsnrctl stop     -------------------------------to stop default listener
$ ps -ef| grep oracle

------------------------------------------------------------------------------------------------------------------

Node 2: (AS ORACLE) 
===================

$ hostname
$ uname
$ bash
$ date
$ df -kg or df -h
$ cat /etc/oratab or cat/var/opt/oracle/oratab  ------> Note ASM and Database Home Paths
$ ps -ef| grep pmon
$ ps -ef| grep tns
$ ps -ef| grep emagent
$ ps -ef|grep oracle
$ ps -ef| grep d.bin
$ ps -ef|grep crs
 root  344170       1   0   Apr 24      -  0:00 /oradb/crs/product/11.1.0/crs_1/bin/oclskd.bin

$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012

$ opatch version

$ /oradb/crs/product/11.1.0/crs_1/OPatch/opatch version
$ /oradb/asm/product/11.1.0/asm_1/OPatch/opatch version
$ /oradb/app/oracle/product/11.1.0/db_1/OPatch/opatch version

$ which opatch
/oradb/app/oracle/product/11.1.0/db_1/OPatch/opatch

$ ps -ef| grep pmon

$ ps -ef|grep oracle or  ps -ef|grep emagent
./emctl status agent or  /oradb/app/oracle/product/agent1/agent11g/bin/emctl status agent

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ ./crsctl check cluster

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ ./crsctl check crs

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------

$ ps -ef|grep oracle or  ps -ef|grep emagent
./emctl stop agent   or  /oradb/app/oracle/product/agent1/agent11g/bin/emctl stop agent

$ ps -ef| grep oracle
$ lsnrctl stop     -------------------------------to stop default listener
$ ps -ef| grep oracle

------------------------------------------------------------------------------------------------------------------

Pre-rootpatch
=============

Node 1: (AS ROOT)
=================

# cd /oradb/crs/product/11.1.0/crs_1/bin
# ./crsctl check cluster
# ./crsctl check crs
# ./crsctl stop crs

# ps -ef| grep oracle

# ssh <Node 2>

# cd /oradb/crs/product/11.1.0/crs_1/bin
# ./crsctl check cluster
# ./crsctl check crs
# ./crsctl stop crs

# ps -ef| grep oracle

# exit
# hostname
<Node 1>

# ps -ef| grep pmon   ---- (0 instances running)
# cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953
# sh custom/scripts/prerootpatch.sh -crshome /oradb/crs/product/11.1.0/crs_1 -crsuser oracle

# ssh <Node 2>

# ps -ef| grep pmon   ---- (0 instances running)
# cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953
# sh custom/scripts/prerootpatch.sh -crshome /oradb/crs/product/11.1.0/crs_1 -crsuser oracle

# exit
# hostname
<Node 1>

------------------------------------------------------------------------------------------------------------------

Pre-patch
=========

Node 1: (AS ORACLE)
===================

$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953
$ custom/scripts/prepatch.sh -crshome /oradb/crs/product/11.1.0/crs_1

$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953
$ custom/server/11724953/custom/scripts/prepatch.sh -dbhome /oradb/asm/product/11.1.0/asm_1

$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953
$ custom/server/11724953/custom/scripts/prepatch.sh -dbhome /oradb/app/oracle/product/11.1.0/db_1

------------------------------------------------------------------------------------------------------------------

Opatch Apply
============

Node 1: (AS ORACLE)
===================

CRS
===
$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077
$ opatch napply -oh /oradb/crs/product/11.1.0/crs_1 -id 11724953

ASM
===
$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/server
$ opatch napply custom/server/ -oh /oradb/asm/product/11.1.0/asm_1 -id 11724953

RDBMS
=====
$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/server
$ opatch napply custom/server/ -oh /oradb/app/oracle/product/11.1.0/db_1 -id 11724953

------------------------------------------------------------------------------------------------------------------

Do the post configure for CRS and RDBMS home
============================================

Node 1: (AS ORACLE)
===================

CRS
===
$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/scripts
$ ./postpatch.sh -crshome /oradb/crs/product/11.1.0/crs_1

ASM
===
$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/server/11724953
$ ./custom/scripts/postpatch.sh -dbhome /oradb/asm/product/11.1.0/asm_1

RDBMS
=====
$ cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/server/11724953
$ ./custom/scripts/postpatch.sh -dbhome /oradb/app/oracle/product/11.1.0/db_1

------------------------------------------------------------------------------------------------------------------

11.1.0.7.11 PSU (13621679) and (9734685)
========================================

Node 1: (AS ORACLE)
===================

13621679
========

$ cd /oradb/app/admin/orasw/patches
$ chmod -R 777 PSU_patches
$ cd PSU_patches
$ pwd 
/oradb/app/admin/orasw/patches/PSU_patches

$ ls -lrt p13621679_111070_AIX5L.zip
$ unzip p13621679_111070_AIX5L.zip
$ ls -lrt
$ cd 13621679
$ pwd

ASM
===
$ /oradb/app/admin/orasw/patches/PSU_patches/13621679
$ opatch apply -oh /oradb/asm/product/11.1.0/asm_1

RDBMS
=====
$ cd /oradb/app/admin/orasw/patches/PSU_patches/13621679
$ opatch apply -oh /oradb/app/oracle/product/11.1.0/db_1

------------------------------------------------------------------------------------------------------------------

9734685
=======

$ cd /oradb/app/admin/orasw/patches/PSU_patches/
$ ls -lrt p9734685_1110711_AIX64-5L.zip
$ unzip p9734685_1110711_AIX64-5L.zip

ASM
===
$ cd /oradb/app/admin/orasw/patches/PSU_patches/9734685
$ opatch apply -oh /oradb/asm/product/11.1.0/asm_1

RDBMS
=====
$ cd /oradb/app/admin/orasw/patches/PSU_patches/9734685
$ opatch apply -oh /oradb/app/oracle/product/11.1.0/db_1

------------------------------------------------------------------------------------------------------------------

After 9734685 Patch applied relink has to be done in both ASM and DB homes ------- Linking Oracle
==========================================================================

Node 1: (AS ORACLE)
===================

RDBMS
=====
$ cd /oradb/app/oracle/product/11.1.0/db_1/rdbms/lib
$ make -f ins_rdbms.mk ipc_g ioracle

ASM
===
$ cd /oradb/asm/product/11.1.0/asm_1/rdbms/lib
$ make -f ins_rdbms.mk ipc_g ioracle

Node 2: (AS ORACLE)
=======

RDBMS
=====
$ cd /oradb/app/oracle/product/11.1.0/db_1/rdbms/lib
$ make -f ins_rdbms.mk ipc_g ioracle

ASM
===
$ cd /oradb/asm/product/11.1.0/asm_1/rdbms/lib
$ make -f ins_rdbms.mk ipc_g ioracle

------------------------------------------------------------------------------------------------------------------

Post-rootpatch
==============

Node 1: (AS ROOT)
=================

# cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953

# /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/scripts/postrootpatch.sh -crshome /oradb/crs/product/11.1.0/crs_1

# ps -ef | grep pmon  --- Both the db and asm instances will be up and running

# ssh <Node 2>

# cd /oradb/app/admin/orasw/patches/PSU_patches/Jan2012/CRS/111077/11724953/custom/scripts/postrootpatch.sh -crshome /oradb/crs/product/11.1.0/crs_1

# ps -ef | grep pmon  --- Both the db and asm instances will be up and running

------------------------------------------------------------------------------------------------------------------

Start ASM

------------------------------------------------------------------------------------------------------------------

Run on each database, only in one node
======================================

NODE 1: (AS ORACLE)
===================

$ sqlplus /nolog
Enter user-name: / as sysdba
Connected to an idle instance.

SQL>startup;
SQL>@?/rdbms/admin/catbundle.sql psu apply
SQL>exit

$. oraenv ---- --- database 2 instance
$ sqlplus /nolog
Enter user-name: / as sysdba

SQL>startup;
SQL>@?/rdbms/admin/catbundle.sql psu apply
SQL>exit

------------------------------------------------------------------------------------------------------------------

Patch Validation:-
================

opatch lsinventory | grep 11724953
opatch lsinventory | grep 13621679
opatch lsinventory | grep 9734685

start local listeners, (if any)

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ ./crsctl check cluster

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ ./crsctl check crs

$ cd /oradb/crs/product/11.1.0/crs_1/bin
$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------

$ hostname
$ uname
$ bash
$ date
$ who -b
$ uptime
$ df -kg or df -h
$ cat /etc/oratab or cat/var/opt/oracle/oratab  ------> Note ASM and Database Home Paths
$ ps -ef| grep pmon
$ ps -ef| grep tns
$ ps -ef| grep emagent
$ ps -ef|grep oracle
$ ps -ef| grep d.bin
$ ps -ef|grep crs

$ ps -ef| grep pmon

$ srvctl status database -d <database_1>                       
$ srvctl status database -d <database_2>                       

$ srvctl status asm -n <Node 1>
$ srvctl status asm -n <Node 2>

$ srvctl status listener -n <Node 1>
$ srvctl status listener -n <Node 2>

$ srvctl status nodeapps -n <Node 1>
$ srvctl status nodeapps -n <Node 2>

$ ps -ef|grep oracle or  ps -ef|grep emagent
./emctl status agent or  /oradb/app/oracle/product/agent1/agent11g/bin/emctl status agent

Run on each database only in one node
=====================================

NODE 1: (AS ORACLE)
===================
SQL> select name db_name,DB_UNIQUE_NAME,instance_name,status,DATABASE_STATUS,open_mode, database_role, host_name,platform_id,version db_version,log_mode,flashback_on ,protection_mode,protection_level,LOGINS,to_char(STARTUP_TIME,'DD-MON-YYYY HH24:MI:SS') "UP TIME" from gv$instance,v$database;

SQL> select to_char(sysdate,'yyyy-mm-dd HH24:MI:SS') from dual
/

SQL> 
col ACTION_TIME for a12
col NAMESPACE for a10
col VERSION for a10
col BUNDLE_SERIES for a15
col COMMENTS for a20
select action_time, action,namespace, version, id,bundle_series, comments from registry$history;

------------------------------------------------------------------------------------------------------------------

check DR sysnc status
=====================
http://select-star-from.blogspot.in/2013/09/data-guard-sync-status.html

------------------------------------------------------------------------------------------------------------------

Stop OEM black out
==================

Confirmation 
Request to stop blackout "Blackout-Apr 24 1998 9:33:42 PM" was processed successfully, Blackout status will be propagated to the target(s) shortly.