Search This Blog

Wednesday, April 9, 2014

ORACLE initialization or shutdown in progress

Our database guy was trying to perform some operations on the oracle 11g database today and got the following error:

ORA-011033: ORACLE initialization or shutdown in progress

This could have a lot of underlying errors, but in our situation this occurs when the power is unexpectedly turned off to the server, causing the database transaction logs to not be closed properly.

Provided that you configured RMAN backups when you installed the instance,  (Which should have been rather apparent during the installation of the software and the creation of the database) recovery from this situation should be quite smooth.

I will highlight the commands manually entered.  Anything else is a response from command(s).

First, we log into the server using the oracle account you created before installing the database software.  Once you are logged into the operating system, on the console you will want to run sqlplus as the SYSDBA account, and properly shutdown the database. (It's probably stuck trying to initialize but cannot, since it will attempt to start on boot when the power came back on)  Then you will want to start the instance, manually mount the control files, but NOT open the database yet, just quit.
ORACLEBIR:/export/home/oracle$ sqlplus '/as sysdba'
SQL> shutdown abort
ORACLE Instance shut down.
SQL> startup nomount
ORACLE Instance started
SQL> alter database mount;
SQL> quit
At this point, your instance has been started and the database files have been mounted.  Now we run RMAN or the Recovery Manager.  Here we will request that RMAN performs a recovery.  If you get more errors or this does not work for you, it would seem your backup and recovery settings are not configured properly, and you are really in trouble.  Otherwise, it should look something like this:
ORACLEBOX:/export/home/oracle$ rman
RMAN> connect target
connected to target database: DBV2 (DBID=2494479496, not open)
using target database control file instead of recovery catalog
RMAN> recover database;
Starting recover at 09-APR-14
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=161 device type=DISK
starting media recovery
archived log for thread 1 with sequence 871 is already on disk as file /u1/app/oracle/oradata/dbv2/redo01.log
archived log file name=/u1/app/oracle/oradata/dbv2/redo01.log thread=1 sequence=871
media recovery complete, elapsed time: 00:00:07
Finished recover at 09-APR-14
RMAN> quit
If you see something like above, congratulations, your database has recovered.  Now, let's go back and actually open the database and reset the logs.  Also, make sure to start enterprise manager, and that your backup admin knows how to connect and use this tool to some degree.  Although some do not like the Enterprise Manager, it is essential to automate tasks for anyone who is not familiar with some of the basic operations.  And you might get hit by a bus, leaving all the work in your colleague's lap.
ORACLEBOX:/export/home/oracle$ sqlplus '/as sysdba'
SQL> alter database open resetlogs;
SQL> quit
ORACLEBOX:/export/home/oracle$ emctl start dbconsole
Oracle Enterprise Manager 11g Database Control Release
Copyright (c) 1996, 2011 Oracle Corporation.  All rights reserved.
Starting Oracle Enterprise Manager 11g Database Control .................... started.
Logs are generated in directory /u1/app/oracle/product/

Your EM Login:

No comments:

Post a Comment