Dec 29 14:27:13 racdb3 ntpd[6193]: synchronized to LOCAL(0), stratum 8
Dec 29 14:27:13 racdb3 ntpd[6193]: kernel time sync enabled 0001
Dec 29 14:36:41 racdb3 kernel: qla2xxx 0000:04:00.0: scsi(7:0:0): Abort command issued -- 1 1587 2002.
Dec 29 14:37:32 racdb3 logger: Oracle CSSD failure. Rebooting for cluster integrity.
Dec 29 14:37:32 racdb3 kernel: md: stopping all md devices.
Dec 29 14:37:33 racdb3 kernel: Synchronizing SCSI cache for disk sdc:
Dec 29 14:37:33 racdb3 kernel: Synchronizing SCSI cache for disk sdb:
Dec 29 14:37:33 racdb3 kernel: Synchronizing SCSI cache for disk sda:
Dec 29 14:37:34 racdb3 kernel: ACPI: PCI interrupt for device 0000:05:00.1 disabled
上面是系统日志，下面是ORACLE日志；一般出现上面的日志后，系统会重启。
[oracle@racdb3 ~]$ cat /u01/app/oracle/admin/racdbidc/bdump/alert_racdbidc1.log
Tue Dec 29 13:27:19 2009
Trace dumping is performing id=[cdmp_20091229132719]
Tue Dec 29 13:27:47 2009
Errors in file /u01/app/oracle/admin/racdbidc/bdump/racdbidc1_asmb_8843.trc:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
Tue Dec 29 13:27:47 2009
ASMB: terminating instance due to error 15064
Tue Dec 29 13:27:47 2009
Errors in file /u01/app/oracle/admin/racdbidc/bdump/racdbidc1_lms1_8735.trc:
ORA-15064: communication failure with ASM instance
Tue Dec 29 13:27:50 2009
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/admin/racdbidc/bdump/racdbidc1_diag_8723.trc
Tue Dec 29 13:27:50 2009
Trace dumping is performing id=[cdmp_20091229132750]
Tue Dec 29 13:27:52 2009
Instance terminated by ASMB, pid = 8843
Tue Dec 29 13:33:51 2009
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Interface type 1 eth1 172.16.1.0 configured from OCR for use as a cluster interconnect
Interface type 1 eth0 192.168.1.0 configured from OCR for use as a public interface
Picked latch-free SCN scheme 1
WARNING: db_recovery_file_dest is same as db_create_file_dest
Autotune of undo retention is turned on.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
processes = 300
sessions = 335
__shared_pool_size = 1543503872
__large_pool_size = 16777216
__java_pool_size = 16777216
__streams_pool_size = 0
spfile = /u01/app/oracle/product/10.2.0/db_1/dbs/spfileracdbidc1.ora
sga_target = 2483027968
control_files = +LLPWDB/racdbidc/controlfile/current.261.703277161, +LLPWDB/racdbidc/controlfile/current.260.703277163
db_block_size = 8192
__db_cache_size = 889192448
compatible = 10.2.0.1.0
log_archive_dest_1 = location=/logstore/archlog
db_file_multiblock_read_count= 16
cluster_database = TRUE
cluster_database_instances= 2
db_create_file_dest = +LLPWDB
db_recovery_file_dest = +LLPWDB
db_recovery_file_dest_size= 85899345920
thread = 1
instance_number = 1
undo_management = AUTO
undo_tablespace = UNDOTBS1
remote_login_passwordfile= EXCLUSIVE
db_domain =
dispatchers = (PROTOCOL=TCP) (SERVICE=racdbidcXDB)
remote_listener = LISTENERS_RACDBIDC
job_queue_processes = 10
background_dump_dest = /u01/app/oracle/admin/racdbidc/bdump
user_dump_dest = /u01/app/oracle/admin/racdbidc/udump
core_dump_dest = /u01/app/oracle/admin/racdbidc/cdump
audit_file_dest = /u01/app/oracle/admin/racdbidc/adump
db_name = racdbidc
open_cursors = 300
pga_aggregate_target = 824180736
Cluster communication is configured to use the following interface(s) for this instance
172.16.1.215
Tue Dec 29 13:33:51 2009
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=8619
DIAG started with pid=3, OS id=8621
PSP0 started with pid=4, OS id=8623
LMON started with pid=5, OS id=8625
LMD0 started with pid=6, OS id=8647
LMS0 started with pid=7, OS id=8655
LMS1 started with pid=8, OS id=8659
MMAN started with pid=9, OS id=8663
DBW0 started with pid=10, OS id=8665
LGWR started with pid=11, OS id=8667
CKPT started with pid=12, OS id=8669
SMON started with pid=13, OS id=8671
RECO started with pid=14, OS id=8673
CJQ0 started with pid=15, OS id=8675
MMON started with pid=16, OS id=8677
Tue Dec 29 13:33:51 2009
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=17, OS id=8679
Tue Dec 29 13:33:51 2009
starting up 1 shared server(s) ...
Tue Dec 29 13:33:52 2009
lmon registered with NM - instance id 1 (internal mem no 0)
Tue Dec 29 13:33:53 2009
Reconfiguration started (old inc 0, new inc 4)
pseudo shared rm latch used
List of nodes:
0 1
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
* domain 0 valid according to instance 1
* domain 0 valid = 1 according to instance 1
Tue Dec 29 13:33:54 2009
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Tue Dec 29 13:33:54 2009
LMS 0: 0 GCS shadows cancelled, 0 closed
Tue Dec 29 13:33:54 2009
LMS 1: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Tue Dec 29 13:33:54 2009
LMS 0: 0 GCS shadows traversed, 0 replayed
Tue Dec 29 13:33:54 2009
LMS 1: 0 GCS shadows traversed, 0 replayed
Tue Dec 29 13:33:54 2009
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=20, OS id=8737
Tue Dec 29 13:33:55 2009
ALTER DATABASE MOUNT
Tue Dec 29 13:33:55 2009
Starting background process ASMB
ASMB started with pid=22, OS id=8767
Starting background process RBAL
RBAL started with pid=23, OS id=8771
Loaded ASM Library - Generic Linux, version 2.0.4 (KABI_V2) library for asmlib interface
Tue Dec 29 13:34:03 2009
SUCCESS: diskgroup LLPWDB was mounted
Tue Dec 29 13:34:07 2009
Setting recovery target incarnation to 2
Tue Dec 29 13:34:07 2009
Successful mount of redo thread 1, with mount id 412816193
Tue Dec 29 13:34:07 2009
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE MOUNT
Tue Dec 29 13:34:08 2009
ALTER DATABASE OPEN
Picked broadcast on commit scheme to generate SCNs
Tue Dec 29 13:34:19 2009
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=25, OS id=9357
Tue Dec 29 13:34:19 2009
ARC0: Archival started
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=26, OS id=9359
Tue Dec 29 13:34:20 2009
Thread 1 opened at log sequence 368
Current log# 9 seq# 368 mem# 0: +LLPWDB/racdbidc/onlinelog/loggroup9-1
Current log# 9 seq# 368 mem# 1: +LLPWDB/racdbidc/onlinelog/loggroup9-2
Successful open of redo thread 1
Tue Dec 29 13:34:20 2009
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Tue Dec 29 13:34:20 2009
ARC1: STARTING ARCH PROCESSES
Tue Dec 29 13:34:20 2009
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Tue Dec 29 13:34:20 2009
SMON: enabling cache recovery
Tue Dec 29 13:34:20 2009
ARC2: Archival started
ARC1: STARTING ARCH PROCESSES COMPLETE
ARC1: Becoming the heartbeat ARCH
ARC2 started with pid=27, OS id=9411
Tue Dec 29 13:34:23 2009
Successfully onlined Undo Tablespace 1.
Tue Dec 29 13:34:23 2009
SMON: enabling tx recovery
Tue Dec 29 13:34:23 2009
Database Characterset is AL32UTF8
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=28, OS id=9514
Tue Dec 29 13:34:25 2009
Completed: ALTER DATABASE OPEN
Tue Dec 29 13:40:19 2009
Shutting down archive processes
Tue Dec 29 13:40:24 2009
ARCH shutting down
ARC2: Archival stopped
Tue Dec 29 13:44:40 2009
Error: unexpected error (6) from the Cluster Service (LCK0)
Tue Dec 29 13:44:40 2009
Errors in file /u01/app/oracle/admin/racdbidc/bdump/racdbidc1_lck0_8737.trc:
ORA-29702: error occurred in Cluster Group Service operation
LCK0: terminating instance due to error 29702
Tue Dec 29 13:44:40 2009
Errors in file /u01/app/oracle/admin/racdbidc/bdump/racdbidc1_lmon_8625.trc:
ORA-29702: error occurred in Cluster Group Service operation
Tue Dec 29 13:44:40 2009
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/admin/racdbidc/bdump/racdbidc1_diag_8621.trc
Tue Dec 29 13:44:40 2009
Trace dumping is performing id=[cdmp_20091229134440]
本文转自 jxwpx 51CTO博客，原文链接：http://blog.51cto.com/jxwpx/252740，如需转载请自行联系原作者