CTSSD Runs in Observer Mode Even Though No Time Sync Software is Running (Doc ID 1054006.1)
In this Document
Symptoms
Cause
Solution
References
APPLIES TO:
Oracle Database - Enterprise Edition - Version 11.2.0.1 and later
Information in this document applies to any platform.
SYMPTOMS
CTSSD runs in observer mode even though no time sync software is running.
or
Cluvfy comp clocksync fails with:
$ ./cluvfy comp clocksync
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check failed for "ntpd"
Check failed on nodes:
racbde1
PRVF-5415 : Check to see if NTP daemon is running failed
Clock synchronization check using Network Time Protocol(NTP) failed
PRVF-9652 : Cluster Time Synchronization Services check failed
Verification of Clock Synchronization across the cluster nodes was unsuccessful on all the specified nodes.
or
The clusterware alert log reports errors like:
2009-12-23 20:06:53.974
[ctssd(13443)]CRS-2409:The clock on host racbde2 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode.
Note you could also see the CRS-2409 error if you have a vendor time synchronization service running but the times are sync'd yet.
CAUSE
NTP or some other time sync service is configured but not running. The install, CVU, and CTSSD check for the following :（配置了NTP或其他时间同步服务，但没有运行。安装，CVU和CTSSD检查如下:）
There is only one and only one time synchronization service running at a time.（一次只有一个且只有一个时间同步服务在运行。）
The time sync service can be a vendor service (NTP, Windows Time Service, etc...) or CTSS.（时间同步服务可以是供应商服务(NTP、Windows time服务等)或CTSS。）
If vendor time service is configured, it needs to be correctly configured AND active.（如果配置了供应商时间服务，则需要正确地配置并激活它。）
For CTSS to be in active mode, a vendor time service must not be running, and must not be configured (correctly or otherwise). This is important. CTSS is conservative and will switch to OBSERVER mode the moment it discovers that a vendor service is running or configured on even one node in the cluster. This is to prevent multiple active time sync services running on the cluster, potentially changing clocks.（要使CTSS处于活动模式，供应商时间服务必须不运行，并且不能配置(正确或不正确)。这是很重要的。CTSS是保守的，当它发现供应商服务甚至在集群中的一个节点上运行或配置时，它将切换到OBSERVER模式。这是为了防止集群上运行的多个活动时间同步服务可能会改变时钟。）
SOLUTION
1) If you choose to use a vendor time sync service (like ntp), make sure it is configured AND running.
or
2) If you choose to let CTSSD handle time synchronization, de-configure the vendor time sync service. For example, for NTP move or remove /etc/ntp.conf or /etc/xntp.conf or /etc/chrony.conf (on Linux 7). For Windows platform, please stop the Windows time service W32Time. For more information, refer to Note 1056693.1 How to Configure NTP or Windows Time for Oracle Clusterware.
After choosing option 2 (and removing /etc/ntp.conf in my case), I now see a good time status reported by cluvfy:
$ ./cluvfy comp clocksync
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Active state. Proceeding with check of clock time offsets on all nodes...
Check of clock time offsets passed
Oracle Cluster Time Synchronization Services check passed
Verification of Clock Synchronization across the cluster nodes was successful.
REFERENCES
NOTE:1053147.1 - 11gR2 Clusterware and Grid Home - What You Need to Know
NOTE:1077289.1 - CTSSD Runs in Active Mode When Non-Mainstream Vendor Time Sync Software is Running