SEI Idea Portal
At many CPACS sites the Execdbcc.sh script does not correctly detect when an execution failure occurs. One of the most common problems is when the script loses connectivity to the database. There are various reasons for this.
The DBCC check is terminated but the script reports a normal completion. The usual root cause for this is that the DBCC Checks take longer than seven days.
When the logs are used to determine whether there is a DBCC error the usual way is to search the logs manually for error messages. Unfortunately, if the DBCC checks die before completion then the logs could be empty, so no errors are found. This means that the CPACS ims database could be corrupted but the log seems to indicate everything is good.
The CheckForDbccMsgs.sh script was written to automatically check all of the DBCC log files in the CPACS log directories for execution and DBCC errors.
The CheckForDbccMsgs.sh script has been tested in all CPACS 4.x, 6.x, and 7.x versions up to and including CPACS 7.0.4 .
The Execdbcc.sh script can fail silently and stop logging. This is why the command "grep Msg /logs/online_dbcc.log" can give a false negative. In other words, if the DBCC check does not complete successfully then the log will not contain any errors.
The KA for the trap E#92311 needs to be updated to refer to the CheckForDbccMsgs.sh script which is the only definitive way to determine whether the DBCC check completed.
The CheckForDbccMsgs.sh script will check all DBCC log files in the /log directory hierarchy for backup and DBCC errors.
Get the latest version of the CheckForDbccMsgs.sh script from the File section of the Related tab in the Salesforce Knowledge Article 000048767
|
Description
The CheckForDbccMsgs_v4.sh script has been implemented and is available for download from Salesforce KB Article 000048767 It was created as a workaround for the Salesforce Problem P-8262 CPACS - Execdbcc.sh does not detect most execution failures At many CPACS sites in the IB the Execdbcc.sh script does not correctly detect when an execution failure occurs. One of the most common problems is when there is a connectivity problem. The DBCC check is terminated but the script reports a normal completion. When the DBCC check logs are used to determine whether there is a DBCC error the usual procedure is to use the command "grep Msg <dbcclog>". Unfortunately, if the DBCC checks do not complete this gives a false negative. The database could still be corrupted but the DBCC check log seems to indicate everything is good. Salesforce KB Article: 000048767, ALM: 1942 "Title: CPACS - Execdbcc.sh does not detect most execution failures" See also ALM 1945 "Title: CPACS L4 - Corruption of syslogs is the single point of failure in replication“. This was also seen at Maricopa and covered by Marc Deshaies in lessons learned. Workaround: Pre-Upgrade, run CheckForDbccMsgs_v4.sh script to check for DBCC errors. |
great article