A client wanting to determine who had cycled a server in Millennium’s Server Control Panel (SCP) at a certain time went to the SCP’s message log cmb_0033 for the answer. When they opened the file, what did they find? Absolutely nothing. No entries or rows had been written to the message log since their last code upgrade. All of the stability issues and change control events that should have been logged were lost. The problem was much bigger than not knowing who had cycled a server. The organization had no way of knowing what else was wrong or how to resolve those issues.
We checked out three parts of the system to track down the problem.
- Permissions settings in the $cer_log directory. The owners of the files in the $cer_log directory had been set to root for the User and root for the Group. That setting works great much of the time, but not for the following message logs: cmb_0000.mlg, cmb_0033.mlg, cmb_0057.mlg, cmb_0066.mlg and cmb_0078.mlg. For these, the User setting should be root and the Group setting should be your domain user name (for instance, d_prod or d_pxxx, with the xxxbeing your Millennium client number if you are remote hosted). Then the permissions on the file can be -rw-rw—-, which allows the user of root as well as anyone in the group of d_prod to read and write to these log files. If you set these permissions properly, you will have lots of additional information about performance and stability issues in your Millennium domain.
- SCP settings. SCP servers should have a User Name set to root and a Group Name set to your domain user (for example, d_prod). You also need to set the SCP Property of LogLevel to 2 on the following servers: 0 (Service Manager), 33 (Server Control Panel), 66 (CPM HNAM Agent) and 78 (CPM Script Install). The LogLevel for SCP Entry ID 57 (CPM Script Batch Priv) should be set to 1, unless you are actively troubleshooting an issue with this server.
- The messages.suppress file in the $cer_mgr directory. If you cat messages.suppress, you will see all of the messages that are not being written to the message log. You are missing significant information if any rows in this file start with the following characters: CMB_, HNAM_, SCP_, InstallOCD, OCDPURGE and Script.
Prognosis: If you check and correct your message log settings before and after an upgrade, you will have much more insight into your Millennium domain and will be better equipped to troubleshoot stability problems and monitor change control.
