Setting Proper LogLevels Is as Easy as 0-1-2-3-4

I am often asked about proper log level settings (LogLevel) for Millennium®servers. The setting determines what types of messages are written into a server’s message log files, which provide critical information to support teams for correcting application problems and some technical issues. If you set the LogLevel too low, you might miss important information. If you set it too high, you might be flooded with extraneous messages (and overlook the ones you need to address).

LogLevel can be set to one of the following values:

         0 = Error (ERROR on Java servers)
         1 = Warning (WARN on Java)
         2 = Audit (AUDIT on Java)
         3 = Information (INFORM on Java)
         4 = Debug (DEBUG on Java)

The settings are cumulative, meaning that servers will write messages from that level as well as all lower levels. For example, a 2 setting would include all Audit (2), Warning (1) and Error (0) messages. You can view LogLevels and change their settings in OnTrack Panther™, SCPView or Olympus. Any time you change a LogLevel you need to cycle the server in order for the change to take effect.

In general, the lower the number, the greater the severity of the messages sent to the log files. Unfortunately, many Audit events are actually failed messages or transactions, which is one reason why a too-low LogLevel might result in lost information…and why many people have questions about what the settings should be. That wasn’t always the case.

In 1997, when most Millennium clients were running on VMS, disk saturation concerns meant that LogLevel was always set to 0/ERROR. When you had two or more application nodes with hundreds of servers pumping log information on the same disk drive, the drive — and everything else — would slow down. AIX, with its SSA drives, had problems as well. The drives shared the same connections as the database. If the executables were using up the connection from the CPU to the disk, then the Oracle database could not get the disk time it needed to operate as well as it could.

Disk saturation is not as much of an issue now that most clients are on 2007.02 or greater, many VMS clients have migrated to some flavor of Unix and the AIX clients are off of SSA drives. Yet logging can still be a problem, even when you are running on a well-tuned storage area network. Today, logging usually manifests itself as a specific queuing problem on a specific executable. If you set servers to LogLevel 0/ERROR, you might miss information that is important to use in finding and correcting problems in your environments. I recommend you no longer use this setting.

I also recommend that no server be set to LogLevel 3/INFORM or 4/DEBUG unless you have an active point asking for debug information. Short of that, these settings cause too many disk writes. I was working with a client recently who had the EKS servers in debug, and the message logs were wrapping in less than 15 minutes. This much logging slowed down all of the EKS rules processing but provided no benefit to the client or its Millennium support team.

So what should you set your LogLevels to?

Monitoring servers — such as 27, 29, 33, 66, 76, 97 and 98 — should be at least a LogLevel 2/AUDIT. The Audit level provides a great deal of useful information when you are working through issues or looking for problems before they actually become outages.

I recommend a LogLevel of 1/WARN for the servers listed below. As a general rule, if your executable is not listed, I recommend you set the LogLevel to 2/AUDIT. You might find you need to make some specific tuning or tweaking to these settings, but they provide a great starting point.

SCP Entry ID     SCP Description

                 72      CPM Notify Server
                 73      Time Repository Server
                 75      User Services
                 80      CPM Script Cache
                 86      CPM Expedite
                 92      OMF PowerVision Document Server
               127      System Print Server (Node Specific)
               130      PathNet Resource Routing Server
               131      SCS Netting Server
               132      SCS Collections Server
               133      PathNet Result Order Posting Server
               134      PathNet Sync Result Order Posting Server
               134      PathNet Synchronous Result Order Posting
               140      Pathnet - AP Case Retrieval Server
               141      Pathnet - AP Automatic Diagnostic Coder
               144      Pathnet - AP Coding Cache Manager
               205     Clinical Event Server For ESI
               209     Clinical Event Server - Batch
               221      Search Server
               231      MDI Result Server
               235     MDI Order Server
               237     BMDI Result Server
               239     DI Inbound Client
               252     FSI ESO-CQM Server
               275     FSI ESO Notify Server
               280     FSI ESO Server Load #1
               281     FSI ESO Server Load #2
               282     FSI ESO Server Load #3
               283     FSI ESO Server Load #4
               284     FSI ESO Server Load #5
               285     FSI ESO Server Batch #1
               286     FSI ESO Server Batch #2
               290     FSI ESO Query Extract Server
               291     FSI Batch CQM Server
               335     PM Trans Async
               340     PM Query
               369     Chart Coding Services
               375     KIA Event Server
               425     FRN Tracking View Server
               455     Health Expectation Server

Prognosis: Keeping your LogLevels properly set will ensure that your IT team is notified of the issues it needs to address without being overwhelmed with meaningless messages.