RHO Remedies 2—High MQ Queue Depths = Slow Transactions

Use one of the following programs to determine if your Millennium system is queuing.

Panther Queues Control

The Queues control in Softek Panther shows all MQ queues on all nodes in a domain, allowing you to easily see where you have backlogs in MQ queues across your entire environment. You can click on the Depth column header to sort by that column, and you can filter out MQ queues that don’t have a backlog so you only see queues that are experiencing a backlog (indicated by a queue depth greater than 0). The Exception queue is handled in a window below the larger Queues window, since this is for transactions that have failed to completely process for some reason.

Panther also provides several sensors that can be configured to send email or pager notifications when user-defined queue depths are exceeded or when no forward progress is made on messages in a queue. These sensors alleviate the need to have someone constantly monitoring the MQ queues and allow your IT staff to more quickly respond to these issues.

Qcpview.exe

Using qcpview.exe to find out if you are queuing:

  1. Start a DOS or CMD prompt on the Citrix session or fat client.
  2. From the winintel directory, type in qcpview <node name="">, login and then type in the command 'list'.</node>
  3. Of the five columns displayed, the CQD, or current queue depth, is the one we are concerned with. Look for CQD values that are greater than 0, which indicates that you have a queue or backlog on that queue. An exception to this is the EXCEPTION queue, where transactions go that cannot be processed due to system resources or transaction issues (such as a 6MB transaction trying to be sent into a 4MB message-sized queue).
  4. When you are on Millennium 2007.x code, you can use the “list -ss” command to show the request queues, the “list -ss –rep” command to show the reply queues, or the “list -ss –both” command to show both request and reply queues. The only caveat with this method is that you have to scroll back up the display to see the CQD value.

Quemon.exe

Quemon.exe is an improvement over qcpview.exe when you are looking for MQ queues backing up. It was designed to be a monitoring tool, where qcpview.exe is a management tool. To launch quemon.exe:

  1. Start a DOS or CMD prompt on the Citrix session or fat client.
  2. Access quemon.exe from the winintel directory by using some switches at the end of the command. Generally I use quemon <node name=""> -mode queue -sort max. A more specific example might look like this: quemon hnaa -mode queue -sort max.</node>
  3. Log into the domain you want to look at.
  4. With quemon.exe, the Cur Depth column indicates where you are backing up. You should see a screen similar to this:

The example above shows 10 transactions in the EXCEPTION queue, indicating the transactions failed to update fully. You would now want your technical support team to investigate how to resolve these errors. The CPMSCRIPTASYNC messages are nurses and physicians interacting with Millennium. A Cur Depth of 100 indicates 100 clinicians are waiting on information they submitted to show up in their Millennium applications, the patients’ charts. This delay has several downstream impacts, including impatient clinicians hitting the Refresh button and putting more work on the system instead of getting access to the updated information they need.

Both the Millennium qcpview.exe and quemon.exe tools can only view one node at a time. Sites with two or more Application nodes require a separate DOS window running for each node. This can become problematic for a small IT staff where the staff is generally doing multiple things at once.