Updated: Sep 21, 2018
by David Hallberg
Last week our series on Millennium® performance issues for RHOs (remote-hosted organizations) walked you through the necessary steps for evaluating if you are queuing in Millennium's Shared Service Proxy (SSP). This week we will cover another area where queuing can be a problem — in Millennium’s OEN/FSI/ESO servers. This has several aliases: OEN is short for Open Engine, FSI is an abbreviation for Foreign System Interface, and ESO is External System, Outbound. They provide the throughput for transactions to and from the other clinical systems that clinicians and office staff rely on to deliver quality patient care.
For example, a hospital’s registration system is usually not Millennium, but might be Epic, McKesson, or Centricity (IDX). After a patient is registered into that system, a set of processes sends that information to Millennium, which then saves and processes the data. On the outbound side, Millennium might send charges to the registration system either in batch or as they occur. If done through the ESO servers, the charges are created in Millennium, converted by a set of server processes into an HL7 message, and sent to the registration system.
Let me back up and explain the EON/FSI/ESO process and why it’s important.
Open Engine is a generic term for all of the servers (executables), middleware, and Oracle tables required to receive data from a non-Millennium system and send Millennium data to a non-Millennium system. I will be using OEN as a generic term for both inbound and outbound Millennium components, unless I specifically call out the differences.
OEN is important since so much information needs to be exchanged with other systems or computers. Without it, all of the people needing to see information held within Millennium would have to have Millennium access and use Microsoft’s copy-and-paste functionality to get information into their system. This process would be very slow and error-prone.
Generically, the figures below show the basic flow of information into and out of Millennium with OEN.
Figure 1 — Inbound Transactions
Figure 2 — Outbound Transactions
Starting at the left of the inbound transaction in Figure 1, we have a Foreign System, which means any computer system that is not part of the current Millennium domain. This system sends data into OEN, where the OEN processes put a row in the CQM_OENINTERFACE_TR_1 table to show the process status of the transaction. Another process takes that transaction and updates Millennium with the information.
The center of the outbound transaction in Figure 2 shows two tables that have the process status information of a transaction. The data first has to qualify as an outbound transaction. Then the CQM_FSIESO_TR_1 table is updated with a status for the transaction, and the transaction is put in the CQM_OENINTERFACE_TR_1 table with all of the formatting that the Foreign System needs to process it into its system.
So how do I find out if I’m having a queuing issue with OEN?
If you have Softek Panther, you can use the Interfaces control and view queuing in the Depth column. Without Panther, any Citrix or fat client should have the Millennium utility called Openview, or if you prefer seeing data flow across your screen, you can use a tool called oen_query_client(qc) and use the -mb -t15 switches to watch transactions. Both of these monitors look at the CQM_OENINTERFACE_TR_1 table. To monitor the CQM_FSIESO_TR_1 table for queuing, use the Interfaces control (if you have Panther) or run CCL commands or SQL scripts. For an explanation of how to use these utilities, please refer to the appropriate Millennium documentation.
What do I do if I’m having queuing issues?
If you discover OEN backlog issues, your first step is to verify the settings of the backlogged OEN process in the Oracle Oen_personality table. If you right-click on the Process Name in the Panther Interfaces control and select Properties and Interface Properties, you can view the Oen_personality data for that OEN server. There are a couple of key values you want to check:
trace_level should be 0
sleep should be 10 or higher
M1 should be DM
lowleveltrace and logtcpip_ack both should be 0
You can change any of these settings through CCL's OEN_PROC_VIEW menu. You want to verify in the Panther Configurations control the appropriate settings in SCP for the servers. Right-click on the SCP Configuration name (Description) or the Entry ID, select Properties, and then verify that there is no Startup Script and that loglevel is set to 0.
Prognosis: Monitoring OEN queuing and verifying the appropriate configuration of servers (executables) will reduce the time it takes for Millennium to accept and send out transactions to other systems supporting your organization.
Next week: Oracle configuration and where to look to see if you are having throughput issues.