In 1997, when I first began performing technical audits on Millennium® infrastructures, the biggest concern was to keep the Millennium middleware functional and to tune the Oracle kernel. My job was to make sure the interactive user transactions made it to the Millennium application nodes and the database was able to keep up with the workload. These days, however, the MQ generally is working well and the dependency on the Millennium middleware is greatly reduced, so there’s a lot less to tweak in those areas. Likewise, Oracle nodes/lpar/vpars have at least 64 GB of RAM and large system global areas (SGAs) and provide much higher throughput for the Millennium solutions.
Unfortunately, as these issues have improved over the past 12 years, other key problems have surfaced. One trend that I’ve found in the last two years is an increase in lost charges. Analysis of Oracle tables and Millennium log files show that clients are losing charges at two points: in the orders/activity to charge mapping and at inbound interfaces.
Most clients don’t believe me when I highlight their billing mistakes. I find they get rather defensive instead of eager for me to show them how to fix their problems, some of which can be done fairly easily.
If you want to investigate whether your system is losing charges — and your organization is losing revenue — start at the message logs for CS Charging Async and CS Charging Sync. You want to look for IncompleteEvent, AddBillCodeType, AddItem_Failed, CheckCharge, CheckChargeModForType, DBAPI_FatalScriptErr, OffsetCharges and ScriptError. These events represent the problems I have seen most recently. All but DBAPI_FatalScriptErr and ScriptError are build or configuration issues that typically need only a little research to resolve.
The DBAPI issues are often Oracle space or connectivity issues that should be resolved with your database administrator. If the DBAPI errors are constraint violations or lock issues, they are probably code issues that require you to take updated Millennium service packages. The ScriptError events are almost always code issues, also requiring you to take updated Millennium service packages.
Your investigation might show a staggering number of ESI errors and warnings. The ESI issues are often from the Omnicell or Pyxis interface, which could indicate you are losing pharmacy charges. Even if it’s only 1-2% of all pharmacy charges, your organization could be losing thousands of dollars in revenue.
From the ESI logs, the failures are often from a “null person_id.” That means Millennium received a transaction but it couldn’t apply the charge or event because the patient was not identified. Other failures could include a “null meddef” or “null manufactur id.” Both of these are build issues in which Millennium is receiving incomplete or mismatched information and therefore cannot update the database. ESI warnings often include aliasing issues, script issues, invalid catalog_cd, action_type_cd, order_id or synonym_id. The problem also could be as simple as missing data or an incorrect date/time format.
The ESI log typically contains only the last seven days of data. Failures, therefore, are purged and the information lost unless you are looking at these issues at least once a week.
All of these items are straightforward to correct — once you know about them. The key is to be willing to believe the data in your log files, use it to identify the failed charges and have a process in place to correct the problem.
Prognosis: Addressing the build issues associated with the charge errors in your message logs and ESI log will translate into increased revenue for your organization.
