I have been working with Millennium® in various capacities since November 1997. Year after year I hear the following statement, and each time it stuns me: “We will always have a backlog. There is nothing we can do to prevent queuing.” I have heard these words from the engineering team lead up to the vice president on the vendor side and from the first-level IT person through the CIO on the client side — including two clients in the past few weeks. I keep shaking my head when I replay the conversations.
So let me be very, very clear: There is no baseline with Millennium that includes transaction backlogs or queues. You should never tell the clinicians or business staff using Millennium — or yourself, for that matter — that “waiting is normal” or “well, that’s just the way the system is.” If queuing truly is the norm for your system, it’s telling you it needs a serious tune-up. I have gathered and analyzed performance data from more than 100 Millennium clients, and I can assure you that I always find ways to reduce a user’s waiting time.
I understand that getting to some of the performance information needed to make the right tuning decisions can be difficult. It’s easiest with the right monitoring tools (such as Softek Panther), but with a bit more effort, you also can use Millennium’s mon_ss and quemon along with Oracle, Citrix and network monitoring tools to uncover the data.
Every layer of the n-tier cient/server architecture was built to be tuned, which leads many people to conclude that tuning is too complex and yields too little benefit — hence their belief that queuing is inevitable. They are right on one point: Tuning can be complex. With the mix of Java, Visual C, Visual Basic, Dcom, ActiveX, C++ and C code on each tier, you will not find a silver bullet. Changing a single setting on any of the tiers will rarely increase the speed of the entire system. Invariably, you have to follow a complete process of data gathering, data analysis, action-plan development, execution and post-change data gathering. Executing one change at a time because someone thinks it reduces risk or hopes it will be that silver bullet is counterproductive to your organization.
The only viable alternative is to tune all the areas that impact Millennium performance. I realize it’s a daunting task. Besides the code types I mentioned earlier, Millennium has many technologies as well — the client PC or thin station, file shares, Citrix or Terminal Server, the main application tier (where the SCP servers run), the database tier, Chart, RRD, Multum, WAS (Choose and Book, XR Reporting, Powerchart Outreach), Linux, Oracle with raw LVs, Oracle with regular filesystem for CAMM, MySql, SQLServer, VMS, AIX, HP-UX, GoldenGate, SonicESB (MDBus), backups, network, storage. And that’s not even everything. So how do you know what to do?
Start by gathering as much information as you can. Then you will know what you know and can begin to take action. Typically, you will at least have nmon for AIX and Linux, T4 for VMS, and Glance or other Unix tools/data for HP-UX. Ask yourself the following questions: Do I have an Ethernet port I am not using? Do I have SCSI cards I am not using? Do I have fiber channel cards I am not using? Do I have memory or CPUs on the lpar/vpar/npar that are not activated or simply not allocated? I’m always amazed when clients with performance issues claim, “There is nothing we can do,” yet they have unused hardware sitting on their data center floor.
If you are being told you have to live with backlogs, then get another opinion. If you believe you have tuned all you can and still are waiting, find someone who has a history of tuning large, complex systems for performance and throughput, and see if they agree. If you are being told you need to buy more hardware to fix your performance problems, get another opinion. The majority of tuning can be done without buying anything new. It only requires the proper sequence of keystrokes.
Prognosis: Once IT managers, CTOs, CIOs and CFOs stop embracing the common belief that queuing is an inevitable part of the Millennium environment, they can begin to address the tuning steps that will reduce user waiting times and improve the patient experience.
