Past Standards Still Haunt Us

A client visit earlier this week brought my attention to tuning standards established years ago for AIX nodes that are still being circulated. Like urban legends, the standards no longer have evidence to support them, but they live on. October seems like a good time to debunk these phantom beliefs, so I’ll be putting my next installment on issues facing RHO clients on hold for a bit longer.

The legend I’ll address this week claims that you need to have AIX disk I/O pacing enabled to make Oracle run faster.

When Millennium® was first created, AIX (IBM’s Unix) was only 32-bit, and Millennium was deployed with the application node and the database node existing on the same node or lpar. Back then, like today, the various AIX nodes had to be able to share resources for failover. For instance, if you had two Millennium application nodes running WebsphereMQ (MQ) and one node crashed, you needed them to share the log and data file systems for MQ. Why? If node 1 crashed, the other node would be able to complete its transactions.

Way back in the 1990s, Serial Storage Architecture (SSA) drives and HACMP (High Availability Cluster Multiprocessing) were used to share data between nodes. One artifact of this technology was that AIX could send more disk I/O messages to the shared SSA drives than the drives could handle. To minimize this risk, the minpout and maxpout throttles on disk I/O were used to control the disk I/O rate. Maxpout set a limit on the number of pending I/Os (typically 33) for any logical volume (LV) or file. The minpout was set lower (typically from 20-24) and would allow the drive to read and write to the file again once the number of messages fell back to that level.

This disk I/O pacing was a workable solution for SSA drives running on a 32-bit system, but it does not translate to today’s AIX client using storage area networks (SANs) on a 64-bit system. The pacing creates an artificial bottleneck and prevents you from getting the most out of your disk subsystem. It should be disabled.

Unfortunately, the disk I/O pacing caused so many issues when it was not set, the myth persists that you need to keep it set. If you are using SAN storage — as most of you are — this simply is not true and never has been true. SAN fabrics have other pacing mechanisms you should use instead of the AIX OS-level disk pacing to achieve optimal throughput. To disable the disk I/O pacing, you can run the following two commands as root on all of your AIX nodes (please note that these changes take effect immediately):

chdev -l sys0 -a minpout=0
chdev -l sys0 -a maxpout=0

Prognosis: To advance toward optimal throughput, discard the outdated tuning standards of AIX disk I/O pacing in favor of SAN pacing mechanisms.

Next week: Illogical tuning of the logical volumes.