| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Technical Briefs |
Department of Pathology, John H. Stroger, Jr. Hospital of Cook County, Chicago, IL
aaddress correspondence to this author at: Department of Pathology, John H. Stroger, Jr. Hospital of Cook County, 1901 West Harrison St., Chicago, IL 60612; fax 312-864-9862, e-mail ntorke{at}gmail.com
Through autoverification, a customized expert system within the Laboratory Information System (LIS), a computer performs the initial review and verification of test results based on a predetermined set of boundaries or rules, as established by the laboratory (1). A carefully designed system can be an important tool in addressing such crucial issues as medical errors, test turnaround time (TAT), shortages in personnel and funding, and operational efficiency. As part of an ongoing modular laboratory automation plan at John H. Stroger, Jr. Hospital of Cook County (JHSHCC, formerly Cook County Hospital, Chicago), autoverification was implemented in our clinical chemistry and urinalysis laboratories.
JHSHCC is a large, urban, tertiary acute-care public hospital and trauma center that serves as the centerpiece of a countywide integrated healthcare delivery system. As part of the hospital’s clinical pathology service, the clinical chemistry laboratory provides more than 150 different test procedures in general chemistry, toxicology, drug screening, endocrinology, urinalysis, and blood gas analysis. The laboratory has modern, automated instrumentation, including preanalytical specimen processors (TECAN FE500), all of which are bidirectionally interfaced to the LIS (Soft Computer Consultants). In 2003, the clinical chemistry laboratory processed 1.3 million specimens (more than 6 million reportable results). The result-review/reporting process for the above workload required 14 full-time equivalents (FTE) of technical personnel. Autoverification was expected to reduce the FTEs necessary for this process.
Autoverification setup and implementation required 2 FTEs in personnel over a period of 6 weeks. Automated chemistry assays were chosen for the initial autoverification attempt for several reasons, including high test volume, clearly defined reference ranges and delta checks, ability to define intermediate ranges for abnormal results, and the flagging capabilities of various automated analyzers. Urinalysis was included for the autoverification setup based on the availability of interfaced automated analyzers (IRIS 939 Udx) with which decision rules could be implemented. Expert user technology, essential for the autoverification setup, was available in our LIS; therefore, no additional process control software (such as "middleware") was necessary. The flagging systems internal to the individual analyzers were used extensively in the development of the autoverification algorithms in the LIS. The decision rules were built in the LIS rather than in the instruments to avoid duplication of setup with each analyzer.
In our patient mix, 
16% of the chemistry test panels have all component results reported as "normal" (i.e., within the reference range). Because a test panel can be autoverified only if all component tests pass the rules, we opted to widen our ranges for acceptability sufficiently beyond reference ranges to obtain a greater yield without compromising the integrity of the process or the quality of patient care. To achieve this goal, a near-midpoint between the median reference range value and the analyzer’s linear analytical measurement limit was used to set the decision point for ending autoverification. For those analytes for which critical values were defined, the linear measurement limit was replaced with a critical value limit to determine when autoverification should not be performed. Table 1
further illustrates the autoverification cutoff logic described above. Decision rules specific to complex test interactions and test panels (e.g., blood urea nitrogen/creatinine ratio and liver function tests) were also made part of the embedded rules. Results qualifying for manual review have included critical values, grossly abnormal results, and those with specific instrument-generated warning flags. In addition, for those tests for which previous results were available in the LIS, a failed delta was used to prevent autoverification. Our delta-check thresholds are customized by analyte. For the majority of analytes, however, current results passing the various rules and showing a delta <20% are autoverified. In our laboratory, 3 concentrations of quality-control materials (low, normal, and high) are assayed at the beginning of each shift and reviewed with modified Westgard rules built into the LIS. Quality-control rule failures also prevented autoverification.
|
We analyzed the impact of autoverification on the process TAT ("specimen received" time to "result" verified/released time; includes automated accessioning, centrifugation, aliquoting, and other steps) by comparing statistical data on 4 high-volume tests that were available 24 h/day and orderable as STAT. For this purpose, relevant TAT data, compiled over a period of 6 months immediately before implementation, were compared with those obtained over 6 months after implementation. Likewise, personnel savings attributable to autoverification were also determined by comparing staffing patterns over 6 months before and after implementation. Result-review errors were compiled by studying LIS-based audit trial-and-error logs over a representative period of 30 days. The consistency in the application of the result-review guidelines was examined and verified by reviewing all of the results generated by the laboratory over 1 week before and after implementation. In addition, the accuracy of the transmission of test results from the various analyzers to the LIS was monitored by comparing the results from the instrument printout to the corresponding results in the LIS according to the following protocol: 10 specimens/week during the first 4 weeks; 10 specimens/month during the next 5 months; 10 specimens/quarter thereafter. Overall, the consistency in the application of the result-review guidelines was significantly increased through the automated verification process.
In our system, nonautoverified results are placed in a review queue for a careful manual review. Currently, 62% of our chemistry test panels, 73% of single-analyte assays, and 43% of the urinalysis results are autoverified. A greater number of urinalysis results required manual review because our rules for urinalysis included correlation of microscopic sediment findings (such as casts, erythrocytes, or leukocytes) with macroscopic results (e.g., presence of protein, occult blood, or leukocyte esterase). During the month before implementation, a daily average of 9 chemistry results and 12 urinalysis results showed human review errors of various degrees, for a combined error rate of 0.06%. In comparison, during the month immediately after introduction of autoverification, an average of 1 chemistry result and 2 urinalysis results (0.009%) were found to be either inappropriately autoverified or prevented from being autoverified. In each of these cases, exclusionary setup rules were found to be the cause of the errors, and modification of the rules had solved the problem. Consequently, review errors have been nearly eliminated. The qualitative, nonnumeric results were verified in accordance with the built-in guidelines.
Before implementation of autoverification, 14 FTEs were required for result-review–related functions (8 for chemistry and 6 for urinalysis). Six months after implementation, the laboratory had assigned 8.5 FTEs for performance of these duties (4.5 for chemistry and 4 for urinalysis). Thus, autoverification has led to personnel savings of 5.5 FTEs, or a reduction of 40% in personnel assigned to result-review function. The impact was greater for chemistry panels, with straightforward review requirements, than for urinalysis results, which required microscopic sediment analysis on several abnormal results. Process TAT for STAT chemistries was reduced by 10 min (48 min before vs 38 min after autoverification) and for STAT urinalysis by 8 min (47 min vs 39 min). The TAT outcomes for 4 high-volume STAT tests (Basic Metabolic Panel, Cardiac Panel, Toxicology Screen, and Urinalysis) are summarized in Fig. 1
. These outcomes represent a combined TAT reduction of 19% for all STAT tests. Interestingly, the greatest impact was on the process TAT for routine orders. The TAT for routine chemistries was reduced by 36 min (151 min vs 115 min) and the TAT for routine urinalysis by 27 min (138 min vs 111 min). With a properly configured preanalytical processor (TECAN FE500), we addressed the problem issues, such as mislabeling, hemolysis, icterus, and fibrin clots, which autoverification will not recognize (2)(3).
|
No regulatory guidelines exist for autoverification program auditing beyond the need for annual testing (4). Our audit process, which is performed twice per year, tests all rules and includes comparison of at least 20 sets of patient results outside of the reference range in the LIS, with instrument printouts to determine whether the autoverification criteria were applied properly. Failure necessitates immediate review of the setup. In our system, failures identified early in the process represented deficiencies in the design of the rules and were corrected immediately. Otherwise, rules were applied consistently and as intended.
The implementation of autoverification has had several unexpected positive impacts on laboratory operations. After implementation, the combined TAT for all routine orders was reduced by 22% (142 min vs 112 min) because delays attributable to manual result review were eliminated. The laboratory’s annualized total workload has increased by 4%, whereas STAT orders have gradually decreased since the implementation of autoverification. For example, during the period of October to December 2004, overall STAT orders decreased 16% compared with the same period in 2003 (272 vs 301). We believe this decrease occurred because results for routine orders were available sooner and STAT orders were therefore unnecessary.
Autoverification is relatively new to laboratory testing; therefore, scientific literature on the subject is limited. There is consensus among published reports, however, that a properly configured autoverification process leads to the application of consistent criteria to the review of every set of results, thus eliminating the variability associated with manual review (5)(6)(7). Studies have also suggested that the use of this process can increase productivity and reduce overall error rates (8)(9)(10). Our results are consistent with these reported findings.
References
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
