Multicenter International Work Flow Study of an Automated Polymerase Chain Reaction Instrument

¹ Clinical Virology, 3rd Floor, Clinical Sciences Building, Manchester Royal Infirmary, Oxford Road, Manchester, M13 9WL, UK;
² Department of Pathology, Thomas Jefferson University Hospital, Philadelphia, PA 19107;
³ Gemeinenschaftslabor City Drs Fenner & Partner–Dept. Molecular Diagnostics, Bergstrasse 14, D-20095 Hamburg, Germany;
⁴ Laboratorio di Patologia Clinica, Ospedale S. Anna, Via Napoleona 60, I-22100 Como, Italy;
⁵ Stichting Streeklaboratorium voor de Volksgezondheid voor Groningen en Drenthe, van Ketwich Verschuurlaan 92, 9721 SW Groningen, The Netherlands;
⁶ Roche Molecular Systems, Inc., Branchburg, NJ 08876-3771;
^a author for correspondence: fax 44-161-276-8840, e-mail pklapper{at}mri5.cmht.nwest.nhs.uk

PCR has the potential of extreme sensitivity, specificity, and diversity. As a consequence, transition of this procedure from research to routine application has been relatively rapid in recent years. This transition has, however, been hampered by the labor-intensive, high-skill requirements of existing protocols. Furthermore, as a number of quality-assessment exercises have demonstrated (1)(2)(3)(4)(5)(6)(7)(8), many of the existing "in-house" developed test procedures are of poor reproducibility in routine application. The introduction of a commercially produced test procedure has improved the reliability of testing, but the procedure has remained a relatively high-skill test method with only certain steps becoming semiautomated.

The COBAS AMPLICOR^TM automated PCR system (Roche Molecular Systems) represents the first approach to full automation of the amplification and detection of nucleic acid targets (9). Evaluations of this instrument (10)(11) suggest that the sensitivity and specificity of the automated method are equivalent to those of manual methods, that intraassay sample carryover does not occur (9)(10), and that intra- and interassay variation using the automated system is low (9). However, there remains a need to evaluate the ability of the instrument, which is limited to 24 samples per thermal cycle amplification and 48 samples for the detection phase, to cope with both the work load and test diversity of a routine diagnostic laboratory. Moreover, the ability of the instrument to decrease the high labor and skill requirements of PCR and to decrease the overall cost of PCR needed to be further evaluated. We conducted a multicenter, international evaluation of the instrument from the standpoint of automation, work flow, and labor savings in laboratories differing in size, diversity of testing, and laboratory working practices.

Five laboratories, chosen to represent a wide range of working hours and practices, were studied in Italy, Germany, The Netherlands, England, and the United States. Each laboratory used the COBAS AMPLICOR instrument and test kits according to the manufacturer's instructions. Operators received on-site training by the manufacturer's representative, according to a standardized procedure. Other than these steps, there was no attempt to standardize test procedures at each site. There were minor site-to-site variations in ancillary equipment and in work flow methods. Four laboratories used single instruments; one laboratory used two instruments. The studies were designed to maximize the number of results produced within a single day shift of ~8 h. Sample batch sizes ranged from 12 to 140 for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG), from 12 to 60 samples for Mycobacterium tuberculosis (Mtb), and from 12 to 48 for hepatitis C virus (HCV). Typically, at least two of the samples in each batch were the positive and negative controls that came with the kit. Each patient sample also had an internal amplification control, assayed as an extra analyte.

The individual steps of each assay were grouped in accordance with the time-based methods of workload evaluation and recording developed by the College of Canadian Pathologists and later adapted for use by the College of American Pathologists (12) and WELCAN-UK (13). Times related to processes such as instrument start-up or shutdown each day were prorated over the individual samples in the various test batches run that day. However, the daily test run sizes were chosen to reflect a variety of typical scenarios. The "hands-off" times were also calculated. The sum of the "hands-on" and hands-off times gave the total time from initiation of testing to production of the final test result. The time for the entire work flow process, including preliminary instrument-related steps, specimen-preparation steps, and all other hands-on and hands-off times, were calculated.

In spite of the slight differences in work habits at each site, the overall hands-on times and times to completion of results were very similar at each site. The average time taken for instrument start-up was 38.7 min. Once the instrument was running, between-run set-up (i.e., adding of reagents and other consumables) could be achieved more rapidly (average, 11.4 min to restart). The manual AMPLICOR assay and the automated COBAS AMPLICOR methods required similar amounts of hands-on sample and reagent preparation time before commencement of the assay (data not shown), the major difference being that once the automated assay begins, the technologist is free to move to other work. With the manual procedure, the technologist must return to repeat multistep procedures for product detection, leaving less time free for other laboratory tasks.

The average hands-on time required for the performance of the various automated assays was considerably less than that required for typical in-house PCR tests. Because the major costs within a clinical laboratory relate to staff cost, reductions in hands-on time, which release technical staff for other laboratory work, typically reduce cost per test. Thus the initially high unit cost per test of automated PCR is tempered by its relatively low hands-on time. Particularly striking was the difference in time to produce first results by automated PCR compared with conventional diagnostic procedures. For example, first results of Mtb analyses are obtained 3 h and 45 min after commencement of the automated PCR assay. This may be compared with the 1–2 weeks required when the fastest radiometric method and gene probe technology for Mtb detection are used.

The majority of the hands-on time is for specimen preparation, with the exact percentage varying by analyte. The time required for preparation of a medium-to-large batch of samples was related to the sample type. The longest sample preparation was for serum (HCV, 100–130 min) and the shortest for swabs (CT/NG, 28–40 min; Table 1 ). The hands-on times to access and prepare samples, complete testing, and report results averaged 3.3 min per sample for the multiplex CT/NG, plus internal control tests; 4.4 min for Mtb; and 7.7 min for HCV. The internal control results were obtained with no additional increase in the workload value because of the integrated automated detection process of the system. Although each of the participating laboratories exhibited differences in approach to testing, the range of total hands-on times when testing 22 samples plus 2 controls determined for each of the tests and across the various centers agreed reasonably well (Table 1 ).

The instrument is sufficiently flexible to cope with a wide variety of different workloads and workload mixtures. One option, reflex testing, allows a sample to be tested for one analyte (e.g., CT) and, if negative results are obtained, sequentially for a second analyte (e.g., NG) and then for amplification of an internal control molecule. A more time-efficient method is to analyze all analytes simultaneously. By switching sample tubes from the amplification to detection positions as soon as amplification of one set of 24 samples is completed, throughput is improved. However, by performing assay and machine preparation steps on the preceding day, it is possible to achieve throughput of 96 samples (patients and controls) and still restrict actual hands-on working operations to the 8-h day (Fig. 1 ). The most time-efficient use of the instrument was achieved when two technologists were available at the beginning of the working day to share the tasks of instrument and sample preparation.

View larger version (23K): [in this window] [in a new window]   Figure 1. Schematic of sample throughput. By performing sample extraction and reagent and machine preparation on the preceding day and allowing detection to run unattended, sample throughput can rise to 94 patient samples plus 2 control samples, representing 284 detections of CT, NG, or IC. S1–S4, sample preparation periods; P, time for preparation of the instrument; A1–A4, amplification periods; D1–D4, detection cycles.

In diagnostic microbiology laboratories, nucleic acid amplification technologies such as PCR are, in many instances, anticipated to supplant culture for an ever-expanding proportion of infectious diseases. PCR will also have a major impact in the management of disease, as illustrated by its current application in the monitoring of circulating viral "load" (14) and in detecting genotypes resistant or refractory to antiviral chemotherapy (15)(16). For these new molecular diagnostics, there is little information regarding standardized workload measurements of hands-on time for the various methods and the impact of automation on the efficiency of sample processing. Workload statistics are particularly useful in evaluating the change in personnel hours required to add new tests, operate new instrumentation, or to switch to automation. The value of statistics in managing the introduction of nucleic acid amplification technologies and their transition to automation is clear. This international study showed that workload measurements for PCR are practicable and can be used to clearly demonstrate that automated PCR reduces the labor cost component of nucleic acid amplification to a level that is comparable with most immunological assays and lower than most culture-based assays.

This first generation instrument that provides automation of amplification and detection of PCR was successful in allowing laboratories and laboratory personnel who had never performed PCR to achieve remarkable productivity after minimal training. The simplification of the PCR protocol greatly aids the introduction of this technology into routine laboratories. The COBAS instrument, which at first sight seems limited in capacity through the use of only 24 samples per thermal cycle run, can in practice cope with very high workloads and can offer a high degree of test diversity for routine diagnostic laboratories. On average 60–84% of the hands-on time for an assay is related to specimen preparation, with the remainder being related to steps involving the COBAS instrument. Automation of specimen preparation would clearly be of major benefit and allow additional savings in time to be achieved.

Acknowledgments

We acknowledge the assistance of Sabine Philippi-Schluz and Hans Burkardt, who together with one of us (C.B.) performed the workload analysis timing in Europe. Roche Molecular Systems Ltd., Branchburg, NJ, kindly provided all test reagents for use in the study. We thank the staffs of the respective laboratories for their diligent and patient work throughout this study.

References

1. Damen M, Cuypers HTM, Zaaijer HL, Reesink HW, Schassberg WP, Gerlich WH, et al. International collaborative study on the second EUROHEP HCV-RNA reference panel. J Virol Methods 1996;58:175-185. [Web of Science][Medline] [Order article via Infotrieve]
2. Jackson JB, Drew JJ, Lin HJ, Otto P, Bremer JW, Hollinger FB, Wolinsky SM. Establishment of a quality assurance program for human immunodeficiency virus type 1 DNA polymerase chain reaction assays by the AIDS Clinical Trials Group, ACTG PCR Working Group, and the ACTG PCR Virology Laboratories. J Clin Microbiol 1993;31:3123-3128. [Abstract/Free Full Text]
3. . for the standardisation of hepatitis C virus PCR. French study group. Improvement of hepatitis C virus RNA polymerase chain reaction through a multicenter quality control study. J Virol Methods 1993;49:79-88.
4. Lina B, Pozzetto B, Andreoletti L, Beguier E, Bourlet T, Dussaix E, et al. Multicenter evaluation of a commercially available PCR assay for diagnosing enterovirus infection in a panel of cerebrospinal fluid specimens. J Clin Microbiol 1996;34:3002-3006. [Abstract/Free Full Text]
5. Quint WG, Heijtink RA, Schirm J, Gerlich WH, Niesters HG. Reliability of methods for hepatitis B virus DNA detection. J Clin Microbiol 1995;33:225-228. [Abstract/Free Full Text]
6. Vandamme AM, Fransen K, Debaisieux L, Marissens D, Sprecher S, Vaira D, et al. Standardisation of primers and an algorithm for HIV-1 diagnostic PCR evaluated in patients harbouring strains of diverse geographical origin. The Belgian AIDS Reference Laboratories. J Virol Methods 1995;51:305-316. [Web of Science][Medline] [Order article via Infotrieve]
7. Zaaijer HL, Cuypers HTM, Reesink HW, Winkel LN, Gerken G, Lelie PN. Reliability of polymerase chain reaction for detection of hepatitis C virus. Lancet 1993;341:722-724. [Web of Science][Medline] [Order article via Infotrieve]
8. Zeuzem S, Ruster B, Roth WK. Clinical evaluation of a new polymerase chain reaction assay (Amplicor HCV) for detection of hepatitis C virus. Z Gastroenterol 1994;32:342-347. [Web of Science][Medline] [Order article via Infotrieve]
9. Di Domenico N, Link H, Knobel R, Caratsch T, Weschler W, Loewy ZG, Rosenstraus M. COBAS AMPLICOR^TM: fully automated RNA and DNA amplification and detection system for routine diagnostic PCR. Clin Chem 1996;42:1915-1923. [Abstract/Free Full Text]
10. Jungkind D, DiRenzo S, Beavis KG, Silverman NS. Evaluation of automated COBAS AMPLICOR^TM PCR system for detection of several infectious agents and its impact on laboratory management. J Clin Microbiol 1996;34:2778-2783. [Abstract/Free Full Text]
11. Pasternak R, Vuorinen P, Pitkajarvi T, Koskela M, Miettinen A. Comparison of manual Amplicor PCR, COBAS AMPLICOR^TM PCR and LCx assays for detection of Chlamydia trachomatis infection in women by using urine specimens. J Clin Microbiol 1997;35:402-405. [Abstract/Free Full Text]
12. . College of American Pathologists. Workload recording method and personnel management manual, 1992 ed 1992:144pp College of American Pathologists Northfield, IL. .
13. WELCAN UK. WELCAN UK workload measurement system for pathology, manual with schedule of unit values; 1992 edition and 1993 supplement 1993:125pp HMSO London. .
14. O'Brien WA, Hartigan PM, Martin D, Esinhart J, Hill A, Benoit S, et al. Changes in plasma HIV-1 RNA and CD4+ lymphocyte counts and the risk of progression to AIDS. N Engl J Med 1996;334:426-431. [Abstract/Free Full Text]
15. Stuyver L, Wyseur A, . Van arnhem W,Hernandez F, Maertens G. 2nd-generation line probe assay for hepatitis-C virus genotyping. J Clin Microbiol 1996;34:2259-2266. [Abstract/Free Full Text]
16. Japour AJ, Welles S, Daquila RT, Johnson VA, Richman DD, Coombs RW, et al. Prevalence and clinical-significance of zidovudine resistance mutations in human-immunodeficiency-virus isolated from patients after long-term zidovudine treatment. J Infect Dis 1995;171:1172-1179. [Web of Science][Medline] [Order article via Infotrieve]