Clinical Chemistry
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


Clinical Chemistry 47: 47-55, 2001;
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (19)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Saunders, G. C.
Right arrow Articles by Cornett, J. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Saunders, G. C.
Right arrow Articles by Cornett, J. H.
Related Collections
Right arrow Molecular Diagnostics and Genetics
Right arrow Automation and Analytical Techniques
(Clinical Chemistry. 2001;47:47-55.)
© 2001 American Association for Clinical Chemistry, Inc.


Articles

Interlaboratory Study on Thermal Cycler Performance in Controlled PCR and Random Amplified Polymorphic DNA Analyses

Ginny C. Saunders1, Juliet Dukes1, Helen C. Parkes1,a and Johanne H. Cornett1

1 Life Sciences Research, LGC Ltd., Queens Road, Teddington, Middlesex, TW11 0LY, United Kingdom.
a Author for correspondence. Fax 44-020-89432767; hcp{at}lgc.co.uk


   Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Background: Intercomparisons of PCR-based data between laboratories require an assurance of assay reproducibility. We performed an interlaboratory study to investigate the contribution made by a variety of thermal cyclers to PCR performance as measured by interblock reproducibility and intrablock repeatability.

Methods: Two standardized assays designed to minimize the introduction of non-thermal-cycler-dependent variations were evaluated by 18 laboratories in the United Kingdom, using 33 thermal cyclers of various makes and models. We used a single-product (590 bp) PCR, established in our laboratory as a robust and specific reaction. The second reaction, a multiproduct random amplified polymorphic DNA (RAPD) PCR, was known to be more susceptible to small changes in block temperature and was therefore considered a way of assessing block uniformity with respect to temperature. Assay repeatability data were analyzed with respect to temperature calibration status, the type of temperature control mechanism, thermal cycler age, and the presence of oil overlay or heated lid systems.

Results: All (100%) of the laboratories produced the correct target for the single-product PCR assay, although substantial variation in yield in replicate reactions was observed in 9.4% of these. The RAPD reaction generated results that varied extensively both within the same block and between different thermal cyclers. For eight replicates of a positive sample, 88% intrablock repeatability was demonstrated in calibrated thermal cyclers, which decreased to 63% in noncalibrated instruments.

Conclusions: Irrespective of the make and model of thermal cycler, temperature-calibrated instruments consistently generated more repeatable RAPD data than noncalibrated instruments. Guidelines are offered on optimizing and monitoring thermal cycler performance.


   Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
PCR, an in vitro nucleic acid amplification technique, is a key procedure that underpins the analysis of genetic material in sectoral applications as diverse as clinical diagnostics; forensic DNA analysis; detection of human, animal, plant, and food pathogens; and molecular biology research. The valid use of this technique is therefore critical to maintain the quality of much of the DNA-based data produced. It is a well-established fact that many different factors can influence the validity of results obtained by PCR, including the presence of inhibitory components, the quality of the DNA template, and the lack of optimization with respect to reaction components and thermal cycling conditions. All of these factors can compromise the reaction specificity and sensitivity, giving rise to false-negative, false-positive, or nonreproducible results (1). Perhaps less widely acknowledged, however, is variation in PCR results attributable to suboptimal thermal cycler performance. It is this particular aspect of the technique that forms the basis of this interlaboratory study.

Thermal cyclers are the programmable heating blocks that control and maintain the temperature of the sample through the three temperature-dependent stages that constitute a single cycle of PCR: template denaturation (~95 °C); primer annealing (~35–65 °C); and primer extension (72 °C). These temperatures are cycled up to 40 times to obtain amplification of the DNA target. If the optimum programmed temperatures are not met because the block overshoots or undershoots them or because they are not consistent across the block, reaction specificity and sensitivity can be compromised, leading to the consequences listed above and ultimately to the false interpretation and reporting of data.

The aim of this study, therefore, was to investigate the effects of thermal cycler performance on PCR reproducibility and repeatability, using two standardized assays designed to minimize the introduction of non-thermal-cycler-dependent variations. The first was a single-product (590 bp) PCR, using primers designed against multicopy ribosomal genes (2). This was established in our laboratory as a robust and specific reaction that consistently produced the correct product when performed under a broad range of conditions. The second test assay was a multiproduct random amplified polymorphic DNA (RAPD)1 reaction. This was relatively nonspecific because of the short 10-base primer and low annealing temperature used, and was shown, in our hands, to be sensitive to slight variations in both annealing temperature and temperature ramp rate (specifically the temperature transition rate from annealing to extension). These characteristics of the RAPD reaction rendered it particularly suitable to assess well-to-well uniformity across the block of a thermal cycler (intrablock repeatability) and to highlight block-to-block variations between different thermal cyclers (interblock reproducibility). The temperature calibration status of the thermal cyclers was also established, and its significance was investigated. For the purposes of this study, thermal cycler temperature calibration was defined as the uniformity of the block temperature with an accuracy ± 1 °C of the programmed temperature across the entire block, as assessed by a recognized calibration service in the year before this study.


   Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
The PCR assays were performed as described below. All reaction volumes were 100 µL, and each assay consisted of a total of eight replicate positive and two duplicate negative reactions. All thermal cycler ramp rates were set to maximum.

single-product pcr
The single-product PCR reaction mixtures consisted of 1x PCR buffer (50 mM KCl, 10 mM Tris-HCl, pH 8.3, and 0.01 g/L gelatin), 2 mM MgCl2 (both from PE Applied Biosystems), 200 µM dNTPs (Pharmacia Biotech), 2 µM each primer (U1, 5'-CAGCCGCCGCGGTAATTC-3'; U2, 5'-CCGTCAATTCCTTTGAGTTT-3'; synthesized and HPLC purified by Genosys Biotechnologies), 0.125 ng of DNA (Saccharomyces cerevisiae; Sigma), 0.02 U/µL Taq DNA polymerase (Pharmacia Biotech), and doubly distilled H2O. Samples were amplified as follows: 5 min at 94 °C (initial denaturation); 30 cycles of 90 s at 94 °C, 15 s at 50 °C, 120 s at 72 °C; and 7 min at 72 °C (final extension), followed by a final hold at 4 °C.

multiproduct rapd
The RAPD reactions consisted of 1x PCR buffer (50 mM KCl, 10 mM Tris-HCl, pH 8.3, and 0.01 g/L gelatin), 1.5 mM MgCl2, 150 µM dNTPs, 3 µM primer (OPA-12, 5'-TCGGCGATAG-3'; Operon Technologies), 0.125 ng of DNA, 0.03 U/µL Taq DNA polymerase, and doubly distilled H2O. Samples were amplified as follows: 7 min at 94 °C (initial denaturation); 30 cycles of 60 s at 94 °C, 90 s at 40 °C, 120 s at 72 °C; and 10 min at 72 °C (final extension), followed by a final hold at 4 °C.

standardization of test assays
Every effort was made to minimize the introduction of variation not caused by thermal cycler performance. This included providing standardized sample DNA, reagents and reaction tubes; minimizing the pipetting undertaken by participants; supplying a detailed protocol; and requesting that only experienced PCR users should take part in the study. The standardized sample DNA and reagents were prepared at LGC (Teddington) Ltd. in two master mixtures per assay, before excess volumes were aliquoted out and dispatched to participants. One master mixture contained 1x PCR buffer, MgCl2, dNTPs, primers, DNA, and doubly distilled H2O. The other contained Taq DNA polymerase only. The isolation of Taq DNA polymerase from the DNA template prevented the occurrence of any nonspecific reactions, whereas the simple addition of one reagent mixture to another by the participant would minimize the introduction of pipetting errors. Because differences in the thickness of tubes from different suppliers may affect the temperature that the sample reaches, standardized thin-walled microtubes (Alpha Lab Supplies) of appropriate sizes were also supplied. In addition to the usual precautions of performing PCR tests in a clean environment, it was also requested that filtered tips or positive-displacement pipettes be used for each liquid transfer to avoid contaminating the test assays with foreign DNA.

The final detailed protocol was produced as a result of comments by participants on a draft copy, circulated for discussion before the study. The protocol stated that all liquid transfers were to be carried out on ice. To ensure complete homogenization of solutions before amplification, prepared reactions were briefly vortex-mixed and the contents centrifuged before being placed in the thermal cycler. To make direct comparisons between thermal cyclers, PCR reactions were positioned in designated wells within the thermal cycler block. Variation attributable to the different block layouts exhibited by different makes and models of thermal cycler was minimized by specifically designating positions to maintain a similar number of corner (n = 4), edge (n = 4; 2 negative and 2 positive reactions) and internal (n = 2) well positions. All assay components were dispatched to participating laboratories by next-day courier under dry ice, with instructions for reagents to be stored at -20 °C upon arrival. A 2-week turnaround time was given for completion of the test assays.

participants of the interlaboratory study
Any laboratory in the United Kingdom that was an established and regular PCR user was invited to take part in this study, either by personal invitation or through a notice placed in the Diagnostics Club newsletter Exchange. A total of 18 laboratories, representing many different sectors of DNA analysis, participated in the study. The test assays were carried out on 33 thermal cyclers of differing model and age and from various manufacturers.

pcr product analysis and data interpretation
Aliquots (10 µL) of the products from both assays were analyzed by electrophoresis on 2% agarose in 1x Tris-borate-EDTA buffer (10x stock: 8.9 mol/L Tris-HCl, pH 8.0, 8.9 mol/L boric acid, 20 mmol/L EDTA) containing 0.5 µg/mL ethidium bromide, and visualized under ultraviolet illumination. The remaining products and a gel photograph of the results were returned to LGC. After arrival at LGC, the products from both assays were reanalyzed by electrophoresis as described above. On this occasion, to facilitate direct block-to-block comparisons, all of the reactions amplified in specific block positions from all thermal cyclers were analyzed on the same gel.

In one case, the operator knew that a particular thermal cycler consistently failed to reach the programmed temperatures by a defined amount, and the programmed temperatures were therefore routinely adjusted to correct for this error. Data were generated using this piece of equipment both at the adjusted and stated temperatures.

All of the results were analyzed and interpreted independently by two individuals. No image analysis equipment was used. When the results were analyzed, two sets of RAPD results were not included: those generated by the nonadjusted thermal cycler mentioned above and one set of results that was not returned by the participating laboratory (a total of 31 instruments analyzed). Analysis of the single-product PCR also excluded data from the nonadjusted thermal cycler (a total of 32 instruments analyzed).

pcr block well performance: statistical treatment and interpretation of "nonrepeatable" counts
The counting scheme used had some properties that may lead to biased indications of consistency for particular positions: There were four corner positions, two edge positions, and two interior positions, potentially biasing counts toward "corner" positions. However, calculation on the basis of binomial probabilities showed that mean counts were unbiased where a single profile predominated. In addition, no total count could be equal to 1; the possible values were 0 and 2–8. The distribution was likely to be, at best, a censored binomial distribution, making formal confidence intervals hard to estimate accurately. Monte Carlo simulations using the observed probabilities showed that for match counts >50% of the possible number of profiles, the distribution followed binomial expectations closely. Confidence intervals, where shown, were accordingly calculated at the 95% level, using the methods described by Howarth (3) and assuming independent profiles within each group.


   Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
single-product pcr assay
For the single-product PCR, the results were analyzed both on the presence or absence of a single amplification product of the expected size (590 bp) and on the relative yield of each product within a single thermal cycler. In general, the overall repeatability of the single-product PCR was good, as one might expect for a reasonably stringent, robust, and optimized reaction. All thermal cyclers generated a single amplification product of the predicted size, and in the majority of cases, the overall product yield showed little variation across the different areas of the heating block. The results from six instruments showed very slight variations in the yield of PCR product generated from the replicate reactions. Of these, only two thermal cyclers were known to be temperature calibrated. More substantial variations in yield from the replicate reactions were observed in 3 of the 32 instruments analyzed. Instrument 13 (noncalibrated) demonstrated considerable well-to-well variation: of the eight positive reactions, two generated a high yield of product (at positions 6 and 8), three generated an "intermediate" amount of product (at positions 2, 3, and 4), and the remaining three generated very little product (at positions 1, 5, and 7). In instruments 14 and 15 (noncalibrated and of unknown calibration status, respectively), seven of the eight tubes generated equivalent amounts of product, whereas the remaining reaction yielded very little product (at position 5 in instrument 14 and position 2 in instrument 15). All of the low-yield reactions were generated on three oil-overlay instruments (10 were used in the study).

Results generated at comparable well positions on different thermal cyclers were also compared. For example, the results obtained from well position 6 are illustrated in Fig. 1 . As expected, the single-product PCR was also highly reproducible between different thermal cyclers, with all instruments investigated (except instrument 19) generating the correctly sized product. The results generated from the unadjusted and adjusted amplifications (instruments 19 and 18, respectively) were of particular interest because they clearly illustrated the importance of temperature calibration to generate positive results. Six amplification failures were recorded when reactions were performed using the unadjusted instrument. However, when that instrument was adjusted to compensate for the known temperature bias, all replicate PCR reactions generated positive results. We do not, however, recommend operator-mediated thermal cycler temperature adjustments as a suitable course of action to alleviate temperature bias.



View larger version (133K):
[in this window]
[in a new window]
 
Figure 1. Comparative results for all single-product reactions performed in well position 6 (edge position) of all thermal cyclers tested.

Lane M, 100-bp molecular weight maker; lane K, 1-kb molecular weight marker; lanes 1–33; single-product PCR results from well 6 (edge position) of all thermal cyclers tested (instruments 1–33). Note: instrument 19 (lane 19) was known to not attain the programmed temperatures. The same thermal cycler, adjusted to compensate for this temperature bias, is illustrated in the adjacent lane (lane 18).

multiproduct rapd assay
Interpretation of the RAPD results was based on the presence or absence of specific bands in each profile. A diagram of the maximum number and positions of the amplified RAPD products is illustrated schematically in Fig. 2 .



View larger version (19K):
[in this window]
[in a new window]
 
Figure 2. Maximum range of bands produced by the RAPD assay.

Relative intensities of the bands as seen on the agarose gel are reproduced. The dashed bands were the faintest and considered on the borderline of detectability. These were therefore not included in the final analysis of the RAPD data.

block-to-block variation
Block-to-block variation was assessed by dividing the profile into four separate categories (shown in Fig. 2Up ): a "core" product (940 bp); six high-molecular weight (HMW) products; five mid-molecular weight products; and a single low-molecular weight (LMW) product. Generation of the remaining amplification products was less repeatable. These, indicated in Fig. 2Up by dashed lines, were considered to be on the borderline of detection and, as a consequence, were not used to generate any data for profile analysis.

In general, RAPD reproducibility was found to vary markedly between different thermal cyclers (typical examples are given in Fig. 3 ), highlighting the variation in results that can occur when set thermal profiles are transferred between different thermal cycler instruments. It should be noted, however, that in contrast to the single-product PCR results, the profiles generated using the nonadjusted and temperature-adjusted thermal cycler (Fig. 2Up , lanes 19 and 18, respectively) showed apparent similarities. This observation is presumably attributable to the less stringent nature of the RAPD analyses. An analysis of the average number of amplification products generated within each of the above categories is illustrated in Table 1 . One fragment, the core product, appeared to be more reproducible than the others. This was present in all RAPD profiles analyzed, with the exception of instruments 1 (total amplification failure at position 1), 15 (failure at positions 3 and 6; very faint product for remaining positions) and 27 (total amplification failure at position 8). Generation of the remaining fragments selected for analysis was, on average, less reproducible both within and between thermal cyclers. A direct comparison between temperature-calibrated and noncalibrated instruments demonstrated no significant difference in the average number of bands generated for both categories of instrument. There was, however, a clear difference in the average number of HMW and LMW bands produced in thermal cyclers using different temperature control mechanisms. The external simulated tube control (ESTC) thermal cyclers produced, on average, fewer HMW bands and more frequently produced the LMW band. In contrast, the internal computer-simulated control (ICSC) method typically produced a higher number of HMW bands and produced the LMW band less frequently.



View larger version (137K):
[in this window]
[in a new window]
 
Figure 3. Comparative results for all RAPD reactions performed in well position 6 (edge position) for all thermal cyclers investigated, illustrating interblock variation.

Lane M, 100-bp molecular weight marker; lane K, 1-kb molecular weight marker; lanes 1–33; RAPD results from well 6 (edge position) for all thermal cyclers tested (instruments 1–33). Note: there are no results in lanes 11 and 15 because these participants withdrew their RAPD results from the study.


View this table:
[in this window]
[in a new window]
 
Table 1. Analysis of the preferential amplification of high- or low-molecular weight RAPD products.

well-to-well variation
The RAPD profiles were further analyzed by comparing the total number of repeatable profiles obtained from an individual block. For this purpose, a repeatable profile was defined as the most frequently occurring profile out of the eight produced by each individual thermal cycler. For example, if a single thermal cycler generated two similar profiles and six dissimilar profiles, then that block was scored as producing two repeatable profiles or six nonrepeatable profiles.

In general, RAPD repeatability was found to vary markedly between different tube positions within the same thermal cycler block; typical examples are illustrated in Fig. 4 . The effects of calibration status, thermal cycler age, temperature control mechanism, and sample evaporation control mechanism on the average number of repeatable profiles generated per thermal cycler was further investigated, and the results obtained are summarized in Table 2 . On average, calibrated instruments were found to generate a greater number of repeatable RAPD profiles compared with noncalibrated instruments. For calibrated instruments, out of the total number of profiles generated per thermal cycler, approximately seven of eight were repeatable. In contrast, for the noncalibrated equipment, this value was reduced to five of eight. The older thermal cyclers that participated in this study (those >3 years of age), appeared, on average, to produce less repeatable profiles than their younger counterparts, but the significance of the age/repeatability relationship was marginal (P = 0.08 for linear regression of age against number of repeatable profiles). An investigation of the relationship between the thermal cycler temperature control mechanism and data repeatability indicated that ICSC instruments were superior, generating a median of 8 and mean of 7 (of 8) repeatable profiles, compared with means of 5.3 and 4.5 for block control (BC) and ESTC instruments, respectively. Furthermore, heated-lid thermal cyclers were found to generate more repeatable data, displaying a median of 8 and mean of 6.6 repeatable profiles compared with the median of 5 and mean of 4.8 generated by instruments that required an oil overlay.



View larger version (106K):
[in this window]
[in a new window]
 
Figure 4. Examples of the RAPD assay results generated in two of the thermal cyclers investigated in this study, illustrating well-to-well and block-to-block variation.

Lane M, 100-bp molecular weight marker; lanes 1–8, RAPD-positive reactions; lanes 9 and 10, RAPD-negative reactions.


View this table:
[in this window]
[in a new window]
 
Table 2. Effect of various factors on well-to-well variation of RAPD profiles.

Instrument temperature calibration is an independent variable, allowing valuable observations to be derived with respect to well position and profile repeatability. However, because of the size of the study, the effects on repeatability of the other variables analyzed were of less statistical significance as these were inherently interconnected. For example, in the comparison between different temperature control mechanisms, five of the eight ICSC instruments (55.5% of the total number of calibrated instruments in the study) were calibrated. Furthermore, all eight ICSC instruments possessed heated lids. However, the ICSC instruments were of a variety of ages: three were older than 4 years, two older than 2 years, two older than 1 year, and 1 of unknown age. Temperature-calibrated instruments were split equally between the heated-lid and oil overlay categories. The heated-lid thermal cyclers consisted of all eight ICSC instruments, four BC, and one ESTC monitoring system. Once again, a range of ages of thermal cyclers was represented in the heated-lid category: four were older than 4 years, one older than 3 years, four older than 2 years, three older than 1 year, and two of unknown age. Moreover, older thermal cyclers tended to have oil overlays and BC as their mechanism of temperature control.

The areas of the thermal cycler blocks that most commonly failed to achieve a uniform temperature were identified by analysis of the proportion of nonrepeatable profiles generated at those positions [(number of repeats observed)/(number of wells)]. Results obtained demonstrated that the internal sites, on average, produced the highest number of nonrepeatable results: 12.5 profiles out of 31, compared with 9.3 and 8.5 for the corner and edge positions, respectively (Fig. 5 ).



View larger version (42K):
[in this window]
[in a new window]
 
Figure 5. Effect of well position on RAPD profile repeatability.

The proportion of nonrepeatable profiles for each well position is shown. Confidence intervals are at the 95% level assuming a binomial distribution. Bars, SD.

The number of non-repeats produced per profile generated was also examined with respect to the temperature calibration status of the thermal cyclers (Fig. 6 ). In this study, calibrated thermal cyclers appeared to generate significantly fewer nonrepeatable RAPD profiles than noncalibrated instruments. Interestingly, the number of non-repeats at corner wells was the same irrespective of calibration status, whereas at the edge and internal positions, the number of nonrepeatable positions was reduced from 0.35 to 0.1 and from 0.5 to 0.1 non-repeats per profile when a comparison was made between noncalibrated and calibrated instruments, respectively.



View larger version (40K):
[in this window]
[in a new window]
 
Figure 6. Effect of instrument temperature calibration status on well-to-well RAPD profile repeatability.

The proportion of nonrepeatable profiles in each position is shown. Confidence intervals are at the 95% level assuming a binomial distribution. Bars, SD.


   Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Based on the analysis of the robust, single-product PCR, the worst performers in this study, i.e., those that produced the most nonrepeatable data, had the following characteristics: no current temperature calibration; oil overlay; BC; and >4 years old. Based on the analysis of the temperature-sensitive RAPD assay and individual thermal cycler characteristics, the best performers, i.e., those that produced the most repeatable data, had the following characteristics: temperature calibration; heated lid facility; internal computer simulated temperature control; and <2 years old. Three instruments in this study had exactly these characteristics, and they all produced eight of eight repeatable RAPD profiles.

thermal cyclers
Inherent differences in the mechanisms of thermal cycler performance could account for some of the variation observed between different makes and models of thermal cycler. For this study, the thermal profile specified in the protocol stipulated that the ramp rates should be set at maximum. The technical specifications of the instruments used in this study, however, showed that the maximum ramp rates achievable varied from 1 to 3 °C/s, depending on the thermal cycler. Furthermore, the RAPD assay is known to be sensitive to small fluctuations in temperature. Consequently, it was important to consider the influence of the various temperature control mechanisms used in the design of the thermal cyclers on the actual temperatures reached by individual samples and their impact on the resulting profiles generated (1). The different mechanisms used by the thermal cyclers in this study to monitor and control block and/or sample temperature were highlighted and are described below.

For these reasons, it is understood that the requirement that all instruments use the same thermal profile introduced a certain bias into the temperatures obtained by the samples themselves. It is therefore essential to stress that, for the purposes of this study, there was no "correct" profile for the RAPD reaction and that analysis of RAPD reproducibility would primarily highlight the implications of transferring a set thermal protocol from one thermal cycler to the next. Other differences in thermal cyclers include (a) the use of a heated lid or mineral oil overlay to minimize evaporation of the sample; (b) the thermal cycler ramp rate; (c) the reaction tube size; and (d) the presence or absence of a cooling facility, i.e., the capacity of the thermal cycler to attain subambient temperatures. All of these variables emphasize the potential risks with regard to test reproducibility that may be associated with transferring a set thermal profile from one type of thermal cycler to another.

calibration
Calibration is the process of establishing the response of an instrument to a set of conditions. In the case of thermal cycler calibration, a programmed temperature is allowed to equilibrate before the actual temperature is measured with a temperature probe placed in a well. Although initial responsibility for calibration of the thermal cycler lies with the manufacturer or vendor, sale or installment of the equipment transfers the responsibility of calibration to the testing laboratory; however, it is perfectly in order and perhaps prudent for the manufacturer to offer calibration recommendations. There probably is not an analytical or research situation where calibration is not strongly recommended and in the majority of cases should be enforced. If results need to be generated using a different thermal cycler instrument, the validity of the data could be open to question if the instrument responds in a different manner to a set of transferred conditions. Information regarding calibration services (definition of calibration with respect to accuracy and uniformity, the number of well positions tested, the range of temperatures tested, and the nature of postcalibration instrument adjustments) can be obtained from thermal cycler manufacturers.

interlaboratory trials
One of the first reported multicenter blinded PCR proficiency trials appeared in 1991 (4). The authors’ final recommendations included standardization of reagents and procedures and ongoing participation in blinded proficiency testing (PT) schemes of laboratories offering PCR tests. Since then, many other trials have been published. Although these have been predominately in the clinical field, there recently have been a growing number of reports in nonclinical sectors (5). A review of the literature revealed that an interlaboratory study on the performance of thermal cyclers had not been undertaken previously. It is clear, however, that there currently is some concern over the quality of the PCR-based data being produced and that interlaboratory studies are seen by some as an efficient way of assessing variability of results. Therefore, we present a short summary of some interlaboratory studies undertaken to study DNA technologies and some comments highlighting the possible effects of thermal cycler performance.

Neumaier et al. (6) offered eminently sensible guidelines on the application of PCR in diagnostics and emphasized the need for standardization and definition of laboratory organization. However, the authors were concerned that because the methodology was in constant change, no general standards could be defined. They identified uniform temperature transition as an important aspect of a successful amplification and specifically noted that the heat conduction in the heating block must be controlled regularly to avoid false-negative results. They made no mention of advising on thermal cycler calibration. A European interlaboratory study investigating the reproducibility of RAPD, amplified fragment length polymorphism, and simple sequence repeat markers found that RAPDs were difficult to reproduce even when standardized reaction mixtures were used (5). The standard thermal profile was obtained by participants copying a thermal profile printed from the PCR instrument of the distributor (it is unclear whether this thermal profile was verified or actually achieved by the participants’ thermal cyclers). In spite of this, significant variation in the RAPD profiles obtained was reported (5). In another interlaboratory study, participants used their own PCR technique to detect Toxoplasma gondii (7). Detection limits were found to vary, and false-positive and false-negative results were reported. The authors concluded that until an appropriate quality assurance scheme was established, they would recommend that PCR be used in association with other laboratory methods. They appreciated the fact that accuracy of the thermal cycler instruments may have contributed to the observed discrepancies, and they also emphasized the urgent need for the standardization of PCR protocols and the possible accreditation of diagnostic laboratories. The authors further stressed that the differences in PCR design and potential discrepancies in performance of PCR highlighted in the study underlined the caution required in applying findings (particularly PCR sensitivity and specificity) from other published clinical studies using different PCR methodologies. Weber et al. (8) conducted a study for the detection of the human polyomavirus JC (JCV), using PCR. Five of the six participants reported a detection limit of 1–10 JCV copies per 10 µL, and the remaining participant reported a detection limit of 100 000 JCV copies per 10 µL. This marked difference in sensitivity was ascribed to nonspecified "thermocycler problems".

enhancing thermal cycler performance and confidence in data
This interlaboratory study has increased awareness among the participants of the importance of validation issues in PCR-based analyses, emphasized the importance of instrument calibration, and raised interest in the possible use of regular PT schemes in DNA analysis. Such schemes could offer several benefits, including external recognition, increased competitiveness, and improved quality of data. They also give an independent assessment of performance and allow a comparison of results between laboratories. Furthermore, tight internal quality control can lead to bias, and an external check may be desirable to compare the precision of one laboratory with another. In other words, although a laboratory may be producing consistent and repeatable results, there is no guarantee that these results are actually correct. This can be verified only after comparison with other laboratories. Of all of the participants in this interlaboratory study, only four had previously participated in some form of PT scheme. The majority of participants agreed that thermal cycler calibration and temperature monitoring are important issues for the generation of reliable and repeatable data.

There are numerous steps that can be taken to maintain or improve thermal cycler performance. Yearly temperature calibration by a reputable thermal cycler manufacturer is considered a minimum requirement. Performance can be further assured throughout the year by in-house calibration using a sensitive test, such as a RAPD assay. Monthly runs of such a biological assay should highlight any variation in performance. One advantage of a temperature-sensitive biological control resides in the fact that it monitors dynamic temperature uniformity, whereas a temperature probe-type calibration usually monitors only static temperature uniformity. In addition, the number of wells monitored limits temperature probe calibration. A biological control could be analyzed periodically in all block positions. It must, however, be stressed that use of a robust assay that is not temperature sensitive will lead the laboratory into a false sense of security and will be of little value. Simple housekeeping duties such as closing the lid of a thermal cycler when not in use and keeping the wells clean by the use of cotton balls and alcohol will ensure that there is a good fit between the block and the tubes, thereby maximizing heat transfer. The use of internal PCR controls is now becoming increasingly popular. These have the benefit of controlling for false-negative reactions at each well position. Such controls can be incorporated into assays in the form of competitive reactions, where the sample and control targets share common primer pairs, or in a noncompetitive reaction, where both targets are amplified with unique primer pairs. Finally, additional confidence in the data generated can be obtained by participating in interlaboratory studies or PT trials. Such an undertaking will allow independent assessment or benchmarking of performance. These schemes have become much more common in recent years. Where comparability of data is required, such multicenter collaborative trials can offer a way of improving the reproducibility of results by offering continual monitoring, standardization, and troubleshooting opportunities.


   Acknowledgments
 
The work presented here was supported under contract with the Department of Trade and Industry as part of the National Measurement System Program Valid Analytical Measurement Program. We would also like to thank Dr. Steve Ellison (LGC Teddington Ltd.), who provided expert assistance with the statistical interpretation of the RAPD data, and all the participants, who willingly gave their time to undertake the trial and contribute to discussions on the results obtained: Advanced Biotechnologies; Advanced Technologies (Cambridge) Ltd.; CABI (International Mycological Institute); Campden and Chorleywood Food Research Association; ERVL (Central Public Health Laboratory); Hammersmith Hospital (MRC Clinical Sciences Research Centre); IACR Rothamsted Experimental Station; ICSM, Charing Cross Hospital (Medical Microbiology); Queens University of Belfast (Food Microbiology); RHM Technologies Ltd.; Rowett Research Institute; Royal Manchester Children’s Hospital (Molecular Genetics Laboratory); St. Bartholomew’s & The Royal London School of Medicine and Dentistry; University College London (Department of Virology); University Diagnostics Ltd.; University of London (Biotechnology, Birkbeck College); and Zeneca Diagnostics.


   Footnotes
 
1 Nonstandard abbreviations: RAPD, random amplified polymorphic DNA; HMW, high molecular weight; LWM, low molecular weight; ESTC, external simulated tube control; ICSC, internal computer-simulated control; BC, block control; PT, proficiency testing; and JCV, human polyomavirus JC.


   References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 

  1. McDowell DG. PCR: factors affecting reliability and validity. In: Saunders GC, Parkes HC, eds. Analytical molecular biology: quality and validation. Cambridge: The Royal Society of Chemistry, 1999:58–80..
  2. Lane DJ, Pace B, Olsen GJ, Stahl DA, Sogin ML, Pace NR. Rapid determination of 16S ribosomal RNA sequences for phylogenetic analyses. Proc Natl Acad Sci U S A 1985;82:6955-6959.[Abstract/Free Full Text]
  3. Howarth RJ. Improved estimators of uncertainty in proportions, point-counting, and pass-fail tests. Am J Sci 1998;298:594-607.[Abstract/Free Full Text]
  4. Sheppard HW, Ascher MS, Busch MP, Sohmer PR, Stanley M, Luce MC, et al. A multicenter proficiency trial of gene amplification (PCR) for the detection of HIV-1. J Acquir Immune Defic Syndr 1991;4:277-283.
  5. Jones CJ, Edwards KJ, Castaglione S, Winfield MO, Sala F, van de Wiel C, et al. Reproducibility testing of RAPD, AFLP and SSR markers in plants by a network of European laboratories. Mol Breed 1997;3:381-390.
  6. Neumaier M, Braun A, Wagener C. Fundamentals of quality assessment of molecular amplification methods in clinical diagnostics. Clin Chem 1998;44:12-26.[Abstract/Free Full Text]
  7. Pelloux H, Guy E, Angelici MC, Aspöck H, Bessières M-H, Blatz R, et al. A second European collaborative study on polymerase chain reaction for Toxoplasma gondii involving 15 teams. FEMS Microbiol Lett 1998;165:231-237.[Web of Science][Medline] [Order article via Infotrieve]
  8. Weber T, Klapper PE, Cleator GM, Bodemer M, Lûke W, Knowles W, et al. Polymerase chain reaction for detection of JC virus DNA in cerebrospinal fluid: a quality control study. J Virol Methods 1997;69:231-237.[Web of Science][Medline] [Order article via Infotrieve]



The following articles in journals at HighWire Press have cited this article:


Home page
J Med MicrobiolHome page
M. M. Collery, D. S. Smyth, J. M. Twohig, A. C. Shore, D. C. Coleman, and C. J. Smyth
Molecular typing of nasal carriage isolates of Staphylococcus aureus from an Irish university student population based on toxin gene PCR, agr locus types and multiple locus, variable number tandem repeat analysis
J. Med. Microbiol., March 1, 2008; 57(3): 348 - 358.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
C. Orlando, P. Verderio, R. Maatman, J. Danneberg, S. Ramsden, M. Neumaier, D. Taruscio, V. Falbo, R. Jansen, C. Casini-Raggi, et al.
EQUAL-qual: A European Program for External Quality Assessment of Genomic DNA Extraction and PCR Amplification
Clin. Chem., July 1, 2007; 53(7): 1349 - 1357.
[Abstract] [Full Text] [PDF]


Home page
J. Clin. Microbiol.Home page
D. Schoder, A. Schmalwieser, G. Schauberger, J. Hoorfar, M. Kuhn, and M. Wagner
Novel Approach for Assessing Performance of PCR Cyclers Used for Diagnostic Testing
J. Clin. Microbiol., June 1, 2005; 43(6): 2724 - 2728.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
L. Birch, C. A. English, M. Burns, and J. T. Keer
Generic Scheme for Independent Performance Assessment in the Molecular Biology Laboratory
Clin. Chem., September 1, 2004; 50(9): 1553 - 1559.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
D. Schoder, A. Schmalwieser, G. Schauberger, M. Kuhn, J. Hoorfar, and M. Wagner
Physical Characteristics of Six New Thermocyclers
Clin. Chem., June 1, 2003; 49(6): 960 - 963.
[Full Text] [PDF]


Home page
Clin. Chem.Home page
C. C. Raggi, P. Pinzani, A. Paradiso, M. Pazzagli, and C. Orlando
External Quality Assurance Program for PCR Amplification of Genomic DNA: An Italian Experience
Clin. Chem., May 1, 2003; 49(5): 782 - 791.
[Abstract] [Full Text] [PDF]


Home page
MicrobiologyHome page
K. Rolland, L. Mereghetti, S. Watt, S. Chatellier, and R. Quentin
tRNA gene clusters at the 3' end of rRNA operons are specific to virulent subgroups of Streptococcus agalactiae strains, as demonstrated by molecular differential analysis at the population level
Microbiology, May 1, 2002; 148(5): 1493 - 1499.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (19)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Saunders, G. C.
Right arrow Articles by Cornett, J. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Saunders, G. C.
Right arrow Articles by Cornett, J. H.
Related Collections
Right arrow Molecular Diagnostics and Genetics
Right arrow Automation and Analytical Techniques


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS