Clinical Chemistry
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Clinical Chemistry 52: 2072-2078, 2006. First published September 21, 2006; 10.1373/clinchem.2006.072405
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
clinchem.2006.072405v1
52/11/2072    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Dorn-Beineke, A.
Right arrow Articles by Neumaier, M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Dorn-Beineke, A.
Right arrow Articles by Neumaier, M.
Related Collections
Right arrow Molecular Diagnostics and Genetics
Right arrow Laboratory Management
(Clinical Chemistry. 2006;52:2072-2078.)
© 2006 American Association for Clinical Chemistry, Inc.

Laboratory Management

Improvement of Technical and Analytical Performance in DNA Sequencing by External Quality Assessment-Based Molecular Training

Alexandra Dorn-Beineke1,3, Parviz Ahmad-Nejad1,3, Ulrike Pfeiffer1, Simon Ramsden2, Mario Pazzagli3 and Michael Neumaier1,a

1 Institute for Clinical Chemistry, University Hospital Mannheim of the University of Heidelberg, Germany.
2 National Genetics Reference Laboratory (Manchester), St Mary’s Hospital, Manchester, United Kingdom.
3 Clinical Biochemistry Unit, Department of Clinical Physiopathology, University of Florence, Florence, Italy.

aAuthor correspondence to this author at: Institute for Clinical Chemistry, University Hospital Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1–3, D-68167 Mannheim, Germany. Fax 49-621-383-3819; e-mail michael.neumaier{at}ikc.ma.uni-heidelberg.de.


   Abstract
Top
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 References
 
Background: From 2003 to 2005, the European Union supported the EQUAL-initiative to develop methodological external quality assessment (EQA) schemes for genotyping (EQUALqual), quantitative PCR (EQUALquant), and sequencing (EQUALseq). As a relevant part of the EQUALseq program, a training course was held subsequent to the first EQA Program (EQAP1). The success of this course was reassessed in a 2nd EQUALseq round (EQAP2).

Methods: In September 2005, a 3-day training course took place. We invited 8 laboratories with below-average performance in EQAP1 to improve their methodological and analytical/proficiency skills by lectures and practical work. To compare the results of the pretraining and posttraining EQUALseq rounds, we distributed 2 samples used in the first EQUAL round, but this time we provided different oligonucleotide sets. We evaluated the results by means of a previously described scoring system.

Results: In EQAP2, 6 laboratories returned complete data sets, corresponding to an overall 14% of the 43 laboratories that had finished EQAP1. The scoring results for samples A (P = 0.0025) and B (P = 0.0125) demonstrated a significant improvement in EQAP2. Overall, a substantial improvement of technical and interpretative skills was demonstrated (P = 0.0051). In general, the workshop experience was highly rated by the participants.

Conclusions: Methodologic EQAPs in DNA sequencing are appropriate tools to uncover strengths and weaknesses in both technique and proficiency, emphasizing the need for mandatory EQAPs. Training courses, together with 2nd-round reiterations, should be implemented into methodological EQAPs in molecular diagnostics to improve technical performance and proficiency in genetic testing.


   Introduction
Top
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 References
 
Molecular diagnostic procedures are now routine in most areas of clinical laboratory sciences. For example, during recent years the testing for classical hereditary diseases has developed into an important tool in human genetic services. Similarly, clinical laboratories are increasingly making use of the results of genetic tests together with pathological biochemical findings for diagnostics in common multifactorial diseases and conditions. Genetic testing has become an integral part of laboratory medicine, as reflected by the rapidly developing number and availability of molecular tests. The rush of technological advances to bring molecular techniques into the field of clinical diagnostics, and the subsequent increased availability of these techniques to a growing number of laboratories requires an increased attention to external quality assurance programs (EQAP)1 to support the providers of genetic testing.

Recently, these developments prompted an expert working group to formulate specific recommendations that address the ethical, legal, and social implications of genetic testing and issue these recommendations to the European Commission (1). Therein, genetic information is being considered as part of the entire spectrum of all health information and not a separate category as such, in the sense of genetic exceptionalism. Importantly, the European Union (EU) recommendations state that quality assurance and the standardization of test development and usage must be considered ethical issues of genetic testing within healthcare systems [recommendations 7 and 17 (1)].

Although it is generally accepted that equally high quality standards must be maintained at all times for all medical data, including genetic data, experiences with different EQAPs show that genetic testing services appear to suffer from a disturbing number of technical and administrative errors and poor reporting (1)(2)(3).

According to a survey carried out in 2003 in 18 member countries of the Organization for Economic Co-operation and Development, the number of genetic tests performed between 2000 and 2002 has grown with an annual increase of 26%, from ~875 000 to 1 402 000, respectively, and this number may be an underestimation (4). To respond to this growing field and the EU recommendations, EQAPs should address 3 main issues: (a) the validity of genetic test results in diagnostics, (b) the quality of methodology used to obtain genetic test results, and (c) training and dissemination of proficiency in analytical procedures. Several diagnostic EQAPs are being activated in Europe in the field of human genetics and laboratory medicine (5).

Within the Organization for Economic Co-operation and Development survey, 72% of the laboratories used DNA sequencing of PCR amplicons for genetic testing, clearly demonstrating the importance of this technique for molecular diagnostics (4). From 2003 to 2005, the EU has supported the EQUAL initiative to develop Europe-wide methodological EQAP schemes for genotyping, quantitative PCR (6), and DNA sequencing (EQUALseq). As part of the EQUALseq program, we recently reported a new approach to assess the methodological quality of DNA sequencing in diagnostic laboratories (7). In brief, 60 participants from 21 European countries were asked to analyze a 4-sample reagent set comprising 2 DNA plasmids, a PCR product, and a prefabricated sequencing reaction and then to interpret the results. Of 60 participating laboratories, 43 (72%) returned a complete data set for evaluation. Eight laboratories demonstrating below-average performance in EQUALseq were invited to participate in a 3-day DNA-sequencing training session to improve their methodological and analytical/proficiency skills. Subsequent to the training, these 8 laboratories were asked to perform an additional round of EQUALseq (EQAP2) in their home laboratories to verify the training effects. We present the results from the EQUALseq training course and EQAP2.


   Materials and Methods
Top
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 References
 
overall design of the equalseq training course
From the 43 laboratories providing results in the pretraining EQAP study (EQAP1), 8 laboratories (19%) with below-average performance participated in the 3-day training course that took place at the Institute for Clinical Chemistry of the University Hospital in Mannheim, Germany (Table 1 ).


View this table:
[in this window]
[in a new window]
  		Table 1. Participating laboratories in the EQUALseq training course.

The training course consisted of 5 lecture rounds and 4 practical laboratory sessions specifically stressing the following topics: (a) history and methodological developments of nucleic acid sequencing: from past to the future (b) DNA-sequencing strategies in clinical laboratory diagnostics, (c) analysis and interpretation of raw data sets, (d) strategies for nucleotide editing from raw data sets, (e) active work with public databases such as Online Mendelian Inheritance in Man (OMIM) and the Basic Local Alignment Search Tool (BLAST), and (f) alignment of multiple sequences and troubleshooting. With respect to practical exercises, each participant received either a PCR product or plasmid DNA to set up and carry through sequencing reactions according to optimized protocols. All practical activities were accompanied and supported by experienced technical staff providing guidance during the analysis and interpretation of the DNA-sequencing procedure. Results of the practical work were presented and discussed afterward during group meetings. At the end of the training, the participants evaluated the workshop with a standardized questionnaire before receiving an EQUALseq reagent set to perform EQAP2 in their home laboratories. The detailed workflow of the training course is shown in Fig. 1 .


Figure 1
View larger version (24K):
[in this window]
[in a new window]
  		Figure 1. Workflow of the EQUALseq training course program.

The sequencing was performed as described in the Materials and Methods section. Samples provided by the coordinator were either PCR products of peripheral human blood DNA, cell culture DNA, or preparations of plasmid DNA. For all participants, optimized protocols were distributed. *P, practical activity; *L, lecture.

overall design of eqap2
To compare the results from the pretraining EQAP1 and the posttraining EQAP2 EQUALseq rounds, we distributed 2 samples used in EQAP1 (samples A and B), but this time we provided different antisense oligonucleotide sets. The participants were again asked (a) to provide the longest possible sequence that could confidently be read after editing, (b) to identify the sequences, and (c) to add notes for each sample. The latter 2 points aimed at the assessment of proficiency in analytics. Results included both raw data and edited sequences that were received after an appropriate turnaround time.

sample preparation
With respect to the material shipped to the participants in EQAP1, we provided 2 samples, a purified PCR-product and plasmid DNA. Specifically, sample A was a dried pellet of a 310-bp PCR product (160 ng), and sample B was a plasmid DNA preparation (2 µg) with an insert of 2355 bp. Both samples had been verified in the coordinator’s laboratory as described previously (7). Each laboratory received a set of 4 vials (2 primers, 2 samples). As in EQAP1, detailed instructions were enclosed in the set provided. According to EQAP1, participation in EQAP2 was free of charge.

primers
Quality-control primers (by mass spectrometry analysis) were purchased from MWG-Biotech. Appropriate aliquots were dried and tubes were labeled. Primers provided for sequencing were as follows: sequencing primer A (30 pmol, dried pellet; 5'-CAG CTG TTT CCT TCA AGA TGC-3') to use in the sequencing reaction with sample A; Sequencing Primer B (30 pmol, dried pellet; 5'-GTA AAA CGA CGG CCA G-3') to use in a sequencing reaction with sample B.

plasmid dna and pcr product
Generation of plasmid DNA and PCR product was described previously in detail (7). In brief, sample A was generated at the coordinator’s laboratory by use of a DNA sample from a patient heterozygous for 2 coding variations in the Hemochromatosis (HFE) gene2 (H63D and S65C) and for 1 homozygous variation in intron 2 (IVS2 + 4) (8)(9). This PCR product was generated in a reaction containing 2 µL of primer FOR (10 pmol/µL; 5'-GCC TCA GAG CAG GAC CTT GG-3'); 2 µL of primer REV (10 pmol/µL; 5'-CAG CTG TTT CCT TCA AGA TGC-3'); 5 µL of 10x buffer; 1.6 µL of 50 mmol/L MgCl2; 1 µL of 10 mmol/L nucleotide mixture (Eppendorf); 0.4 µL of Platinum Taq DNA polymerase (5 U/µL; Invitrogen); 280 ng of patient DNA; and 34 µL of H2O. The amplification was carried out on a Mastercycler gradient (Eppendorf) with initial denaturation at 95 °C for 120 s, followed by 35 cycles of 95 °C for 30 s, 62 °C for 30 s, and 72 °C for 30 s, and final elongation at 72 °C for 600 s. The amplicon was purified, aliquoted, and subsequently dried as 160-ng aliquots.

The plasmid sample B used in this trial was transformed in DH5{alpha} cells and isolated from overnight cultures with a Nucleo-Bond PC 500 EF Kit (Macherey-Nagel).

sequencing performed at the coordinator’s facility
The sequencing at the coordinator's laboratory was previously described in detail (7). Briefly, sequencing reactions were performed with the ABI PRISM® BigDye® Terminator v1.1 Ready Reaction Cycle Sequencing Kit (Applied Biosystems). The reaction was run on a Mastercycler gradient and reactions were subsequently purified by use of the DyeExTM 2.0 Spin Kit (QIAGEN) according to the manufacturer’s recommendations. A 4-µL portion of this purified probe was mixed with 16 µL of HiDi buffer and run on an ABI PRISM 310 genetic analyzer equipped with a 47 cm x 50 µm (i.d.) 310 capillary filled with POP6 polymer (all from Applied Biosystems). The injection time was 30 s, and the running time was 50 min.

instructions given
As previously described in EQAP1 (7), participants received detailed instructions for actions to be performed on all materials. Concerning both samples, the participants were asked to perform the sequencing reaction and to provide the longest possible sequence without any mistakes (e.g., "N-readings"). The instructions explicitly stated not to perform primer-walking and to use exclusively the primers provided by the coordinator. Electropherograms, identification of the sequences, and additional comments were to be returned electronically.

scoring of participants’ results
The results of both EQAPs were evaluated and compared with the previously described scoring system, termed SEQMA-score (7).

In brief, 6 points could be achieved in analyzing sample A: 1 point was awarded for the identification of the HFE PCR product, 1 point for the detection of the structure of the PCR product (e.g., genomic DNA, cDNA), 1 point for the correct comment on the deviation from the wild-type HFE sequence, and 1 point for the correct identification of each variation (H63D, S65C, IVS2 + 4).

For sample B, the sequencing stretch obtained by the coordinator’s laboratory (447 nucleotides) was taken as "100% reference", starting from position 102 downstream of the primer annealing sites. On that basis, the following basic and additional points were awarded. One point was awarded for every 50 bases read correctly. The scoring was terminated after the 5th sequencing error. Two additional points were given for a correct sequence beyond the reference 447 bases. One additional point was awarded for a sequencing stretch shorter than 447 bases without any errors, and 1 point was given for the correct identification of the sequence analyzed (coding sequence of the human Toll-like receptor 2). The final score was determined, and the number of mistakes (maximum of 5 points) was subtracted.

equalseq training course questionnaire
The standardized questionnaire was distributed to each participant at the end of the 3-day workshop. The evaluation included all lectures and practical activities as well as the support during the practical approaches. Furthermore, general questions were asked about participation and experience on EQAPs and the training course organization. Each participant could give suggestions for future improvements. The participants were asked to evaluate on a 5-grade scale or free text. The results were promptly collected, analyzed (Microsoft Office 2000 Professional, Microsoft), and presented to the participants (Fig. 1Up ).

statistical analysis
Statistical analyses between groups were tested by the Mann–Whitney U-test with a nonparametric approach with the SASTM software Version 8.2 (SAS Institute). P values <0.05 were considered significant.


   Results
Top
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 References
 
eqap2: results for sample a and sample b
Of the 8 participating laboratories, 6 returned complete data sets, corresponding to an overall 14% of the 43 laboratories that had finished the first EQUALseq round.

An alignment of the edited sequences of sample A for EQAP1 and EQAP2 is shown in Fig. 2 . The scoring results for sample A demonstrate a significant improvement in EQAP2 (P = 0.0025, Table 2 ). In Fig. 3 and in Table 2 , the results for sample B are shown. In EQAP2, 5 of 6 laboratories (83%) edited their sequence strings compared with only 3 laboratories (50%) in EQAP1. While sequence-reading length was not improved for the number of ambiguous N-readings (P = 0.1495), the reduction in the number of false base calls showed a significant improvement (P = 0.0405). Together, the overall results for sample B are significantly better compared with sample 1 in EQAP1 (P = 0.0125). The total results obtained with the SEQMA-score are also given in Table 2 . A significant improvement of technical and interpretative skills is clearly demonstrated for the 6 participating laboratories (P = 0.0051).


Figure 2
View larger version (38K):
[in this window]
[in a new window]
  		Figure 2. Alignment of the edited sequence strings of sample A (EQAP2) and sample 3 (EQAP1).

The overlapping reading frames of the 2 samples were aligned for each participant. The arrows depict the direction of reading. The compliance to Z92910 human HFE wild-type, N-readings, false bases, and variations detected by the participants are shown. The variations detected in the comments are described by "+" and "–".


View this table:
[in this window]
[in a new window]
  		Table 2. Scoring results.


Figure 3
View larger version (58K):
[in this window]
[in a new window]
  		Figure 3. Comparison of the raw and edited data from sample B (EQAP2) and sample 1 (EQAP1).

Both samples are the same plasmid DNA (pCR TOPO2.1 with a 2355-bp insert). The direction of reading (see arrows) was determined by the primers provided: sample B: 5'-GTAAAACGACGGCCAG-3' [standard sequencing primer M13 forward (-20)], sample 1: 5'-AACAGCTATGACCATG-3' (standard sequencing primer M13 reverse). (A), it is clearly shown that in EQAP1 only 3 laboratories (50%) edited their raw data. (B), in EQAP2, laboratory EQSBU039 produced 2 raw data files (EQSBU039 K and EQSBU039 A), because 2 participants from the same laboratory sequenced the sample, but only 1 edited file was send back. Five of 6 laboratories (83%) edited their sequences.

equalseq training course: results from the questionnaire
With respect to appreciation, the workshop experience was generally rated highly by the participants. Table 3 shows the results of the evaluation. None of the participants had previously participated in a DNA-sequencing EQAP or a training workshop. There was consensus among all participants that an EQAP for DNA sequencing (similar to the EQUALseq) itself should be backed by a concomitant training offer or workshop activities.


View this table:
[in this window]
[in a new window]
  		Table 3. EQUALseq training course: results from the questionnaire.


   Discussion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
References
 
This EQUALseq project on methodology of DNA sequencing offered specific training courses to laboratories and enabled performance comparison to assess improved analytical and proficiency skills in a confirmatory second EQAP round.

In conclusion, this study shows that a significant improvement in methodological and analytical DNA-sequencing skills can be reached by the combination of EQAP and technical/proficiency training. We propose that synergistic effects can be achieved that are superior to the isolated actions with a confirmatory second round under methodologically comparable conditions. It is noteworthy to stress that EQUALseq, although designed as a methodological approach to the DNA sequencing, addresses important aspects beyond the technical issues in a clinical molecular diagnostics setting. Particularly, the administrative and reporting skills that have been identified as a major source of failing reports (1)(2)(4) have been made amenable by the training workshop.

We propose that training courses, together with second-round reiterations, should be implemented into methodological EQAPs in molecular diagnostics to improve technical performance and proficiency in genetic testing. This approach is strongly consistent with the goals of professional organizations like the EC4 (European Communities Confederation of Clinical Chemistry and Laboratory Medicine) which aims at the cooperation and harmonization of standards and the establishment of training courses in clinical chemistry and laboratory medicine (10)(11).


   Acknowledgments
 
The EQUALseq program is part of the EQUAL program, which is funded by the 6th framework program of the European Commission. We thank Peter Findeisen and Wolfgang Kaminski for supporting us during the training course with excellent lectures.


   Footnotes
 
3 These authors contributed equally to this work.

1 Nonstandard abbreviations: EQAP, external quality assurance program; EU, European Union; EQAP2, Round 2 of EQAP study; EQAP1, pretraining EQAP study.

2 Human genes: HFE, Hemochromatosis gene.


   References
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 References
 

  1. European Commission. 25 recommendations on the ethical, legal and social implications of genetic testing. http://europa.eu.int/comm/research/conferences/2004/genetic/recommendations_en.htm (accessed March 2006)..
  2. Ibaretta D, Elles R, Cassiman JJ, Rodriguez-Cerezo E, Dequeker E. Towards quality assurance and harmonization of genetic testing services in the European Union. Nat Biotechnol. 2004;22:1230-1235.
  3. Neumaier M, Braun A, Gessner R, Funke H. Experiences with external quality assessment (EQA) in molecular diagnostics in clinical laboratories in Germany. Clin Chem Lab Med 2000;38:161-163.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  4. OECD. Quality assurance and proficiency testing for molecular genetic testing. http://www.oecd.org/dataoecd/25/12/34779945.pdf. (accessed March 2006)..
  5. Eurogentest. Molecular genetic testing – External quality assessment scheme provision (2005). http://www.eurogentest.org/cocoon/egtorg/web/files/public/unit1/EuroEQAprovision2005.pdf.xhtml (accessed March 2006)..
  6. Ramsden SC, Daly S, Geilenkeuser WJ, Duncan G, Hermitte F, Marubini E, et al. EQUAL-quant: an international external quality assessment scheme for real-time PCR. Clin Chem 2006;52:1584-1591.[Abstract/Free Full Text]
  7. Ahmad-Nejad P, Dorn-Beineke A, Pfeiffer U, Brade J, Geilenkeuser W-J, Ramsden S, et al. Methodologic European external quality assurance for DNA-sequencing: the EQUALseq program. Clin Chem 2006;52:716-727.[Abstract/Free Full Text]
  8. de Diego C, Murga MJ, Martinez-Castro P. Frequency of HFE H63D, S65C, and C282Y mutations in patients with iron overload and controls from Toledo, Spain. Genet Test 2004;8:263-267.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  9. Floreani A, Navaglia F, Basso D, Zambon CF, Basso G, Germano G, et al. Intron 2 [IVS2, T-C + 4] HFE gene mutation associated with S65C causes alternative RNA splicing and is responsible for iron overload. Hepatol Res 2005;33:57-60.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  10. EC4. EC4 Strategic Plan 2005–2008 – Goals of EC4. http://www.e-c4.org/goals_activities.html#3 (accessed June 2006)..
  11. Zerah S, McMurray J, Bousquet B, Baum H, Beastall GH, Blaton V, et al. EC4 European syllabus for post-graduate training in clinical chemistry and laboratory medicine: version 3 – 2005. Clin Chem Lab Med 2006;44:110-120.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]



The following articles in journals at HighWire Press have cited this article:


Home page
 Clin. Chem.Home page
J. Marki-Zay, C. L. Klein, D. Gancberg, H. G. Schimmel, and L. Dux
European External Quality Control Study on the Competence of Laboratories to Recognize Rare Sequence Variants Resulting in Unusual Genotyping Results
Clin. Chem., April 1, 2009; 55(4): 739 - 747.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
clinchem.2006.072405v1
52/11/2072    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Dorn-Beineke, A.
Right arrow Articles by Neumaier, M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Dorn-Beineke, A.
Right arrow Articles by Neumaier, M.
Related Collections
Right arrow Molecular Diagnostics and Genetics
Right arrow Laboratory Management

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS