Clinical Chemistry Siemens Point of Care - Urinalysis
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Clinical Chemistry 52: 728-736, 2006. First published February 2, 2006; 10.1373/clinchem.2005.061887
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Data Supplements
Right arrow All Versions of this Article:
clinchem.2005.061887v1
52/4/728    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (6)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Patton, S. J.
Right arrow Articles by Elles, R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Patton, S. J.
Right arrow Articles by Elles, R.
Related Collections
Right arrow Molecular Diagnostics and Genetics
Right arrow Laboratory Management
(Clinical Chemistry. 2006;52:728-736.)
© 2006 American Association for Clinical Chemistry, Inc.

Laboratory Management

Benchmark for Evaluating the Quality of DNA Sequencing: Proposal from an International External Quality Assessment Scheme

Simon J. Pattona, Andrew J. Wallace and Rob Elles

European Molecular Genetics Quality Network, National Genetics Reference Laboratory, St. Mary’s Hospital, Manchester, United Kingdom.

aAddress correspondence to this author at: European Molecular Genetics Quality Network (EMQN), c/o National Genetics Reference Laboratory, St. Mary’s Hospital, Hathersage Road, Manchester M13 0JH, United Kingdom. Fax 44-161-276-6606; e-mail simon.patton{at}cmmc.nhs.uk.


   Abstract
Top
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 References
 
Background: In the past 15 years, clinical laboratory science has been transformed by the use of technologies that cross the traditional boundaries between laboratory disciplines. However, during this period, issues of quality have not always been given adequate attention. The European Molecular Genetics Quality Network (EMQN) has developed a novel external quality assessment scheme for evaluation of DNA sequencing. We report the results of an international survey of the quality of DNA sequencing among 64 laboratories from 21 countries.

Methods: Current practice for DNA sequence analysis was established by use of an online questionnaire. Participating laboratories were provided with 4 DNA samples of validated genotype. Evaluation of the results included assessing the quality of sequence data, variant genotypes, and mutation nomenclature. To accommodate variations in mutation nomenclature, variants indicated by participants were scored for compliance with 3 acceptable marking schemes.

Results: A total of 346 genotypes were analyzed. Of these, 19 (5%) genotyping errors were made. Of these, 10 (53%) were false-negative and 9 (47%) were false-positive results. A further 27 (8%) errors were made in naming mutations. Results were analyzed for 3 indicators of data quality: PHRED quality scores, Quality Read Length, and Quality Read Overlap. Most laboratories produced results of acceptable diagnostic quality as judged by these indicators. The results were used to calculate a consensus benchmark for DNA sequencing against which individual laboratories could rank their performance.

Conclusions: We propose that the consensus benchmark can be used as a baseline against which the aggregate and individual laboratory standard of DNA sequencing may be tracked from year to year.


   Introduction
Top
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 References
 
Clinical laboratory science has been transformed by generic molecular genetic technologies that cross traditional boundaries between diverse laboratory disciplines, including genetics, hematology, clinical chemistry, and microbiology. However, in bringing these new technologies into diagnostic practice, issues of quality have not been given adequate attention. External quality assessment (EQA) 1 schemes have been established to help laboratories measure their performance against other centers and against fixed standards. Examples include schemes offered by the United Kingdom National External Quality Assessment schemes (UKNEQAS) (1), the College of American Pathologists (CAP) (2), Quality Control in Molecular Diagnostics (QCMD) (3), Multi-National External Quality Assay (EQA) Programmes in Clinical Molecular Diagnostics (EQUAL project) (4), the European Cystic Fibrosis Thematic Network (5), and the European Molecular Genetics Quality Network (EMQN) (6).

The EMQN was formed in 1997 in response to concerns about the lack of provision of EQA schemes for diagnostic molecular genetics laboratories, to improve and harmonize EQA for molecular genetic testing in Europe. From 1999 to 2002, EMQN received funding from the European Commission under the Framework IV program for Standards, Measurement and Testing (contract no. SMT4-CT98-7515). Since then, it has been supported through subscription of the participants, and has been coordinated from the National Genetics Reference Laboratory at Manchester, United Kingdom (7). Recently, the European Commission’s Joint Research Centre highlighted the need for improved quality assurance and harmonization of genetic testing services in the European Union (8). To date, most EQA schemes for molecular genetics have focused on gene-specific targets relevant to clinical applications such as inherited breast cancer (9), cystic fibrosis (10)(11), familial thrombophilia (12), and hepatitis B testing(13). Laboratory performance has been assessed on the basis of the ability to accurately detect genotypes. Some EQA schemes related to clinical testing have also sought to assess clinical interpretation by use of expert opinion and consensus guidelines such as those provided by EMQN (14) and the European Academy for Andrology (15). Although there is a strong demand to add additional gene-specific schemes, the number of gene targets relevant to clinical testing, >1000 reported to date (8), place a practical limit on the coverage of EQA in this format. In response, EQA scheme providers have begun to develop generic technical schemes that are widely applicable to clinical, core technology, and other testing laboratories (16)(17)(18)(19)(20). DNA sequencing is an example of a widely used analytical technology in many biological laboratories. To address this need, the EMQN has developed an EQA scheme for the evaluation of DNA sequence analysis. The first pilot scheme ran in 2002 and involved 34 laboratories from 15 countries (21). Here we report the results of the second international survey of the quality of DNA sequencing among 64 laboratories from 21 countries. The EQA scheme assessed laboratory performance in DNA sequencing through the ability to detect variants and also assessed raw DNA sequence data, using standard statistical measures.


   Materials and Methods
Top
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 References
 
design of the eqa scheme
Laboratories registered for the scheme on-line, using the EMQN website (6), and were allocated a unique identification number to allow anonymization of their participation to all but the EQA scheme organizer. The EQA scheme was split into 2 parts. Part 1 consisted of an online questionnaire to assess current practice in DNA sequencing. This included the laboratory setting, the sequencing technology used, and use of reagents such as fluorescently tagged primers. This information allowed the organizers to adjust the practical scheme to meet the requirements of all participants.

samples
The scheme organizer provided 450-bp PCR-amplified fragments of exon 10 of the cystic fibrosis transmembrane conductance regulator (CFTR) gene (GenBank accession no. M55115). The 4 test samples included all of the main types of sequence changes laboratories were likely to encounter. In addition, a stated wild-type control was provided. Samples were prepared by amplification using primer pair 1 (see below). To accommodate laboratories with sequencing strategies using universal primers, samples tagged with M13 tails (primer pair 2 or 3) were also provided. The primer sequences were as follows:

The amplification conditions were as follows: initial denaturation for 5 min at 95 °C, followed by 35 cycles of 94 °C for 1 min, 55 °C for 1 min, and 72 °C for 1 min, with a final 5-min extension at 72 °C. The reaction conditions were 20 ng of DNA, 1x CM102 Reddyload PCR mixture (Abgene), and 2 µM each of the forward and reverse primers in a total volume of 40 µL. Each sample was purified free of excess primers by use of a QuickStepTM Kit (Edge Biosystems).

validation
Before distribution, sample genotypes were validated by the scheme organizer and by an independent laboratory. All variant genotypes were concordant between the 2 centers.

distribution
EQA samples for distribution were aliquoted by use of a liquid-handling robot (Tecan Miniprep 75) into prelabeled 0.6-mL screw-cap tubes. Each sample tube contained 20 µL of DNA at a concentration of 35 ng/µL. The primer tubes contained 20 µL of either the forward or reverse primer at a concentration of 5 µM. Each participating laboratory received a package containing the scheme materials and instructions sent by next-day courier.

instructions
Participants were provided with a datasheet including a reference sequence, the protein translation, and location of primer sequences. Laboratories were asked to return a preprinted proforma with their genotyping results, along with color copies of the sequencing electropherograms and electronic copies of the sequence data files.

evaluation
Evaluation of the results included assessment of the genotypes and the quality of sequence data. To accommodate variations in the mutation nomenclature used, 3 marking schemes were devised (for an example, see Table 1 ). Correct genotypes were assigned a maximum of 2.00 marks. Incorrect data led to deductions from this score, with a completely wrong genotype scoring 0.00 marks. A score was awarded for 3 criteria (correct assignment of the nucleotide change, the systematic name, and the mutation effect) for each allele. The total score for each correct genotype was 3.00 marks. The maximum possible score for the scheme was 12.00 marks.


View this table:
[in this window]
[in a new window]
  		Table 1. Marking scheme 1 (fully HGVS compliant; nucleotide numbering from the translation initiation site).

The quality of the sequence data provided by participants was also assessed, and laboratory performance was ranked. The following measures were used.

PHRED quality scores.
Each sequence data file was analyzed by use of a PHRED basecaller, which we used in conjunction with the InterPhace program (available from CodonCode Corporation) (22). PHRED assigns a quality score for each called base on a logarithmic scale. The PHRED quality score estimates the confidence with which each base has been called. A PHRED score of 20 equates to 99% confidence (1% probability of error). We measured the quality of the data by calculating the number of bases that equaled or exceeded 3 thresholds: PHRED 20, 30, and 40 (99%, 99.9%, and 99.99% confidence, respectively).

Quality Read Length (QRL).
Sequence data files were assembled into contigs by use of Pregap4, a contig assembly program available as part of the Staden analysis package (23). Sequencing data at the beginning and end of each read were quality-clipped wherever a window of 30 bases dropped below a mean PHRED score of 20. The proportion of unclipped data was expressed as a percentage of the total possible read for the fragment, i.e., the number of bases from the end of the sequencing primer to the end of the fragment. The QRL is a measure of the proportion of each sequence read in a single orientation comprising good analytical quality data (minimum base-calling confidence of 99%).

Quality Read Overlap (QRO).
The QRL scores for each sample (from both forward and reverse sequence reads) were combined together to yield a region of data, the QRO, where good diagnostic quality data were obtained in both orientations. The QRO is expressed as a percentage of the maximum possible QRO, which is the number of nucleotides between the ends of the forward and reverse sequencing primers on the target DNA. The QRO represents the data from each pair of reads that can be considered as "reportable quality".


   Results
Top
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 References
 
A total of 64 laboratories from 21 countries (see Table 1Up in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol52/issue4 ) registered for the scheme, including 47 public- and 7 private-sector diagnostic laboratories, 4 public-sector academic laboratories, 2 private-sector research laboratories, and 4 commercial core sequencing facilities. The services offered by participating laboratories focus predominantly on mutation detection with a broad range in the scale of activity. There was significant bias toward sequencing platforms provided by a single manufacturer (Fig. 1 ). All 64 laboratories completed the online questionnaire and were provided with samples for the practical part of the scheme. Sixty-one (95%) returned their results and raw data by the closing date and were included in the scheme evaluation.


Figure 1
View larger version (25K):
[in this window]
[in a new window]
  		Figure 1. Sequencing platforms used by participating laboratories.

a Includes one each of the following instruments: ABI373, Alf-Express II, Beckman CEQ 8000, LICOR IR DNA 4200, MegaBASE 1000, MegaBASE 500, MJ BaseStation, and Pyrosequencer. b ABI (Applied Biosystems Incorporated).

genotyping
All laboratories obtained results from the materials provided. Thirty-six (59%) of the participants scored the maximum 12.00 marks for genotyping when measured against the marking schemes (Table 1Up ). Of the 28 (41%) laboratories that failed to obtain maximum marks, 19 (5%) genotyping errors were made from 346 genotypes analyzed; 10 of the genotyping errors (53%) were false-negative results and 9 (47%) were false-positive results. In one case, a laboratory recorded a false-positive mutation that was not reported by any other laboratory. A further 27 errors were made in the naming of the mutations (Table 2 ). When a laboratory had miscalled or missed an allele, we examined their electronic sequence data to establish whether the change was identifiable. In all cases, we could see the expected change in the electronic data provided by the laboratory. A genotype score was calculated as the sum of the total score divided by the maximum possible score (Fig. 2 ; also see Fig. 1 in the online Data Supplement). Seventeen (28%) laboratories had a score that fell below the mean (0.94).


View this table:
[in this window]
[in a new window]
  		Table 2. Breakdown of the genotyping results.


Figure 2
View larger version (26K):
[in this window]
[in a new window]
  		Figure 2. Laboratory genotyping performance rankings based on genotyping scores (calculated as the sum of the total score divided by the maximum possible score).

quality of sequence data
The output from 57 (93%) of the participating laboratories was analyzed. Data from 4 laboratories were excluded because the format used was incompatible with the analysis programs.

The majority of sequence data exceeded a PHRED 20 quality score (Fig. 3A ; also see Fig. 2A in the online Data Supplement). The proportion of base-calls meeting this standard averaged over all centers was 87.49% (range, 67.45%–95.89%). This was mirrored by the data for the PHRED 30 (Fig. 3B ; also see Fig. 2B in the online Data Supplement) and PHRED 40 (Fig. 3C ; also see Fig. 2C in the online Data Supplement) analyses, although a lower proportion of data produced by the centers exceeded these thresholds [mean scores of 79.77% (range, 46.28%–92.38%) and 63.78% (range, 15.07%–83.72%), respectively].


Figure 3
View larger version (70K):
[in this window]
[in a new window]
  		Figure 3. PHRED quality score distributions.

(A), PHRED 20 quality score distribution; (B), PHRED 30 quality score distribution; (C), PHRED 40 quality score distribution.

The distribution of QRL and QRO scores expressed as a percentage (Figs. 4 and 5 ; also see Figs. 3 and 4 in the online Data Supplement) broadly mirrored the distributions of the PHRED scores (mean score, 86.09%; range, 57.48%–99.78%). However, there was a more pronounced drop off at the lower end of the distribution. Data from this subset of laboratories were generally of lesser quality in both sequence orientations. Laboratories with PHRED 40 scores above the mean value tended to have good QRL and QRO scores (for example, data from laboratories 99 and 98). However, there were some exceptions to this correlation as some laboratories with poorer quality data as measured by the PHRED score thresholds had good QRL and QRO scores. For example, as can be seen in Fig. 3Up (also see Fig. 2 in the online Data Supplement for Laboratory ID codes), the quality of the sequence data from laboratory 34 as measured by PHRED was below average. However, as can be seen in Fig. 5 (see also Fig. 4 in the online Data Supplement), the QRO from this same laboratory was above average, indicating that the data must be of even quality in both orientations. To resolve this apparent anomaly, we contacted the laboratory directly. The combination of the chemistry, polymer, and capillary length they use is optimized to allow them to rapidly run several different types of assays at the same time. Although their protocol is not optimized to give "perfect" sequence data, it does provide consistent quality data with long read lengths in both orientations. Overall, the trend was for laboratories that produce low PHRED quality scores to also have low scores for QRL and QRO.


Figure 4
View larger version (26K):
[in this window]
[in a new window]
  		Figure 4. QRL distribution.


Figure 5
View larger version (26K):
[in this window]
[in a new window]
  		Figure 5. QRO distribution.

To allow laboratories to gauge their performance, we established a consensus standard for the quality of the sequencing data by ranking the performance of each participant against the %QRO (Fig. 6 ). To ensure that the data will be comparable in successive EQA schemes and to allow laboratories to determine their performance relative to their peers, we normalized the ranking on a centile scale of 0–100 and established decile performance indicators (D1 to D10). Including an identical fragment in successive EQA schemes will allow tracking of the aggregate standard of DNA sequencing among the participants.


Figure 6
View larger version (33K):
[in this window]
[in a new window]
  		Figure 6. Performance ranking of laboratories based on %QRO.

Decile performance indicators are shown as D1 through D10. Laboratories highlighted by an open arrow made genotyping errors leading to a false-negative result; filled arrows indicate false-positive genotyping errors; and shaded arrows indicate false-positive and -negative genotyping errors. x axis ranking for each data point is calculated as laboratory %QRO ranking/number of laboratories ranked x 100.

In Fig. 6Up , laboratories that made genotyping errors are highlighted by white (false-negative), black (false-positive), and black-and-white-shaded (false-positive and -negative) arrows. The spread of these laboratories across the ranked data set suggests that it is not the quality of data that leads to genotype errors. The most likely explanation is human error, failure of mutation scanning software, or a combination of both because many laboratories use a manual check after an initial pass through a software system. In all cases involving false negatives, the assessors were able to identify the change from the raw data.


   Discussion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
References
 
The reported EQA scheme establishes an objective method for judging the quality of DNA sequencing performance. One key problem identified was that of mutation naming. It was apparent that several different mutation nomenclature systems were used. To accommodate this, we allowed 3 alternative nomenclature systems. However, we recommended in the final report to the participants (24) that laboratories should follow the mutation nomenclature scheme produced by the Human Genome Variation Society (HGVS) (25). Although we appreciated that this could cause inconsistencies with preexisting locus-specific naming schemes, the HGVS recommendations are clear on this point, stating:

When new reports describe variants according to current recommendations, instead of using traditional descriptions, experts in the field experience problems "recognizing" these variants. However, nomenclature rules should be universal and thus can not be made to apply for specific situations.

We noticed a common error in mutation naming whereby laboratories incorrectly prefixed the mutation with a "c." instead of a "g." when classifying according to a genomic sequence. We did not deduct marks for this but recommended that laboratories take greater care when naming any changes they find in a DNA sequence. Twenty-seven (59%) of the errors identified were errors in nomenclature, indicating that this is an evolving and poorly standardized area. The remaining 19 (41%) errors were false-positive or -negative observations.

There appears to be no correlation between the quality of sequence data and the likelihood of a laboratory making a genotyping mistake (Fig. 6Up ). Identifying mutations is made difficult by the high information content of sequence data. The feature that sets DNA sequencing apart from other mutation scanning systems is the large number of data points to be analyzed. Electronic DNA sequencing data can be analyzed for mutations visually or by use of software. Visual inspection is generally accepted as being the "gold standard", but the mental effort needed means that only small amounts of data can be reliably analyzed. Many software packages have been designed for research applications, where false negatives are not as critical. The performance of automated systems, for example, SeqScape (26) and Mutation Surveyor (27), falls short of the needs of analytical laboratories. Laboratories should ensure that any software that they use to analyze sequence data is adequately validated for the intended end use of the data, especially in a clinical setting.

This EQA scheme measures quality and provides laboratories and core sequencing facilities with an indicator of performance. The lowest mean PHRED 20 score in this series (67.45%) indicated that for the 450-bp fragment sequenced, on average, 303 bp (67.45% x 450 bp) of the sequence had a base-calling error rate ≤1%. Therefore, by definition, 147 bp (32.55%) of that sequence had an error rate ≥1%. Given that more than 100 bp of data had been called at this level of error, it is highly likely that these data will contain base-calling errors. Assuming an error rate of 1% for the 147-bp poor-quality portion of the read, the likelihood of making at least one mistake in base-calling is 77.2% [probability calculated as 1 – (0.99147) x 100]. Sequence quality underlies base-calling accuracy, although base-calling errors need not necessarily lead to variant detection error. Normalizing the QRO ranking on a centile scale and using decile performance indicators allows laboratories to measure the improvement or deterioration of their performance relative to their peers in successive EQA schemes. The inclusion of one material common to successive schemes will allow aggregate performance to be followed (even if different sets of laboratories participate from scheme to scheme). These measures are suggested as a benchmark for laboratory performance in the analysis of DNA sequence.


   Acknowledgments
 
This work was supported by the UK Department of Health.


   Footnotes
 
1 Nonstandard abbreviations: EQA, external quality assessment; EMQN, European Molecular Genetics Quality Network; QRL, Quality Read Length; QRO, Quality Read Overlap; and HGVS, Human Genome Variation Society.


   References
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 References
 

  1. The United Kingdom National External Quality Assessment Scheme (UKNEQAS) for Molecular Genetics.http://www.ukneqas-molgen.org.uk (accessed October 2005)..
  2. College of American Pathologists (CAP).http://www.cap.org (accessed October 2005)..
  3. Quality Control in Molecular Diagnostics (QCMD).http://www.qcmd.org (accessed October 2005)..
  4. EC4-EQUAL project.http://www.ec-4.org/equal/home.htm (accessed October 2005)..
  5. European Cystic Fibrosis Thematic Network.http://www.cfnetwork.be/ (accessed October 2005)..
  6. The European Molecular Genetics Quality Network (EMQN).www.emqn.org (accessed October 2005)..
  7. National Genetics Reference Laboratory (Manchester).http://www.nrgl.org.uk/manchester.html (accessed October 2005)..
  8. Ibaretta D, Elles R, Cassiman JJ, Rodriguez-Cerezo E, Dequeker E. Towards quality assurance and harmonisation of genetic testing services in the European Union. Nat Biotechnol 2004;22:1230-1235.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  9. Mueller CR, Kristoffersson U, Stoppa-Lyonnet D. External quality assessment for mutation detection in the BRCA1 and BRCA2 genes: EMQN's experience of 3 years. Ann Oncol 2004;15(Suppl 1):114-117.
  10. Dequeker E, Cassiman JJ. Genetic testing and quality control in diagnostic laboratories. Nat Genet 2000;25:259-260.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  11. Dequeker E, Ramsden S, Grody WW, Stenzel TT, Barton DE. Quality control in molecular genetic testing. Nat Rev Genet 2001;2:717-723.[Web of Science][Medline] [Order article via Infotrieve]
  12. Preston FE, Kitchen S, Jennings I, Woods TA. A UK National External Quality Assessment scheme (UK NEQAS) for molecular genetic testing for the diagnosis of familial thrombophilia. Thromb Haemost 1999;82:1556-1557.[Web of Science][Medline] [Order article via Infotrieve]
  13. Valentine-Thon E, van Loon AM, Schirm J, Reid J, Klapper PE, Cleator GM. European proficiency testing program for molecular detection and quantification of hepatitis B virus DNA. J Clin Microbiol 2001;39:4407-4412.[Abstract/Free Full Text]
  14. Voelckel M-A, Ramsden S, Matthijs G. Draft best practice guidelines for the molecular diagnosis of Prader-Willi and Angelman syndromes (2002). Available on the EMQN website.http://www.emqn.org (accessed October 2005)..
  15. Simoni M, Bakker E, Krausz C. EAA/EMQN best practice guidelines for molecular diagnosis of Y-chromosomal microdeletions: state of the art 2004. Int J Androl 2004;27:240-245.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  16. . CasiniRaggi C, Pinzani P, Paradiso A, Pazzagli M, Orlando C. External quality assurance program for PCR amplification of genomic DNA: an Italian experience. Clin Chem 2003;49:782-791.[Abstract/Free Full Text]
  17. Orlando C, Casini Raggi C, Pazzagli M. Molecular diagnosis and quality control; a program for DNA analysis with PCR amplification. [Abstract]Clin Chem Lab Med 2000;7:A10.
  18. Braun A, Deufel T, Geilenkeuser WJ, Neumaier M, Rohle G, Roscher A, et al. External quality assessment of molecular biology-based methods used in laboratories of clinical chemistry and human genetics. Clin Chem Lab Med 1998;36:231-234.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  19. Neumaier M, Braun A, Gessner R, Funke H. Experiences with external quality assessment (EQA) in molecular diagnostics in clinical laboratories in Germany. Working Group of the German Societies of Clinical Chemistry (DGKC) and Laboratory Medicine (DGLM). Clin Chem Lab Med 2000;38:161-163.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  20. Birch L, English CA, Burns M, Keer JT. Generic scheme for independent performance assessment in the molecular biology laboratory. Clin Chem 2004;50:1553-1559.[Abstract/Free Full Text]
  21. Patton S, Taylor G. Final report of a pilot EQA scheme for DNA sequencing (2002). Available on the EMQN website.http://www.emqn.org (accessed October 2005)..
  22. CodonCode Corporation.http://www.codoncode.com (accessed October 2005)..
  23. SourceForge.nethttp://staden.sourceforge.net (accessed October 2005)..
  24. Patton S, Wallace A. Final report: EMQN pilot external quality assessment scheme for DNA sequencing (2003). Available on the EMQN website http://www.emqn.org (accessed October 2005)..
  25. Human Genome Variation Society (HGVS).http://www.hgvs.org (accessed October 2005)..
  26. Patel Y, Wallace A. Mutation scanning—Applied Biosystems VariantSEQr and SeqScape v2.1—an assessment using a model system [Technology Assessment Report]. September 2005. Available on the NGRL website. http://www.ngrl.org.uk/Manchester (accessed October 2005)..
  27. Patel Y, Wallace A. Automated mutation detection using SoftGenetics Mutation SurveyorTM v2.51 [Technology Assessment Report]. September 2005. Available on the NGRL website. http://www.ngrl.org.uk/Manchester (accessed October 2005)..



The following articles in journals at HighWire Press have cited this article:


Home page
 J. Clin. Microbiol.Home page
B. Afshar, N. K. Fry, W. Bellamy, A. P. Underwood, T. G. Harrison, and and Members of the European Working Group for Legi
External Quality Assessment of a DNA Sequence-Based Scheme for Epidemiological Typing of Legionella pneumophila by an International Network of Laboratories
J. Clin. Microbiol., October 1, 2007; 45(10): 3251 - 3256.
[Abstract] [Full Text] [PDF]

Home page
 Clin. Chem.Home page
C. Orlando, P. Verderio, R. Maatman, J. Danneberg, S. Ramsden, M. Neumaier, D. Taruscio, V. Falbo, R. Jansen, C. Casini-Raggi, et al.
EQUAL-qual: A European Program for External Quality Assessment of Genomic DNA Extraction and PCR Amplification
Clin. Chem., July 1, 2007; 53(7): 1349 - 1357.
[Abstract] [Full Text] [PDF]

Home page
 Nucleic Acids ResHome page
P. Sonego, M. Pacurar, S. Dhir, A. Kertesz-Farkas, A. Kocsor, Z. Gaspari, J. A.M. Leunissen, and S. Pongor
A Protein Classification Benchmark collection for machine learning
Nucleic Acids Res., January 12, 2007; 35(suppl_1): D232 - D236.
[Abstract] [Full Text] [PDF]

Home page
 Clin. Chem.Home page
E. Bakker
Is the DNA sequence the gold standard in genetic testing? Quality of molecular genetic tests assessed.
Clin. Chem., April 1, 2006; 52(4): 557 - 558.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Data Supplements
Right arrow All Versions of this Article:
clinchem.2005.061887v1
52/4/728    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (6)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Patton, S. J.
Right arrow Articles by Elles, R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Patton, S. J.
Right arrow Articles by Elles, R.
Related Collections
Right arrow Molecular Diagnostics and Genetics
Right arrow Laboratory Management

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS