Clinical Chemistry
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Clinical Chemistry 54: 1705-1715, 2008. First published August 21, 2008; 10.1373/clinchem.2008.108506
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Data
Right arrow All Versions of this Article:
clinchem.2008.108506v1
54/10/1705    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kohlmann, A.
Right arrow Articles by Haferlach, T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kohlmann, A.
Right arrow Articles by Haferlach, T.
(Clinical Chemistry. 2008;54:1705-1715.)
© 2008 American Association for Clinical Chemistry, Inc.

Cancer Diagnostics

Intraplatform Reproducibility and Technical Precision of Gene Expression Profiling in 4 Laboratories Investigating 160 Leukemia Samples: The DACH Study

Alexander Kohlmann1,a, Elisabeth Haschke-Becher2, Barbara Wimmer2, Ariana Huber-Wechselberger2, Sandrine Meyer-Monard3, Heike Huxol3, Uwe Siegler3, Michel Rossier4, Thomas Matthes4, Michela Rebsamen4, Alberto Chiappe4, Adeline Diemand4, Sonja Rauhut1, Andrea Johnson5, Wei-min Liu5, P. Mickey Williams5, Lothar Wieczorek5 and Torsten Haferlach1

1 Munich Leukemia Laboratory, Munich, Germany; 2 Institut für Medizinische und Chemische Labordiagnostik, Allgem. öffentliches KH der Elisabethinen, Linz, Austria; 3 Hematology Department, University Hospital Basel, Basel, Switzerland; 4 Service de Médecine de Laboratoire, Hôpitaux Universitaires de Genève, Genève, Switzerland;5 Roche Molecular Systems, Inc., Pleasanton, CA.

aAddress correspondence to this author at: MLL Münchner Leukämielabor GmbH, Max-Lebsche-Platz 31, 81377 München, Germany. Fax +49-89-990-17-389; e-mail alexander.kohlmann{at}mll-online.com.


   Abstract
Top
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 References
 
Background: Gene expression profiling has the potential to offer consistent, objective diagnostic test results once a standardized protocol has been established. We investigated the robustness, precision, and reproducibility of microarray technology.

Methods: One hundred sixty individual patient samples representing 11 subtypes of acute and chronic leukemias, myelodysplastic syndromes, and nonleukemia as a control group were centrally collected and diagnosed as part of the daily routine in the Munich Leukemia Laboratory. The custom AmpliChip Leukemia research microarray was used for technical analyses of quadruplicate mononuclear cell lysates in 4 different laboratories in Germany (D), Austria (A), and Switzerland (CH) (the DACH study).

Results: Total-RNA preparations were successfully performed in 637 (99.5%) of 640 cases. Mean differences between pairs of laboratories in the total-RNA yield from the same sample ranged from 0.02 µg to 1.03 µg. Further processing produced 622 successful in vitro transcription reactions (97.6%); the mean differences between laboratories in the cRNA yield from the same sample ranged from 0.40 µg to 6.18 µg. After hybridization to microarrays, a mean of 47.6%, 46.5%, 46.2%, and 46.4% of probe sets were detected as present for the 4 laboratories, with mean signal-intensity scaling factors of 3.1, 3.7, 4.0, and 4.2, respectively. In unsupervised hierarchical cluster and principal component analyses, replicates from the same patient always clustered closely together, with no indications of any association between gene expression profiles due to different operators or laboratories.

Conclusions: Microarray analysis can be performed with high interlaboratory reproducibility and with comparable quality and high technical precision across laboratories.


   Introduction
Top
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 References
 
Microarray analysis has been used in multiple studies to identify differentially expressed genes associated with distinct clinical and therapeutically relevant classes of leukemias(1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14). The data are limited, however, on the actual robustness and performance of gene expression profiling for detailed diagnostic workups, not only for both acute and chronic leukemias in particular but also for the broader use of microarray technology in a clinical laboratory setting. As a prerequisite to the application of these approaches, sample-preparation procedures must be adequately standardized and simplified for routine use in a clinical laboratory. In addition, the classification performance of microarray assays must be proved in prospective studies that incorporate both separate and adequately powered independent patient cohorts for algorithm development and validation(15).

Thus far, variables such as shipment time and isolation procedures for total RNA have been shown not to substantially influence the underlying diagnostic gene expression signatures in a patient sample, and such procedures have led to robust results(16)(17). Such information does not abrogate the need for carefully designed clinical research studies, because preanalytical variables are known to influence gene expression profiles, as has been demonstrated for various types of sample-purification and cell-fractionation techniques(18)(19)(20)(21). Therefore, consensus guidelines for microarray gene expression analyses, such as those published by 3 hematology networks (the International BFM Study Group, the German Competence Network "Acute and Chronic Leukemias," and the European LeukemiaNet), are helpful but represent just a starting point for the task of incorporating molecular profiling into clinical practice(22)(23).

We report the technical precision of gene expression signatures after analyzing quadruplicate lysates of mononuclear cells from leukemia patients and control individuals on a customized microarray designed for the subclassification of leukemias(24). After central collection and diagnostic workup of 160 individual patient samples representing 11 distinct leukemia subclasses, myelodysplastic syndromes, and nonleukemia controls, a baseline microarray analysis was performed in a hematology laboratory experienced with this technology. Three additional identical sets of replicate samples were then blinded and shipped to 3 independent laboratories to repeat the analyses of these biological samples and to assess the reproducibility of microarray analysis.


   Materials and Methods
Top
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 References
 
study design
One hundred sixty different samples of bone marrow (n = 115) or peripheral blood (n = 45) were obtained from individual untreated patients in the Munich Leukemia Laboratory (see Table S1 in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol54/issue10). All patients gave their informed consent for participation after having been advised of the purpose and investigational nature of the study. The study design adhered to the tenets of the Declaration of Helsinki and was approved before its initiation by the ethics committees of the participating institutions. The diagnostic workup of leukemia was performed in Munich, Germany, as previously described(25)(26)(27)(28)(29)(30)(31). Full sample details are given in the online Data Supplement. For gene expression analysis, mononuclear cells were purified with Ficoll density-gradient centrifugation, and replicate lysates of 5 x 106 cells each were prepared (Buffer RLT; Qiagen). Four replicates of the mononuclear cell lysates were prepared for each patient. Each participating laboratory (in Basel, Linz, Geneva, and Munich) was provided with one frozen mononuclear cell lysate from each patient for microarray analysis. Each participating laboratory designated specific operators for conducting the study. Three of the 4 laboratories had no previous experience with the Affymetrix gene expression microarray sample-preparation assay (Linz, Basel, and Geneva). Thus, the operators in these laboratories were introduced to this technology in a 5-day training course on a standardized sample-preparation protocol for microarray analysis that used commercially available total RNA from cancer cell lines MCF-7 and HepG2 (Ambion). Furthermore, each participating center was provided with identical laboratory supplies and equipment for microarray analysis, including reagent kits, enzymes, spectrophotometers, and heat block instruments.

microarray analysis
After shipment on dry ice, total RNA was extracted from the homogenized patient lysates and processed with a previously described protocol (RNeasy Kit; Qiagen) at each laboratory between March and August 2007(16). The Affymetrix HG-U133 Plus 2.0 microarrays were used for training the operators in the assay, and the custom AmpliChip Leukemia research microarray (Roche Molecular Systems) was used for patient samples. The AmpliChip Leukemia research microarray was specifically designed for the classification of leukemias(32). The chip contains 1 480 distinct probe sets with an 11-µm feature size. Of these probe sets, 1 457 are used for generating normalized signal intensities of disease-related genes; the remaining 23 probe sets interrogate control sequences and housekeeping genes. Full details of the sample-preparation workflow, as well as a gene-by-gene ANOVA of the custom microarray probe sets, are given in data files in the online Data Supplement.

quality report and statistical methods
A total of 637 AmpliChip Leukemia microarray gene expression profiles were included in the analysis. A detailed data-quality report was generated for each gene expression profile to define the overall quality of each microarray experiment (see the online Data Supplement). The quality thresholds for passing the study’s quality criteria were defined as: (a) a purified total-RNA amount of >1.0 µg per sample, (b) a cRNA yield for each target preparation of >8.0 µg, (c) a percentage of probe sets called as present of >20%(33), and (d) a scaling factor of <10.0.

Preprocessing the data after imaging included a summarization and quantile-normalization step to generate probe set–level signal intensities for each microarray experiment; this step was performed as previously described(34). The expression signals of a microarray were normalized to the quantiles of a predefined distribution (β distribution with parameters p = 1.2 and q = 3). To assess the interlaboratory consistency and reproducibility of the gene expression microarray analysis, we generated Bland–Altman plots(35). To assess whether systematic differences in total-RNA and cRNA yields existed among centers, we performed a repeated-measures regression analysis for each yield variable as an outcome. We assumed an unstructured covariance structure common to each patient, allowing for different variances for each center and different covariances between each pair of centers. We further assumed that yields from different patient samples were uncorrelated. Tests of means of yields were performed with variances estimated with this model, and P values were corrected with the Bonferroni correction for multiple testing.

Data visualization and exploratory analyses, such as box plots, unsupervised hierarchical clustering, and principal components analysis, were performed with R software versions 2.4.1 and 2.5.1 (http://www.R-project.org), which included the affy, made4, and Heatplus packages(36). The repeated-measures analysis was performed with SAS version 9.1 and the PROC MIXED analysis routine (SAS Institute). All microarray data are available in files in the online Data Supplement.


   Results
Top
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 References
 
assessment of sample-preparation quality
Upon receipt of the mononuclear cell lysates, each of the participating laboratories purified comparable amounts of total RNA from the 160 quadruplicate samples (Fig. 1A ; detailed summary in the online Data Supplement). At least 1.0 µg of total RNA was obtained from 637 (99.5%) of the 640 preparations. Table S2 in the online Data Supplement summarizes the estimates of variance, ranges (minimum, maximum), and median total-RNA yields for each of the 4 centers. The 4 laboratories isolated a mean of 7.2 µg, 7.5 µg, 7.6 µg, and 8.2 µg of total RNA from mononuclear cells. Three preparations from one center failed to yield total-RNA amounts adequate for microarray analyses (Table S3 in the online Data Supplement). The laboratory in Munich produced slightly higher total-RNA yields than the other 3 laboratories.


Figure 1
View larger version (22K):
[in this window]
[in a new window]

  		Figure 1. Analysis of basic sample-preparation variables.

Each panel shows the distributions of a sample-preparation variable for data from the 4 laboratories (BAS, Basel; GEN, Geneva; LIN, Linz; MUC, Munich). (A), total-RNA yield after purification of 160 quadruplicate sample lysates. (B), total-RNA input for the cDNA synthesis reaction. (C), cRNA yield after the in vitro transcription reaction. (D), cRNA input used for the fragmentation assay. Note: Three total-RNA sample purifications failed at the Linz center; therefore, process data presented for the Linz center in (B), (C), and (D) are for 157 sample preparations.

We next assessed the impact of the input total-RNA quantity on overall assay performance and consistency across laboratories. The current version of the sample-preparation protocol does not specify a fixed total-RNA amount but rather permits a total-RNA input range of 1.0–8.0 µg for the cDNA-synthesis step. Fig. 1BUp shows that the 4 laboratories used varying amounts of total RNA for cDNA synthesis. For example, the Linz center intentionally added no more than 3.0 µg total RNA into the sample-preparation workflow, yet the cRNA yields from this center were comparable to those of the other centers (Fig. 1CUp ). Of the 637 processed total-RNA samples, 622 (97.6%) produced successful in vitro transcription reactions, i.e., a cRNA yield >8.0 µg. Table S2 in the online Data Supplement summarizes the variance estimates, ranges, and medians for cRNA yields obtained from each of the 4 centers. Fifteen preparations from 11 patient lysates produced <8.0 µg of cRNA and thus were considered as failed preparations (see Table S4 in the online Data Supplement). Although the preparation for one sample (no. 112) failed in the majority of the laboratories, those for the other samples failed primarily in one of the 4 centers.

Finally, the recommended cRNA target amount of 11.0 µg for the fragmentation assay was achieved for most samples (Fig. 1DUp ). For 40 patient samples (58 sample preparations total), <11.0 µg of cRNA was used as the input into the fragmentation step in preparation for hybridization to the microarray. The online Data Supplement lists the cRNA amounts for the individual samples.

assessment of sample-preparation reproducibility
Bland–Altman analyses were performed to investigate whether systematic differences in total-RNA yields existed among laboratories for the same sample (Fig. 2 ). For example, each dot in the comparison of the Munich and Geneva laboratory results represents a patient sample. The mean total yield in micrograms for the 2 total-RNA preparations on the x axis is plotted against the difference in micrograms between the 2 laboratories on the y axis. For this example, there is a 95% chance that measurements of the same sample from these 2 laboratories will be within 5.8 µg of each other. This 95% interval of agreement ranged from –2.24 µg to 3.55 µg, with a mean interval of agreement of 0.65 µg. More detailed information and raw values for all 6 pairwise comparisons of the centers are presented in Table S5 and Fig. 1A in the online Data Supplement. For all total-RNA comparisons, any 2 centers are expected to have no more than a 7.95-µg difference in total-RNA yield for the same sample. The mean observed difference in total-RNA yield between 2 centers for a given patient sample ranged from 0.02 µg to 1.03 µg.


Figure 2
View larger version (37K):
[in this window]
[in a new window]

  		Figure 2. Plots of pairwise laboratory comparisons of total-RNA yield for each patient sample.

Each Bland–Altman plot is a visual representation of the consistency of the total-RNA results between any 2 centers. For each comparison, the mean value of the 2 total-RNA purification results from a single patient sample at the 2 centers (x axis) is plotted against the absolute difference between the same 2 results (y axis). The solid horizontal line represents the overall mean of the differences, and the dashed lines show the range containing the mean of the differences ± 2 SDs, which is referred to as the limits of agreement. Depicted in each plot are the data for 160 patient samples. Laboratory abbreviations: BAS, Basel; GEN, Geneva; LIN, Linz; MUC, Munich.

Similarly, we also generated Bland–Altman plots for the cRNA yields to investigate whether systematic differences in cRNA yields had arisen among laboratories for the same sample (Fig. 3 ). For example, comparing the results for the Munich and Geneva laboratories shows that there is a 95% chance that measurements of the same sample from these 2 laboratories will be within 39.52 µg of each other. This 95% interval of agreement ranged from –19.01 µg to 20.51 µg, with a mean interval of agreement of 0.75 µg. More detailed information and raw values for all 6 pairwise comparisons of the centers are presented in Table S6 and Fig. 1BUp in the online Data Supplement. In summary, any 2 centers would be expected to have a difference in cRNA yield of <49.38 µg for the same sample. The mean observed difference between 2 centers in cRNA yield for a given patient sample ranged from 0.40 µg to 6.18 µg.


Figure 3
View larger version (39K):
[in this window]
[in a new window]

  		Figure 3. Plots of pairwise laboratory comparisons of cRNA yield for each patient sample.

Each Bland–Altman plot is a visual representation of the consistency between any 2 centers in the cRNA results. For each comparison, the mean value of the 2 cRNA purification results from a single patient sample at the 2 centers (x axis) is plotted against the absolute difference between the same 2 results (y axis). The solid horizontal line represents the overall mean of the differences, and the dashed lines show the range containing the mean of the differences ± 2 SDs, which is referred to as the limits of agreement. Depicted in each plot are the data for 160 patient samples. Laboratory abbreviations: BAS, Basel; GEN, Geneva; LIN, Linz; MUC, Munich.

assessment of microarray data quality
In further processing the sample replicates, we hybridized the 622 preparations with successful in vitro transcription reactions and adequate cRNA yields to AmpliChip Leukemia microarrays, and we evaluated various general microarray data-quality variables. Overall, all 4 laboratories generated very reproducible data sets with respect to the percentage of probe sets called as present, the scaling factor, and the ratio of the intensities of 3' probes to 5' probes for the housekeeping gene GAPDH1 (glyceraldehyde-3-phosphate dehydrogenase) (Fig. 4 ). In addition, we processed the 15 preparations from 11 patient lysates (lysate nos. 4, 26, 46, 67, 93, 102, 109, 112, 113, 122, and 146) with inadequate cRNA yields (Table S4 in the online Data Supplement) to microarrays to test whether the various sample-preparation cutoff criteria were useful predictors of subsequent microarray data of poor quality.


Figure 4
View larger version (26K):
[in this window]
[in a new window]

  		Figure 4. Analysis of microarray QC variables.

Sample sizes were 160 for the Basel (BAS), Geneva (GEN), and Munich (MUC) laboratories and 157 for the Linz (LIN) laboratory. The horizontal dashed red lines indicate QC cutoff levels. (A), Percentage of probe sets called as present (%P) on the AmpliChip Leukemia microarray. The cutoff level is set at 20.0%. (B), Scaling factor. The cutoff level is set at 10.0. (C), Ratio of intensities of 3' probes to 5' probes for the housekeeping gene GAPDH. The cutoff level is set at 3.0. (D), Distributions of a modified CV over all probe sets for the samples that passed QC criteria at all centers. The quantile-normalized signal intensity was used to generate this plot.

After their hybridization to microarrays, we detected a mean of 47.6%, 46.5%, 46.2%, and 46.4% of the probe sets as present at the Basel, Geneva, Linz, and Munich centers, respectively (Fig. 4AUp ). One sample consistently did not pass the probe-set call filter of >20.0% present. This sample (no. 112) also had previously not passed the cRNA-yield filter in the majority of the centers. As demonstrated in Fig. 4BUp , the data sets from the 4 laboratories also had comparable scaling factors, with mean values of 3.1, 3.7, 4.0, and 4.2 for the Basel, Geneva, Linz, and Munich centers, respectively. Again, sample no. 112 was different, with scaling factors clearly higher than the cutoff limit of 10.0. Two other samples from the Linz center also showed scaling factors >10.0, and these samples were examined in more detail. Both samples had low cRNA yields (2.6 µg for sample no. 67; 4.3 µg for sample no. 93) and thus usually would not have been recommended for hybridization to a microarray. In fact, these samples had almost identical total-RNA input amounts and cRNA yields after the in vitro transcription reaction. Sample no. 26 prepared by the Munich laboratory also had a scaling factor above the cutoff limit and at 13.3 µg clearly had passed the recommended cRNA yield. This result indicates the need to carefully integrate several sample-preparation variables into a final quality result to decide whether a sample can be hybridized to a microarray.

Another QC variable, the so-called 3'/5' GAPDH ratio, is presented in Fig. 4CUp . The signal-intensity ratios of GAPDH probes located at both the 3' and 5' positions of the gene frequently are used as indicators of the integrity of the starting total RNA. Although the manufacturer has specified no absolute cutoff value, many microarray studies have referred to a ratio of <3.0(23). In the present study, 612 of 637 samples had ratios <3.0, and only 2 microarray data sets showed ratios >4.0. These results indicate the high quality of the starting material collected during the routine diagnostic workups.

A final microarray data-quality variable is the distribution of the CV within each laboratory for the quantile-normalized signals over all of the probe sets represented on the AmpliChip Leukemia microarray (Fig. 4DUp ). All participating laboratories generated remarkably comparable signal-intensity distributions after normalization, with mean (SD) values of 27.6% (1.5%), 27.5% (1.6%), 27.7% (1.5%), and 27.5% (1.5%) for the Basel, Geneva, Linz, and Munich laboratories, respectively.

interlaboratory similarity in gene expression profiles
We next investigated the similarity in the generated gene expression profiles with unsupervised hierarchical clustering. In total, 160 quadruplicates of mononuclear cell lysates were provided to the 4 laboratories. In 157 cases, quadruplicate gene expression profiles were generated in all of the centers. Of these cases, case no. 112 consistently showed low quality; therefore, all replicates from this sample were excluded from further analyses. Thus, of the 160 samples in this study, only 4 samples were excluded on the basis of these criteria, with each laboratory having shown failures. The hierarchical clustering analysis was based on 1 457 probe sets that were represented on the AmpliChip Leukemia microarray. As depicted in the unsupervised clustering dendrogram in Fig. 5 , the 156 quadruplicate samples are split into several major branches. A major branch was seen at the left, where all chronic lymphocytic leukemias (CLLs) grouped together (branch 1). Next to the CLL samples were 3 major branches that contained the majority of acute leukemias (branch 2), chronic myeloid leukemias (branch 3), and myelodysplastic syndromes and samples from nonleukemia patients (branch 4). In all cases, the quadruplicates for the individual patients were the most similar to each other and clustered together. There was no indication of any association between gene expression profiles due to different operators or laboratories. This finding is visualized more clearly in an enlarged section of the CLL branch (Fig. 5 ).


Figure 5
View larger version (49K):
[in this window]
[in a new window]

  		Figure 5. Unsupervised hierarchical clustering.

The unsupervised clustering of the gene expression profiles of all samples was performed with the 1 457 probe sets on the AmpliChip Leukemia microarray for the 624 experiments included in the study that passed the quality filter. We excluded sample nos. 65, 66, and 68 because they failed the total-RNA purification sample-preparation step at one center (Linz). Sample no. 112 was also excluded because it failed multiple microarray QC criteria at several centers. Quantile-normalized signals were used to cluster the profiles. The similarity of samples was computed by Euclidean distance, and Ward’s method was then used to cluster the gene expression profiles on the basis of these measures. The upper part of the figure shows a dendrogram of all the samples. The algorithm identified 4 major branches that represent CLL (branch 1), the majority of acute leukemias (branch 2), chronic myeloid leukemias (branch 3), and myelodysplastic syndromes and samples from nonleukemia patients (branch 4). The lower part of the figure shows an enlarged section of the CLL branch. Each line is annotated by sample class, the laboratory site of the analysis, corresponding colored sample identifier, and sample type. The files in the online Data Supplement include a detailed worksheet that identifies and annotates the individual lines of the dendrogram. BAS, Basel; GEN, Geneva; LIN, Linz; MUC, Munich; AML, acute myeloid leukemia; c-ALL, common acute lymphoblastic leukemia; Pre-B-ALL, pre–B-cell acute lymphoblastic leukemia; CML, chronic myeloid leukemia; MDS, myelodysplastic syndrome; Pro-B-ALL, pro–B-cell ALL.

In addition, we performed principal component analysis as another way to visualize the similarity of profiles. As is shown in Fig. 6 , the individual gene expression profiles of the quadruplicates grouped closely together. As exemplified by the samples representing CLL, chronic myeloid leukemia, pro–B-cell acute lymphoblastic leukemia with t(11q23)/mixed-lineage leukemia (trithorax homolog, Drosophila) (MLL), and acute myeloid leukemia with t(15;17), corresponding types of biological samples grouped according to their underlying similarity (Fig. 6A ) but not according to the center where the microarray experiments were performed or the operators who performed the analyses (Fig. 6B ).


Figure 6
View larger version (16K):
[in this window]
[in a new window]

  		Figure 6. Visualization of the first 2 components of a principal component analysis.

Each data point represents the gene expression profile of a single sample. Data points are colored according to the sample type (A) or center location (B). The analysis focuses, as an example, on patient samples for CLL; chronic myeloid leukemia (CML); pro–B-cell acute lymphoblastic leukemia with t(11q23)/MLL (Pro-B-ALL with t(11q23)/MLL); and acute myeloid leukemia (AML) with t(15;17). The first 2 principal components account for 83.8% of the variation in the data (component 1, 55.7%; component 2, 28.1%). See Fig. 2Up in the online Data Supplement for a 3-dimensional representation. GEN, Geneva; LIN, Linz; BAS, Basel; MUC, Munich.

Finally, the relationships between gene expression profiles within sample types can be visualized by pairwise classification analyses, in which various sets of training data have been used to develop a classifier for distinguishing between the distinct leukemia subtypes(7)(24). For example, when we applied a prediction algorithm to classify the 4 quadruplicates of sample no. 142 [a case of pro–B-cell acute lymphoblastic leukemia with t(11q23)/MLL; class 2] from the respective DACH study centers as independent test samples, the classifier makes consistent calls for the quadruplicates, and all testing-data gene expression profiles get the correct call for class 2 (see Fig. 3 in the online Data Supplement). Again, this result underlined the finding that data variation due to different preparation locations had less influence on the generated gene expression profiles than the type of biological sample.


   Discussion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
References
 
In this study, we investigated the robustness, precision, and reproducibility of microarray gene expression profiling. Overall, only a few patient samples failed the complete sample-preparation workflow. For example, one set of chronic myeloid leukemia replicates (sample no. 112) consistently failed the target cRNA-yield amount at all 4 laboratories. The replicates also had poor values for the percentage of probe sets called as present and for scaling factors. When we examined the patient record in more detail, it became clear that the shipment period for this particular sample was 5 days, from bone marrow biopsy to entry into the laboratory. Although diagnostic signatures have been reported to be robust and stable after 24 h of shipment time for samples of certain types of acute leukemias(17), further investigations of more types of hematologic malignancies will be required to define clear limits for sample-shipment times. Because most hematologic diagnostic centers are open at least 6 days a week, a limit of 48 h between the time of biopsy and receipt of the sample seems reasonable as a criterion for entry into the microarray sample-preparation workflow. With regard to other preparation failures, we noted no other obvious correlations with respect to sample type, laboratory, or operator. Given that many preparations failed in only one laboratory and that we generally prepared samples in batches of 8 with the same master mixes, one can only speculate about the reasons for these rare failures. Some of these preparations failed only at the level of cRNA yield. In this study, the cRNA-yield cutoff value was set rather stringently at 8.0 µg; however, many of the samples with low cRNA yields still demonstrated acceptable percentages of genes called as present and acceptable scaling factors (sample nos. 4, 46, 102, 109, 113, 122, and 146). Thus, this finding indicates that multiple variables need to be factored into a conclusion on whether a sample is suitable for hybridization to a microarray and whether the generated microarray data can be used for classification analysis.

With respect to the available data on interlaboratory comparability and reproducibility of gene expression signatures, a few studies will be mentioned briefly. To examine the imprecision among replicates from a given tissue sample, Dobbin et al. designed a study of selected lung adenocarcinoma tumor cell lines, extracted samples of tumor RNA, and 12 large and histologically homogeneous tissue samples (predominantly tumors)(37). Sample replicates were randomized and distributed to 4 different laboratories for subsequent processing and microarray analysis. The comparability between laboratories seemed generally lower than within laboratories for all 3 sample types; however, the loss in comparability seemed to be fairly minor, and high intralaboratory and interlaboratory correlations were observed overall. Second, data from the MicroArray Quality Consortia (MAQC), a community-wide effort initiated and led by US Food and Drug Administration scientists and involving 137 participants from 51 organizations, demonstrated promising results with respect to the consistency of microarray data(38). The results demonstrated both intraplatform consistency within and across multiple test sites and a high level of interplatform concordance with respect to the genes identified as differentially expressed in different RNA samples. More recently, Dumur et al. investigated the interlaboratory performance of a novel microarray-based gene expression test to determine the tissue of origin in poorly differentiated and undifferentiated cancers(39). In this study, multiple laboratories were recruited to analyze replicate portions of various tumor tissues. Interestingly, even without standardizing preanalytical conditions, such as reagents for sample preparation, instrumentation, and protocols, cross-laboratory comparisons showed highly reproducible results.

In conclusion, the high number of successful sample preparations and the high-quality microarray data we obtained from technical quadruplicates confirmed that microarrays can offer consistent objective diagnostic test results once a standardized protocol has been established and is followed by trained operators. This study is a first step toward the implementation of high-density gene expression microarrays into clinical practice. A larger study is currently under way to address the actual assay performance in terms of classification accuracy, sensitivity, and specificity of a microarray-based stratification of leukemia subtypes(32).


   Acknowledgments
 
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.

Authors’ Disclosures of Potential Conflicts of Interest: Upon submission, all authors completed the Disclosures of Potential Conflict of Interest form. Potential conflicts of interest:

Employment or Leadership: Alexander Kohlmann, Munich Leukemia Laboratory; Wei-min Liu, Roche Molecular Systems, Inc; P. Mickey Williams, Roche Molecular Systems, Inc.; Lothar Wieczorek, Roche Molecular Systems, Inc.; Torsten Haferlach, Munich Leukemia Laboratory.

Consultant or Advisory Role: Torsten Haferlach, F. Hoffman-LaRoche Ltd., Basel, Switzerland.

Stock Ownership: None declared.

Honoraria: None declared.

Research Funding: Michel Rossier, one-year research agreement with Roche for another study unrelated to the present manuscript.

Expert Testimony: None declared.

Role of Sponsor: The funding organization played a direct role in the design of study, review of, and interpretation of data. Necessary study supplies and bench-top instruments, as well as sample preparation training courses, were sponsored by Roche Molecular Systems, Inc., Pleasanton, CA.

Acknowledgments: We thank Wen Wei and Julie Tsai for their contribution in assay development and operator training, Karen Yu, James Sun, and Li Qiu for their contribution in data management, as well as Xiaoying Chen for analyzing the operator proficiency microarray data sets.


   Footnotes
 
1 Human genes: GAPDH, glyceraldehyde-3-phosphate dehydrogenase; MLL, mixed-lineage leukemia (trithorax homolog, Drosophila).


   References
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 References
 

  1. Alcalay M, Tiacci E, Bergomas R, Bigerna B, Venturini E, Minardi SP, et al. Acute myeloid leukemia bearing cytoplasmic nucleophosmin (NPMc+ AML) shows a distinct gene expression profile characterized by up-regulation of genes involved in stem-cell maintenance. Blood 2005;106:899-902.[Abstract/Free Full Text]
  2. Armstrong SA, Staunton JE, Silverman LB, Pieters R, Den Boer ML, Minden MD, et al. MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat Genet 2002;30:41-47.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  3. Bullinger L, Dohner K, Bair E, Frohling S, Schlenk RF, Tibshirani R, et al. Use of gene-expression profiling to identify prognostic subclasses in adult acute myeloid leukemia. N Engl J Med 2004;350:1605-1616.[Abstract/Free Full Text]
  4. Chiaretti S, Li X, Gentleman R, Vitale A, Vignetti M, Mandelli F, et al. Gene expression profile of adult T-cell acute lymphocytic leukemia identifies distinct subsets of patients with different response to therapy and survival. Blood 2004;103:2771-2778.[Abstract/Free Full Text]
  5. Chiaretti S, Li X, Gentleman R, Vitale A, Wang KS, Mandelli F, et al. Gene expression profiles of B-lineage adult acute lymphocytic leukemia reveal genetic patterns that identify lineage derivation and distinct mechanisms of transformation. Clin Cancer Res 2005;11:7209-7219.[Abstract/Free Full Text]
  6. Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999;286:531-537.[Abstract/Free Full Text]
  7. Haferlach T, Kohlmann A, Schnittger S, Dugas M, Hiddemann W, Kern W, et al. Global approach to the diagnosis of leukemia using gene expression profiling. Blood 2005;106:1189-1198.[Abstract/Free Full Text]
  8. Kohlmann A, Schoch C, Schnittger S, Dugas M, Hiddemann W, Kern W, et al. Molecular characterization of acute leukemias by use of microarray technology. Genes Chromosomes Cancer 2003;37:396-405.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  9. Kohlmann A, Schoch C, Schnittger S, Dugas M, Hiddemann W, Kern W, et al. Pediatric acute lymphoblastic leukemia (ALL) gene expression signatures classify an independent cohort of adult ALL patients. Leukemia 2004;18:63-71.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  10. Ross ME, Zhou X, Song G, Shurtleff SA, Girtman K, Williams WK, et al. Classification of pediatric acute lymphoblastic leukemia by gene expression profiling. Blood 2003;102:2951-2959.[Abstract/Free Full Text]
  11. Ross ME, Mahfouz R, Onciu M, Liu HC, Zhou X, Song G, et al. Gene expression profiling of pediatric acute myelogenous leukemia. Blood 2004;104:3679-3687.[Abstract/Free Full Text]
  12. Schoch C, Kohlmann A, Schnittger S, Brors B, Dugas M, Mergenthaler S, et al. Acute myeloid leukemias with reciprocal rearrangements can be distinguished by specific gene expression profiles. Proc Natl Acad Sci U S A 2002;99:10008-10013.[Abstract/Free Full Text]
  13. Valk PJ, Verhaak RG, Beijen MA, Erpelinck CA, Barjesteh van Waalwijk van Doorn-Khosrovani S, Boer JM, et al. Prognostically useful gene-expression profiles in acute myeloid leukemia. N Engl J Med 2004;350:1617-1628.[Abstract/Free Full Text]
  14. Yeoh EJ, Ross ME, Shurtleff SA, Williams WK, Patel D, Mahfouz R, et al. Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. Cancer Cell 2002;1:133-143.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  15. Simon R, Radmacher MD, Dobbin K, McShane LM. Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J Natl Cancer Inst 2003;95:14-18.[Free Full Text]
  16. Campo Dell'Orto M, Zangrando A, Trentin L, Li R, Liu WM, Te Kronnie G, et al. New data on robustness of gene expression signatures in leukemia: comparison of three distinct total RNA preparation procedures. BMC Genomics 2007;8:188.[CrossRef][Medline] [Order article via Infotrieve]
  17. Kohlmann A, Schoch C, Dugas M, Rauhut S, Weninger F, Schnittger S, et al. Pattern robustness of diagnostic gene expression signatures in leukemia. Genes Chromosomes Cancer 2005;42:299-307.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  18. Breit S, Nees M, Schaefer U, Pfoersich M, Hagemeier C, Muckenthaler M, et al. Impact of pre-analytical handling on bone marrow mRNA gene expression. Br J Haematol 2004;126:231-243.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  19. Debey S, Schoenbeck U, Hellmich M, Gathof BS, Pillai R, Zander T, et al. Comparison of different isolation techniques prior gene expression profiling of blood derived cells: impact on physiological responses, on overall expression and the role of different cell types. Pharmacogenomics J 2004;4:193-207.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  20. Langebrake C, Gunther K, Lauber J, Reinhardt D. Preanalytical mRNA stabilization of whole bone marrow samples. Clin Chem 2007;53:587-593.[Abstract/Free Full Text]
  21. Wright C, Bergstrom D, Dai H, Marton M, Morris M, Tokiwa G, et al. Characterization of globin RNA interference in gene expression profiling of whole-blood samples. Clin Chem 2008;54:396-405.[Abstract/Free Full Text]
  22. Ioannidis JP. Is molecular profiling ready for use in clinical decision making?. Oncologist 2007;12:301-311.[Abstract/Free Full Text]
  23. Staal FJ, Cario G, Cazzaniga G, Haferlach T, Heuser M, Hofmann WK, et al. Consensus guidelines for microarray gene expression analyses in leukemia from three European leukemia networks. Leukemia 2006;20:1385-1392.[CrossRef][Medline] [Order article via Infotrieve]
  24. Haferlach T, Kohlmann A, Basso G, Bene MC, Downing J, Shurtleff S, et al. An international multi-center study to define the application of microarrays in the diagnosis and subclassification of leukemia (MILE study): Interim analysis based on 1,889 patients achieves 95.4% prediction accuracy. Blood 2006;108:34A-35A.
  25. Haferlach T, Kern W, Schnittger S, Schoch C. Modern diagnostics in acute leukemias. Crit Rev Oncol Hematol 2005;56:223-234.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  26. Schnittger S, Schoch C, Kern W, Hiddemann W, Haferlach T. FLT3 length mutations as marker for follow-up studies in acute myeloid leukaemia. Acta Haematol 2004;112:68-78.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  27. Kern W, Voskova D, Schoch C, Schnittger S, Hiddemann W, Haferlach T. Prognostic impact of early response to induction therapy as assessed by multiparameter flow cytometry in acute myeloid leukemia. Haematologica 2004;89:528-540.[Abstract/Free Full Text]
  28. Schoch C, Schnittger S, Klaus M, Kern W, Hiddemann W, Haferlach T. AML with 11q23/MLL abnormalities as defined by the WHO classification: incidence, partner chromosomes, FAB subtype, age distribution, and prognostic impact in an unselected series of 1897 cytogenetically analyzed AML cases. Blood 2003;102:2395-2402.[Abstract/Free Full Text]
  29. Haferlach T, Schoch C, Loffler H, Gassmann W, Kern W, Schnittger S, et al. Morphologic dysplasia in de novo acute myeloid leukemia (AML) is related to unfavorable cytogenetics but has no independent prognostic relevance under the conditions of intensive induction therapy: results of a multiparameter analysis from the German AML Cooperative Group studies. J Clin Oncol 2003;21:256-265.[Abstract/Free Full Text]
  30. Schoch C, Schnittger S, Bursch S, Gerstner D, Hochhaus A, Berger U, et al. Comparison of chromosome banding analysis, interphase- and hypermetaphase-FISH, qualitative and quantitative PCR for diagnosis and for follow-up in chronic myeloid leukemia: a study on 350 cases. Leukemia 2002;16:53-59.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  31. Haferlach T, Bacher U, Kern W, Schnittger S, Haferlach C. Diagnostic pathways in acute leukemias: a proposal for a multimodal approach. Ann Hematol 2007;86:311-327.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  32. Haferlach T, Kohlmann A, Basso G, Bene MC, Downing JR, Shurtleff S, et al. An international multi-center study to define the application of microarray-based gene expression profiling in the diagnosis and sub-classification of leukemia (MILE study): Analysis of completed stage I with 2030 patients achieved a 95.4% accuracy. Blood 2007;110:931A.
  33. Liu WM, Mei R, Di X, Ryder TB, Hubbell E, Dee S, et al. Analysis of high density expression microarrays with signed-rank call algorithms. Bioinformatics 2002;18:1593-1599.[Abstract/Free Full Text]
  34. Liu WM, Li R, Sun JZ, Wang J, Tsai J, Wen W, et al. PQN and DQN: algorithms for expression microarrays. J Theor Biol 2006;243:273-278.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  35. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307-310.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  36. Eisen MB, Spellman PT, Brown PO, Botstein D. Cluster analysis and display of genome-wide expression patterns. Proc Natl Acad Sci U S A 1998;95:14863-14868.[Abstract/Free Full Text]
  37. Dobbin KK, Beer DG, Meyerson M, Yeatman TJ, Gerald WL, Jacobson JW, et al. Interlaboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays. Clin Cancer Res 2005;11:565-572.[Abstract/Free Full Text]
  38. Shi L, Reid LH, Jones WD, Shippy R, Warrington JA, Baker SC, et al. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol 2006;24:1151-1161.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  39. Dumur CI, Lyons-Weiler M, Sciulli C, Garrett CT, Schrijver I, Holley TK, et al. Interlaboratory performance of a microarray-based gene expression test to determine tissue of origin in poorly differentiated and undifferentiated cancers. J Mol Diagn 2008;10:67-77.[Abstract/Free Full Text]



The following articles in journals at HighWire Press have cited this article:


Home page
 BloodHome page
K. I. Mills, A. Kohlmann, P. M. Williams, L. Wieczorek, W.-m. Liu, R. Li, W. Wei, D. T. Bowen, H. Loeffler, J. M. Hernandez, et al.
Microarray-based classifiers and prognosis models identify subgroups with distinct clinical outcomes and high risk of AML transformation of myelodysplastic syndrome
Blood, July 30, 2009; 114(5): 1063 - 1072.
[Abstract] [Full Text] [PDF]

Home page
 Brief Funct Genomic ProteomicHome page
U. Bacher, A. Kohlmann, and T. Haferlach
Perspectives of gene expression profiling for diagnosis and therapy in haematological malignancies
Briefings in Functional Genomics, May 27, 2009; (2009) elp011v1.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Data
Right arrow All Versions of this Article:
clinchem.2008.108506v1
54/10/1705    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kohlmann, A.
Right arrow Articles by Haferlach, T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kohlmann, A.
Right arrow Articles by Haferlach, T.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS