Clinical Chemistry Siemens Point of Care - Urinalysis
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


Clinical Chemistry 50: 1994-2002, 2004. First published September 13, 2004; 10.1373/clinchem.2004.033225
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Data
Right arrow All Versions of this Article:
clinchem.2004.033225v1
50/11/1994    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (39)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Dumur, C. I.
Right arrow Articles by Ferreira-Gonzalez, A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Dumur, C. I.
Right arrow Articles by Ferreira-Gonzalez, A.
Related Collections
Right arrow Molecular Diagnostics and Genetics
(Clinical Chemistry. 2004;50:1994-2002.)
© 2004 American Association for Clinical Chemistry, Inc.


Molecular Diagnostics and Genetics

Evaluation of Quality-Control Criteria for Microarray Gene Expression Analysis

Catherine I. Dumur1, Suhail Nasim1, Al M. Best2, Kellie J. Archer2, Amy C. Ladd1, Valeria R. Mas3, David S. Wilkinson1, Carleton T. Garrett1 and Andrea Ferreira-Gonzalez1,a

Departments of1 Pathology,2 Biostatistics, and3 Surgery, Virginia Commonwealth University, Richmond, VA.

aAddress correspondence to this author at: Molecular Diagnostics Division, Department of Pathology, Virginia Commonwealth University, Clinical Support Center Building, Room 247, 403 North 13th St., PO Box 980248, Richmond, VA 23298-0248. Fax 804-225-4738; e-mail agonzalez{at}hsc.vcu.edu.


   Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Background: Development of quality-control criteria to ensure reproducibility of microarray results for potential clinical application is still in its infancy.

Methods: In the present studies we developed quality-control criteria and evaluated their effect in microarray data analysis using total RNA from cell lines, frozen tumors, and a commercially available reference RNA. Quality-control criteria such as A260/A280 ratios, percentage of rRNA, and median size of cDNA and cRNA synthesis products were evaluated for robustness in microarray analysis. Furthermore, precision studies using a reference material were performed on the Affymetrix® HG-U133A high-density oligonucleotide microarrays. The same reference RNA sample was examined in 16 different chips run on 2 different days in the four different modules of the Affymetrix fluidics workstation. Fresh and frozen fragmented cRNAs were also compared. An ANOVA model was fit to identify the main sources of variation.

Results: Good-quality samples showed >30% rRNA in the electropherograms and cDNA and cRNA synthesis products with median sizes of 2.0 and 3.0 kb, respectively. Precision studies showed that the main source of variation was the day-to-day variability, minimally affecting hybridization exogenous control genes. Altogether, the results showed that the Affymetrix Genechip® system is highly reproducible when RNA that meet the quality-control criteria are used (overall P >0.01).

Conclusions: These results confirm the need to establish defined quality-control criteria for sample quality to distinguish between analytical and biological variability.


   Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Large-scale gene expression analysis using microarray technologies offers an unparalleled opportunity to study the intricate interactions occurring among genes associated with physiologic and pathologic events. Measurement of gene expression, by any laboratory methodology, possesses an associated error attributable to random and systematic errors. Those errors might become more significant in the case of high-density oligonucleotide microarrays, such as the Affymetrix GeneChips®, where the expression of thousands of genes can be assessed simultaneously. This commercially available oligonucleotide microarray contains pairs of 25-nucleotide sequences (probe pairs) synthesized on silica wafers; one of each pair exactly matches the sequence of interest, and the other contains a mismatched nucleotide in the center(1)(2). A single transcript sequence is interrogated by at least one group of 11 probe pairs that constitute a probe set. These microarrays generate vast amounts of data from which significant changes in transcript concentrations need to be identified. Thus, it is of critical importance to minimize experimental noise, standardize sample-handling protocols, and perform appropriate experimental replication and statistical analyses.

The issue of quality control (QC)1 and variability of data generated from microarray technology is recognized as critical to the potential use of microarrays in clinical applications. Assessing the quality of the initial RNA sample is as critical as ensuring successful cDNA and cRNA synthesis before microarray hybridization. However, QC criteria for total RNA samples and subsequent synthesis products used in microarray analysis have yet to be defined.

Monitoring the quality of the initial RNA sample is as important as choosing the appropriate statistical analysis because the quality of the sample might strongly influence the diagnostic and predictive power obtained from the microarray data. It is anticipated that application of different algorithms to reduce the number of genes to a nonredundant informative set would yield a set of genes useful for clustering tumor specimens into groups that share common features(3). Previous reports have shown that expression of known tumor markers or expression patterns correlating with the response to a specific therapy can create characteristic profiles of the groups. These profiles can then be used to identify newly acquired unknown specimens(4)(5). Nevertheless, a class prediction algorithm applied to an unknown sample of poor quality may lead to misclassification. A recent study showed that data quality was decreased when moderately degraded RNA samples were used(6). Furthermore, severely degraded RNA samples demonstrated low overall quality and poor correlation with their intact sample counterparts. The authors concluded that such samples should be used with caution if considered for microarray analysis and subsequent classification(6).

The standard Affymetrix protocol uses as starting material 5–40 µg of high-quality total RNA, which is used as a template for the synthesis of cDNA molecules from which biotinylated cRNA is synthesized. All procedures associated with probe preparation, hybridization, washing, and scanning have been standardized(7). Because microarray technologies in general offer a complex association between expression of a particular gene and the ultimate light intensity measured by the scanner, the accuracy and consistency of the data need to be ensured. Multiple sources of technical variation can contribute to the final observed gene expression obtained with microarrays. Some of the sources of technical variation could be different lot numbers for reagents and/or chips, different individuals handling the whole or part of the process, and day-to-day variability. In the Affymetrix platform, each sample is hybridized and stained independently in a single chip per module, adding the module-to-module effect to the list of different sources of variation. In addition, the biological sample loaded in every chip is composed of chemically fragmented (frg) cRNA molecules that can be freshly prepared immediately before hybridization or can be stored frozen until needed. The freezing process may be considered another source of variability that may need to be taken into account in the search for relevant biological discoveries. Historically, distinguishing between significant and nonsignificant biological changes in gene expression depended on the number of replicates analyzed and on the fold-change cutoff. This cutoff varied from 2- to 2.5-fold depending on the researcher’s criteria and experimental design(8)(9)(10). Such strategies for microarray data analysis do not account for the different sources of variation introduced by the system.

In this study we focused our attention on a subset of the above-mentioned sources of variation to address the day-to-day, module-to-module, and fresh/frozen frg cRNA effects on the same biological sample. We used 16 replicate GeneChips to study these components of variability. We also evaluated the quality of gene expression data, using measurable QC criteria that take into account the quality of the total RNA samples that were isolated from different sample types. In addition, cDNA and cRNA synthesis products were monitored to account for technical variability during sample preparation that could negatively impact the quality of microarray data.


   Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
rna samples and qc criteria
Total cellular RNA was isolated from different sample types, including fresh cultured cell lines from a human prostate lineage [M12 and P69SV40Tag(11)(12)], snap-frozen tissues from 5 different human ovarian cancers, 1 hepatocellular carcinoma, and 27 different frozen bone marrow cell pellets from leukemia patients. All specimens for analysis were obtained under Virginia Commonwealth University Institutional Review Board-approved protocols and anonymized. Multiple 10-µm frozen sections of the tissue samples were used for total RNA isolation using TRIZOL reagent (InvitrogenTM Life Technologies) with or without a subsequent cleanup process with RNeasy reagents (QIAGEN Inc.) according to the manufacturer’s protocols. RNA purity was judged by the ratio of absorbance at 260 and 280 nm (A260/A280) and at 260 and 270 nm (A260/A270). Furthermore, RNA integrity as well as cDNA and cRNA synthesis products were assessed by running 1 µL of every sample in RNA 6000 Nano or Pico LabChips® on the 2100 Bioanalyzer (Agilent) depending on the RNA concentration, according to the manufacturer’s protocol. The Universal Human Reference RNA (Stratagene), isolated from 10 human cell lines, was used for the precision study.

affymetrix genechip standard protocol
The Affymetrix standard protocol has been described extensively elsewhere(7)(13). Briefly, starting with 5 µg of total RNA from every sample, we generated double-stranded cDNA using a 24mer oligodeoxythymidylic acid primer with a T7 RNA polymerase promoter site added to the 3' end (Superscript cDNA Synthesis System; Life Technologies, Inc.). After second-strand synthesis, in vitro transcription was performed with the Enzo BioArray High Yield RNA Transcript Labeling Kit (Enzo Diagnostics) to produce biotin-labeled cRNA. We fragmented 20 µg of the cRNA product and hybridized it for 18–20 h into HG-133A microarrays containing 22 283 probe sets. Each microarray was washed and stained with streptavidin-phycoerythrin and was scanned at a 6-µm resolution by the Agilent G2500A Technologies Gene Array scanner (Agilent Technologies) according to the GeneChip Expression Analysis Technical Manual procedures (Affymetrix). After scanning of the chips, the raw intensities for every probe were stored in electronic files (in .DAT and .CEL formats) by the Microarray Suite 5.0 software (Affymetrix). The .CEL files from this study can be accessed through the following website: http://www.ctrf-cagenomics.vcu.edu/publiclyavaildata.htm (This website can be viewed only with a browser such as Microsoft® Internet Explorer, Ver. 6.0 or greater).

hybridization qc criteria
Immediately after scanning of every chip, several variables were monitored for QC purposes: the scaling factor; the noise (RawQ)(14) value for a given probe array hybridization; the percentage of probe sets declared "Present" (%P) by the detection call algorithm; the 3'/5' ratios of the signal intensity values for two housekeeping genes, glyceraldehyde-3-phosphate dehydrogenase (GAPDH) and ß-actin; and the presence or absence call for the ribosomal RNAs (rRNAs) 18S and 28S. The scaling factor is used to scale all probe sets to an arbitrary target value (TGT); in this study, the TGT value was set at 100. The RawQ value is calculated by taking the mean, over all the cells used in background computation, of the following value for each cell: standard deviation of the pixel intensity divided by the square root of the pixel number.

study design to assess precision
The precision study experiment was designed to assess variation attributable to day, use of fresh/frozen cRNA, and the four different modules of the Affymetrix fluidics workstation. To eliminate operator variations, the same person completed the synthesis and hybridization of all 16 chips run for this study. Five identical aliquots of 40 µg of the Reference RNA were used as template for cDNA synthesis, cRNA in vitro transcription, and labeling reactions. The final frg cRNA aliquots were pooled together. In addition, on the first day, four chips were run with fresh frg cRNA and four chips were run with frozen frg cRNA, using the same hybridization solution. On the second day, a fresh hybridization solution was prepared to replicate the entire experiment with frozen frg cRNA (see Table 1 for the complete design). Hybridization was performed using the same lot number of the HG-U133A arrays to minimize chip-to-chip variability.


View this table:
[in this window]
[in a new window]
 
Table 1. Precision study design.1

expression values
For every probe set, normalization, background subtraction, and expression summaries were calculated by three commonly used methods. First the Microarray Suite 5.0 (MAS5) method (Affymetrix) was used to obtain probe set summaries. A detailed description of this algorithm has been published elsewhere(15). Numerical expression summaries were stored in electronic files (in .CHP format). Next, model-based expression indexes (MBEI)(16) were calculated by use of a multiplicative model to account for probe affinity effects in calculating probe set expression summaries. Finally, the robust multiarray average (RMA) method was used(17); this method uses quantile normalization followed by a median polish to remove probe affinity effects when calculating probe set summaries.

statistical analysis
We used ANOVA on each probe set to analyze the significance of each separate identifiable sources of variation (i.e., day, module, frozen, and the day x module and frozen x module two-way interactions) on gene expression. In addition, the significance of any of the above effects was assessed by the whole model test (with 11 degrees of freedom). The ANOVA model was fit to the MAS5, MBEI, and RMA probe set summaries separately. Additionally, we used different statistical methods that are widely used by the microarray community for class comparison to identify genes differentially expressed among the experimental conditions in our study. First, for each probe set an F-test was performed. In addition, univariate permutation-based P values, using 2000 permutations, were calculated for the same class comparisons. Both the F-test and the permutation-based P values were obtained with use of BRB-ArrayTools, Ver. 3.1.0(18), an Excel Add-in that performs analyses of microarray data. The significance analysis of microarrays (SAM)(19) method was also used, which allows one to control for the false discovery rate. The publicly available SAM Excel Add-in, developed at Stanford University (Stanford, CA) was used to conduct this analysis. To account for multiple comparisons, we set the {alpha} to 0.0001 for declaring probe sets as significantly differentially expressed, implying that of the 22 283 probe sets present in the HG-U133A array, we expected by chance that ~2 probe sets on the list will be false positives.


   Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
rna sample qc criteria
We ascertained the QC criteria for total RNA samples and cDNA and cRNA synthesis products. These were designed to ensure good-quality microarray data and to allow us to minimize technical variability among different sample types and chip runs. High-quality RNA samples were expected to show an A260/A280 ≥1.8 and A260/A270 ≥1.1 when the RNA preparation was free from protein and/or phenol contamination, respectively. In addition, the two most distinct and intense peaks on the electropherogram obtained with the bioanalyzer, corresponding to the 18S and 28S rRNAs, should show 28S/18S ratios >1.4, with the sum of both rRNA peak areas accounting for more than 30% of all RNA, indicating that the RNA sample was intact or undegraded (Table 2 ). We also included as QC criteria for the sample processing, the median cDNA and cRNA synthesis product length, where medians of 2.0 and 3.0 kb were considered optimal for cDNA and cRNA synthesis, respectively. Finally, total RNA samples meeting all of the above-mentioned QC criteria typically showed ratios of the signal intensities for probe sets designed to match the 3' and 5' regions of two housekeeping transcripts, GAPDH and ß-actin, near 1.00 (Table 2 ).


View this table:
[in this window]
[in a new window]
 
Table 2. QC criteria for RNA sample purity, cDNA and cRNA preparation, and hybridization in the HG-U133A microarrays.

RNA samples from different sources were tested for QC purposes. Good-quality RNA was obtained from the Stratagene Universal Human Reference RNA sample. It met almost all of the QC criteria except for the 28S/18S rRNA ratio, which was <1.4 (Table 2Up ). Nevertheless, the percentage of rRNA was >30%, and the cDNA and cRNA syntheses were successful as indicated by median lengths of 2.0 and 3.0 kb for the respective products.

In addition, we obtained good-quality RNA from freshly harvested cell lines, using the TRIZOL procedure. These samples systematically met all QC criteria (Table 2Up and Fig. 1A ). Interestingly, total RNA isolated from snap-frozen tissue samples by the same procedure did not meet the QC criteria, leading to poor-quality cDNA and cRNA products (Fig. 1B ). The RNA isolated from these samples showed good A260/A280 (>1.8) and A260/A270 ratios (>1.1). In addition, these samples were intact or undegraded, with rRNA 28S/18S ratios >1.4 and percentages of rRNA >30%. However, as clearly shown in Table 2Up , they systematically failed to produce cDNA and, therefore, cRNA molecules >300 and 700 bp, respectively. Moreover, all of these samples showed GAPDH and ß-actin 3'/5' ratios much greater than 1.00, which was in agreement with unsuccessful cDNA and cRNA synthesis (Table 2Up ). Suspecting the presence of inhibitors in these RNA preparations, which might be inherent to the tissue-handling process, we reextracted RNA from the same snap-frozen tissue samples with the TRIZOL protocol and followed with an extra RNeasy cleanup step. We found that RNA samples extracted with this modified procedure met all of the QC criteria, with cDNA and cRNA synthesis products within median lengths of 2.0 and 3.0 kb, respectively (Fig. 1C ) and 3'/5' ratios close to 1.00 for both housekeeping genes, GAPDH and ß-actin (Table 2Up ).



View larger version (37K):
[in this window]
[in a new window]
 
Figure 1. Sample preparation QC criteria.

We analyzed 1 µL of the total RNA, cDNA, and cRNA samples for size distribution in the corresponding RNA 6000 Nano LabChip assay and DNA 7500 LabChip assay on the Agilent 2100 Bioanalyzer. Electropherograms are plotted for total RNA isolated from fresh harvested M12 cells from human prostate origin by the TRIZOL procedure (A); total RNA isolated from snap-frozen ovarian tumor sample 169 by the TRIZOL procedure (B); total RNA isolated from snap-frozen ovarian tumor sample 169 by the TRIZOL procedure followed by the RNeasy cleanup protocol (C); total RNA isolated by the TRIZOL procedure from frozen acute myelogenous leukemia bone marrow cells from sample 27 that were mishandled during the storage process (D); and total RNA isolated by the TRIZOL procedure followed by the RNeasy cleanup protocol from a snap-frozen liver tumor sample containing 80% necrosis, run on the RNA 6000 Pico LabChip assay (E).

On the other hand, of 27 frozen bone marrow samples, 5 showed different patterns of RNA degradation attributable to inadequate handling and storage conditions. These five samples failed to meet the QC criteria, showing electropherogram profiles consistent with RNA degradation. A typical result is shown in Fig. 1DUp . Accordingly, these samples showed low percentages of rRNA and shorter cDNA and cRNA products. The results from two representative bone marrow samples exhibiting different degrees of degradation are summarized in Table 2Up (for complete results, see Table S1 in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol50/issue11/). Interestingly, poor-quality RNA was obtained from snap-frozen tumor samples containing different degrees of necrosis. The results from a representative tissue specimen containing 80% necrosis are summarized in the last row of Table 2Up . As expected, the RNA failed to meet the RNA QC criteria, with a percentage of rRNA <30%. This reflected the presence of degraded or partially degraded RNA (Fig. 1EUp ). Although the overall yield of the cDNA and cRNA syntheses were notably less than the usual yield, the median lengths of the cDNA and cRNA products were 2.0 and 3.0 kb, respectively (Table 2Up ). This suggested that the RNA sample contained a subset of intact RNA, likely arising from the viable tumor cells located in the remaining 20% of the tissue sample.

Overall, our results strongly suggest that the rRNA content calculated from the electropherogram profile, in conjunction with the size distribution of the cDNA and cRNA products, were good indicators of the integrity and purity of total RNA samples, respectively.

precision study
We analyzed the significance of the changes observed in different chip characteristics attributable to day-to-day, module-to-module, and fresh/frozen frg cRNA sources of variations, as well as the interactions day x module and frozen x module. In general, the different characteristics analyzed for each HG-U133A chip showed almost no significant change across the 16 chips (P >0.05; Table 3 ). However, the RawQ value was significantly affected by the day effect (P <0.05), and the probe set designed to bind to the 3' end of the ß-actin transcript was significantly affected by all the tested sources of variation (P <0.005), as indicated in bold font in Table 3 .


View this table:
[in this window]
[in a new window]
 
Table 3. ANOVA results for hybridization criteria for the HG-U133A microarrays across the 16 chips.

The 22 283 probe sets arrayed on the HG-U133A chip were first analyzed with the MAS5 probe set expression summaries. The response variable was the log2 of the signal intensity. After fitting of the model, the log2 estimate of the least-squares mean was used to assess the fold change and the significance of that change for each effect included in this study by probe set. Panels A–C in Fig. 2 display the fresh/frozen, day-to-day, and module-to-module estimates, respectively, plotted against negative log10 P value when the ANOVA included all probe sets; panels D–F display the model results when the ANOVA was restricted to Present probe sets. Fresh/frozen frg cRNA was not a significant source of variation (Fig. 2 , A and D) in either all-probe sets or in Present probe sets. On the other hand, day-to-day was a significant source of variation (P <0.0001), affecting the gene expression results for at least two Present probe sets (Fig. 2E ). One of these probe sets queries a hybridization exogenous control gene. The largest dispersion of the data was observed for the module-to-module effect. However, most of the changes were not significant at a P value <0.0001 (Fig. 2 , C and F). Overall, the ANOVA restricted to Present probe sets demonstrated a tight dispersion, below a twofold change cutoff. Nevertheless, we observed up to eightfold changes among some "Absent" probe sets and among those with low signal intensities. To further investigate the impact of Absent probe sets, we summarized the results from the ANOVA model depending on how many of the n = 16 observations were labeled as Present or Absent by the MAS5 detection call algorithm. For 98.84% of the probes (22 023 of 22 283), there were no significant effects in the model (overall P >0.01). However, 15 probe sets called Absent and 8 probe sets called Present showed significant effects in the ANOVA model (overall P <0.001; Table 4 ). Similar results were obtained when we performed the ANOVA on the expression summaries calculated with MBEI and RMA. As expected with highly robust methods of analysis, the ANOVA results issued from the RMA algorithm showed less dispersion when we analyzed all probe sets compared with the values calculated with MAS5 (data not shown).



View larger version (26K):
[in this window]
[in a new window]
 
Figure 2. Single effect variability.

Volcano plots of the fresh-frozen (A), day-to-day (B), and module 1-to-module 2 (C) variation effects are shown. The x axis is the log2 estimate of the least-squares mean of the signal intensities for each probe set across the 16 chips. The y axis is the negative log10 P value as obtained from the ANOVA study. The horizontal line in each graph indicates the testing P value equal to 0.0001, and the vertical lines indicate twofold change of effect estimates. Present only probe sets are plotted for the fresh-frozen (D), day-to-day (E), and module-to-module (F) variation effects in all 16 chips.


View this table:
[in this window]
[in a new window]
 
Table 4. ANOVA whole-model test of significance for the 22 283 probe sets of the HGU133A microarray across the 16 chips.

Because all 16 chips were run with separate aliquots from the same RNA source, any differences would reflect differences in sample handling and labeling or from random variation. We used the following algorithms to calculate probe set expression summaries: MAS5, MBEI, and RMA.

From the ANOVA results represented by the "volcano" plots in Fig. 2Up , we found that the day-to-day effect was the only source of technical variability that significantly affected the expression values of two Present probe sets. Hence, we decided to investigate the impact of that effect on gene expression by performing class comparisons between days 1 and 2, using SAM and permutation-based tests. Expression summaries for 10 probe sets were declared significantly different by both methods for the three different expression summaries (Fig. 3 ). These 10 probe sets had significantly higher intensities on day 1 than on day 2. Interestingly, these were probe sets designed to detect the set of hybridization exogenous controls recommended by Affymetrix: BioB, BioC, BioD, and Cre(7).



View larger version (13K):
[in this window]
[in a new window]
 
Figure 3. Probe sets significantly affected by the day-to-day variability.

Scatterplot of mean log2-transformed signal intensities for the two classes (n = 8 arrays per class): day 1 vs day 2. The 10 probe sets that were significantly affected by the day-to-day variation are plotted in red.


   Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
To ensure highly reproducible microarray data, total RNA samples and cDNA and cRNA synthesis steps need to meet rigorous QC criteria. Here, we have established quantitative criteria that allow us to control every stage of the process. These criteria ensure optimum quality of the RNA sample, in terms of chemical purity and integrity, as well as the efficiency of the synthesis of the final product that hybridizes the microarray. Of all the criteria that we evaluated, the 28S/18S rRNA ratios were the least indicative of the integrity or the quality of the RNA sample. Although it has been suggested that 28S/18S rRNA ratios should be ≥2.0(20), in good-quality total RNA samples, we have seen successful cDNA and cRNA syntheses with 28S/18S rRNA ratios as low as 0.8 (Table 2Up ). Furthermore, we have tested RNA samples isolated from mishandled bone marrow specimens that displayed different degrees of degradation, and the 28S/18S rRNA ratios were >1.5 (see Table S1 in the online Data Supplement). We therefore suggest that the percentage of rRNA in the electropherograms might be a suitable QC criterion to evaluate total RNA integrity instead of just the 28S/18S rRNA ratio. These results are in agreement with previously reported data, where it was shown that the 28S/18S ratio for a particular RNA sample has no practical value and should be used only along with the absence of prominent degradation products(6). Here we demonstrated that the sole use of the 28S/18S ratio as a QC criterion could be misleading.

More importantly, when examining the total RNA electropherogram, one must also consider the possible sources of degradation. Our results showed that RNA degradation caused by sample mishandling can vary depending on the extent of technical damage, with the rRNA contribution ranging from 0% in completely degraded RNA samples to 20% in partially degraded RNA samples. From the results obtained in this study, we established a cutoff value of 30% rRNA contribution to the total area under the electropherogram for RNA samples to be considered as intact or undegraded (see Figs. S1 and S2 in the online Data Supplement). On the other hand, tissue samples containing different degrees of necrosis yielded a mixture of intact and degraded RNA, producing electropherograms with <30% rRNA contribution and variable 28S/18S ratios. In these samples, however, the cDNA and cRNA syntheses were performed successfully, as opposed to those that were performed with RNA samples that were degraded by different technical causes.

Our results with freshly harvested cell lines showed that high-quality RNA could be obtained from a simple TRIZOL extraction; however, snap-frozen tissues required a subsequent RNA cleanup process to successfully synthesize cDNA and cRNA molecules. As a standard practice, it would be advisable to evaluate every different type of sample with all of the QC criteria defined in this study to ensure consistency in sample handling and processing. Otherwise, microarray analysis may misclassify technical variability as biological variability.

The performance of every chip was also evaluated for indications of successful hybridization, washing, and staining procedures. A set of control genes within every chip can be used to monitor the sample quality at the hybridization step. Thus, 3'/5' ratios close to 1.0 for housekeeping genes such as GAPDH and ß-actin are expected values for good-quality samples and efficient extension reactions during the cDNA and cRNA synthesis steps. Moreover, values such as the noise (RawQ or the pixel to pixel variation for very low intensity cells within the chip), scaling factor, and percentage of Present probe sets (%P) are very good indicators of outlier samples or chips within similar sample types and runs(21). From the results of the present study, we concluded that the %P was a good indicator of quality of the microarray, correlating with the quality of the sample only when comparisons were made within the same tissue or cell type. Thus, when we compared two different RNA extraction protocols applied on five different ovarian tumor samples, a significant difference (paired t-test, P <0.01) was observed in the %P calls. Hence, ovarian tumor RNA extracted with the TRIZOL procedure showed a mean (SD) %P calls of 52.00 (3.60), whereas for the same ovarian tumor RNA extracted with the TRIZOL procedure followed by a QIAGEN cleanup step, the %P calls were significantly increased, to 57.66 (1.48).

Among all hybridization criteria analyzed in the present study, we found that only the RawQ value and the expression summary of the 3' probe set querying ß-actin were significantly (P <0.05) affected by some of the sources of variation analyzed. The significant day-to-day effect on the RawQ value was probably attributable to the fact that the hybridization and staining solutions were freshly prepared every day. Thus, slight differences in the final composition of the above-mentioned solutions could impact the overall noise (RawQ value) of individual chips run on different days. Surprisingly, the expression summary of the 3' probe set querying ß-actin seemed to be significantly affected by every effect analyzed; however, the extent of the changes was <1.2-fold for all sources of variation (data not shown). Moreover, the fact that the 5' probe set querying the same transcript was not significantly affected by any of the effects suggests that the changes observed in the 3' probe set might be related to a rather less robust probe design than actual differences in the concentration of that particular transcript.

In this study, we used an ANOVA model to find significant changes in the signal intensity values attributable to all identifiable technical effects. Using the MAS5 algorithms, we found a subset of 15 significantly altered probe sets (P <0.001), corresponding to Absent probe sets, with signal intensities below the mean (SD) background for all chips [50.70 (2.21)]. We therefore concluded that the variation of the values for these probe sets should be considered to be related to noise, or random variation, more than to any of the sources of variation analyzed in this study.

On the other hand, we found a subset of eight significantly affected probe sets (P <0.001) corresponding to Present probe sets according to the MAS 5.0 algorithms. Seven of them had signal intensities <200, and the variation across the 16 chips was always <1.25-fold change. The eighth probe set from this group corresponded to the Filamin 1 gene, with a mean signal intensity value of 610. Because that value was nearly 10 times the background value for this chip, we considered this probe set a good candidate for false discovery because of its large variability across the technique itself, not reflecting the biological differences of these 16 replicates. To evaluate the performance of this particular probe set, we analyzed another probe set that interrogates the same gene, Filamin 1, designed to hybridize a similar region of the transcript. The second probe set for this gene showed no significant variation across the 16 chips, reflecting the real biological status of the 16 replicates. We therefore presumed that the first probe set against Filamin 1 could be considered an outlier probe set within the HG-U133A array, whose results might be attributable to random variation.

The rate of false discovery of a significant biological change in gene expression, P <0.0001, would be 1 false positive gene in 10 000 genes. For the 22 283 probe sets analyzed in the HG-U133A chip, only one Absent gene was found to be significantly affected by the whole model of sources of variability.

We also explored other methods of identifying differentially expressed genes that could account for the day-to-day variability. Using {alpha}-adjusted t-tests and SAM on the three different expression value sets obtained with MAS5, MBEI, and RMA, we created a common list of differentially expressed genes containing 10 probe sets. These expression values were significantly higher on the first day than on the second day. Interestingly, these 10 probe sets were part of the 18 probe sets that are designed to detect the set of hybridization exogenous controls recommended by Affymetrix: BioB, BioC, BioD, and Cre(7). The remaining eight exogenous control probe sets were not significantly affected by the day-to-day effect, suggesting that these 10 probe sets might be more sensitive to subtle variations in target concentration. These transcripts are part of a cocktail that is added to the sample at the moment of hybridization. Because that solution was freshly made each day, variation could have been a result of slight pipetting errors at the moment of hybridization mixture preparation. These results are thus in agreement with the expected run-to-run variability. Even more importantly, the fact that only the hybridization exogenous controls were significantly affected by the day-to-day variability when high-quality total RNA samples were analyzed strongly suggests that a platform such as the Affymetrix high-density oligonucleotide microarrays could be extremely useful for clinical applications in cancer and other diseases stratification.

Finally, from these studies we concluded that total RNA samples could be successfully used in microarray experiments when they met rigorous QC criteria that ensure reliable gene expression data. These QC criteria can also be applicable to RNA samples to be analyzed on different microarray platforms, where efficient cDNA and cRNA synthesis steps are required.


   Acknowledgments
 
We thank Dr. Joy L. Ware for kindly providing the M12 and P69SV40Tag cell lines. We are also very thankful to Cynthia Wolpert and Tricia Beisch for technical assistance in the tissue acquisition process, as well as to Irene Gonzalez for clinical data management. This work was supported entirely by Commonwealth Technology and Research Fund funding (CTRF #SE2002_02).


   Footnotes
 
1 Nonstandard abbreviations: QC, quality control; frg, fragmented; GAPDH, glyceraldehyde-3-phosphate dehydrogenase; MAS5, Microarray Suite 5.0; MBEI, model-based expression index; RMA, robust multiarray average; and SAM, significance analysis of microarrays.


   References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 

  1. Lipshutz RJ, Fodor SP, Gingeras TR, Lockhart DJ. High-density synthetic oligonucleotide arrays. Nat Genet 1999;21:20-24.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  2. Lockhart DJ, Dong H, Byrne MC, Follettie MT, Gallo MV, Chee MS, et al. Expression monitoring by hybridization to high-density oligonucleotide arrays. Nat Biotechnol 1996;14:1675-1680.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  3. Schmidt U, Begley CG. Cancer diagnosis and microarrays. Int J Biochem Cell Biol 2003;35:119-124.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  4. Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999;286:531-537.[Abstract/Free Full Text]
  5. Perou CM, Sorlie T, Eisen MB, van de Rijn M, Jeffrey SS, Rees CA, et al. Molecular portraits of human breast tumours. Nature 2000;406:747-752.[CrossRef][Medline] [Order article via Infotrieve]
  6. Schoor O, Weinschenk T, Hennenlotter J, Corvin S, Stenzl A, Rammensee HG, et al. Moderate degradation does not preclude microarray analysis of small amounts of RNA. Biotechniques 2003;35:1192-1201.[Web of Science][Medline] [Order article via Infotrieve]
  7. . Affymetrix GeneChip®. Expression analysis technical manual 2001 Affymetrix Santa Clara, CA. .
  8. Schena M, Shalon D, Heller R, Chai A, Brown PO, Davis RW. Parallel human genome analysis: microarray-based expression monitoring of 1000 genes. Proc Natl Acad Sci U S A 1996;93:10614-10619.[Abstract/Free Full Text]
  9. Amundson SA, Bittner M, Chen Y, Trent J, Meltzer P, Fornace AJ, Jr. Fluorescent cDNA microarray hybridization reveals complexity and heterogeneity of cellular genotoxic stress responses. Oncogene 1999;18:3666-3672.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  10. Zhou L, Halvorsen Y-D, Cryan EV, Pelton PD, Burris TP, Demarest KT. Analysis of the pattern of gene expression during human adipogenesis by DNA microarray. Biotechnol Techniques 1999;13:513-517.[CrossRef]
  11. Bae VL, Jackson-Cook CK, Brothman AR, Maygarden SJ, Ware JL. Tumorigenicity of SV40 T antigen immortalized human prostate epithelial cells: association with decreased epidermal growth factor receptor (EGFR) expression. Int J Cancer 1994;58:721-729.[Web of Science][Medline] [Order article via Infotrieve]
  12. Bae VL, Jackson-Cook CK, Maygarden SJ, Plymate SR, Chen J, Ware JL. Metastatic sublines of an SV40 large T antigen immortalized human prostate epithelial cell line. Prostate 1998;34:275-282.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  13. Sugita M, Geraci M, Gao B, Powell RL, Hirsch FR, Johnson G, et al. Combined use of oligonucleotide and tissue microarrays identifies cancer/testis antigens as biomarkers in lung carcinoma. Cancer Res 2002;62:3971-3979.[Abstract/Free Full Text]
  14. . Affymetrix. Affymetrix microarray suite user’s guide, version 5.0 2001 Affymetrix Santa Clara, CA. .
  15. Hubbell E, Liu W-M, Mei R. Robust estimators for expression analysis. Bioinformatics 2002;18:1585-1592.[Abstract/Free Full Text]
  16. Li C, Wong WH. Model-based analysis of oligonucleotide arrays: model validation, design issues and standard error application. Genome Biol 2001;2:RESEARCH0032.[Medline] [Order article via Infotrieve]
  17. Irizarry RA, Bolstad BM, Collin F, Cope LM, Hobbs B, Speed TP. Summaries of Affymetrix GeneChip probe level data. Nucleic Acids Res 2003;31:e15.[Abstract/Free Full Text]
  18. Radmacher MD, McShane LM, Simon R. A paradigm for class prediction using gene expression profiles. J Comput Biol 2002;9:505-511.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  19. Tusher VG, Tibshirani R, Chu G. Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci U S A 2001;98:5116-5121.[Abstract/Free Full Text]
  20. Skrypina NA, Timofeeva AV, Khaspekov GL, Savochkina LP, Beabealashvilli RS. Total RNA suitable for molecular biology analysis. J Biotechnol 2003;105:1-9.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  21. Schadt EE, Li C, Su C, Wong WH. Analyzing high-density oligonucleotide gene expression array data. J Cell Biochem 2000;80:192-202.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]



The following articles in journals at HighWire Press have cited this article:


Home page
Brief Funct Genomic ProteomicHome page
X. Ye and R. Lotan
Potential misinterpretation of data on differential gene expression in normal and malignant cells in vitro
Brief Funct Genomic Proteomic, July 1, 2008; 7(4): 322 - 326.
[Abstract] [Full Text] [PDF]


Home page
J. Mol. Diagn.Home page
C. I. Dumur, M. Lyons-Weiler, C. Sciulli, C. T. Garrett, I. Schrijver, T. K. Holley, J. Rodriguez-Paris, J. R. Pollack, J. L. Zehnder, M. Price, et al.
Interlaboratory Performance of a Microarray-Based Gene Expression Test to Determine Tissue of Origin in Poorly Differentiated and Undifferentiated Cancers
J. Mol. Diagn., January 1, 2008; 10(1): 67 - 77.
[Abstract] [Full Text] [PDF]


Home page
BloodHome page
S. S. Winter, Z. Jiang, H. M. Khawaja, T. Griffin, M. Devidas, B. L. Asselin, and R. S. Larson
Identification of genomic classifiers that distinguish induction failure in T-lineage acute lymphoblastic leukemia: a report from the Children's Oncology Group
Blood, September 1, 2007; 110(5): 1429 - 1438.
[Abstract] [Full Text] [PDF]


Home page
Hum Mol GenetHome page
G. Sadri-Vakili, B. Bouzou, C. L. Benn, M.-O. Kim, P. Chawla, R. P. Overland, K. E. Glajch, E. Xia, Z. Qiu, S. M. Hersch, et al.
Histones associated with downregulated genes are hypo-acetylated in Huntington's disease models
Hum. Mol. Genet., June 1, 2007; 16(11): 1293 - 1306.
[Abstract] [Full Text] [PDF]


Home page
BioinformaticsHome page
K. J. Archer and T. Guennel
An application for assessing quality of RNA hybridized to Affymetrix GeneChips
Bioinformatics, November 1, 2006; 22(21): 2699 - 2701.
[Abstract] [Full Text] [PDF]


Home page
BiostatisticsHome page
K. J. Archer, C. I. Dumur, S. E. Joel, and V. Ramakrishnan
Assessing quality of hybridized RNA in Affymetrix GeneChip experiments using mixed-effects models
Biostat., April 1, 2006; 7(2): 198 - 212.
[Abstract] [Full Text] [PDF]


Home page
J. Mol. Diagn.Home page
T. M. Daly, C. M. Dumaual, C. A. Dotson, M. W. Farmen, S. K. Kadam, and R. D. Hockett
Precision Profiling and Components of Variability Analysis for Affymetrix Microarray Assays Run in a Clinical Context
J. Mol. Diagn., August 1, 2005; 7(3): 404 - 412.
[Abstract] [Full Text] [PDF]


Home page
Nucleic Acids ResHome page
S. Imbeaud, E. Graudens, V. Boulanger, X. Barlet, P. Zaborski, E. Eveno, O. Mueller, A. Schroeder, and C. Auffray
Towards standardization of RNA quality assessment using user-independent classifiers of microcapillary electrophoresis traces
Nucleic Acids Res., March 30, 2005; 33(6): e56 - e56.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Data
Right arrow All Versions of this Article:
clinchem.2004.033225v1
50/11/1994    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (39)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Dumur, C. I.
Right arrow Articles by Ferreira-Gonzalez, A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Dumur, C. I.
Right arrow Articles by Ferreira-Gonzalez, A.
Related Collections
Right arrow Molecular Diagnostics and Genetics


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS