| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Division of Laboratory Sciences, National Center for Environmental Health, Centers for Disease Control and Prevention, Atlanta, GA 30341-3724.
2 Departments of Laboratory Medicine and Pathology, Children’s Hospital and Harvard Medical School, Boston, MA 02115.
3 IVD Consulting, D-35041 Marburg, Germany.
aAddress correspondence to this author at: Division of Laboratory Sciences, National Center for Environmental Health, Centers for Disease Control and Prevention, Mailstop F25, 4770 Buford Hwy NE, Atlanta, GA 30341-3724. Fax 770-488-4192; e-mail mkimberly{at}cdc.gov.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: We evaluated five proposed secondary reference materials, including two diluted preparations of Certified Reference Material 470 (CRM470), two preparations of a serum-based material with recombinant CRP added, and one serum-based material with isolated CRP added. Twenty-one manufacturers participated in the comparison with 28 different assays. We examined imprecision, linearity, and parallelism with these materials and with fresh serum.
Results: All materials had similar imprecision; CVs for the undiluted materials were 2.1–3.7%. None of the materials was linear across all assays. Each had between one and three cases of nonlinearity, with one preparation of CRM470 having the fewest cases of nonlinearity. Although none of the materials was parallel across all assays, the differences in slope from fresh serum were similar across all assays.
Conclusions: All materials performed similarly with regard to imprecision, linearity, and parallelism. As one preparation of CRM470 had slightly better characteristics than the other materials and because CRM470 had been certified previously as a reference material for the acute-phase reactant range, it will be used in the next phase to standardize hsCRP assays.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
0.1 mg/L and the ability to measure CRP in a range below the detection limit of conventional assays (9)(10). Rifai and Ridker (11) have proposed that medical decision points established by prospective epidemiologic studies be used to interpret individual patient CRP results in risk assessment for cardiovascular disease. This is similar to the approach used by the National Cholesterol Education Program for blood lipids (12) and requires that assays be standardized so that they provide comparable results.
Standardization of any measurand requires a reference measurement system, including a primary reference measurement procedure, primary reference materials, and secondary reference materials (13). No primary reference measurement procedure and no generally accepted primary reference material exist for CRP, and no secondary reference materials are available with CRP concentrations in the range for assessing cardiovascular risk. Two secondary reference materials exist with concentrations in the acute-phase-reactant range: (a) WHO 1st International Standard 85/506 (WHO IS 85/506), with a CRP concentration of 98 mg/L, and (b) European Community Bureau of Reference Certified Reference Material 470 (CRM470), with a CRP concentration of 39.2 mg/L (14)(15), which was value-assigned using WHO IS 85/506. However, their behavior and characteristics when diluted to concentrations found in chronic inflammation have not been thoroughly investigated and, therefore, have not been demonstrated to be applicable for CRP in the high-sensitivity range.
Several studies have been published that evaluated hsCRP assays (16)(17)(18)(19)(20). Two studies found that the CRP concentrations demarcating quartiles of a healthy population were method-dependent (16)(17). The variability among the assays can lead to misclassification of patients and hampers implementation of population-based medical decision points. The authors of the studies concluded that standardization efforts are needed to improve the clinical utility of hsCRP assays.
As part of the CDC’s effort to standardize hsCRP assays, we evaluated proposed secondary reference materials (PRMs) to select a material that can be used to standardize hsCRP assays. This evaluation included CRM470 because it is the accepted secondary reference material for CRP in the acute-phase-reactant range and because it is the material used by most manufacturers to calibrate their assays. In addition, both frozen and lyophilized materials from other manufacturers were evaluated. We used the IFCC standardization projects for lipoprotein(a) (21) and myoglobin(22) as models in protocol development.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
proposed reference materials and controls
Five proposed secondary reference materials (PRM1 to PRM5, described below) were used: two (PRM1 and PRM2) were prepared by Oriental Yeast Co., Ltd (Shiga, Japan) and contained recombinant CRP (rCRP); one (PRM3) was prepared by Scipac (Sittingbourne, Kent, United Kingdom) and contained isolated CRP (iCRP); and two (PRM4 and PRM5) were dilutions of CRM470. PRM1 was frozen; PRM2, PRM3, and CRM470 were lyophilized. The participants purchased CRM470 from the Institute for Reference Materials and Methods (Geel, Belgium) and were provided three PRMs (PRM1 to PRM3) and a frozen (FC) and lyophilized control (LC). FC and LC were prepared by Oriental Yeast Co., Ltd. Detailed descriptions of the preparation of PRM1 to PRM3, FC, and LC are available in the Data Supplement that accompanies the online version of this article (http://www.clinchem.org/content/vol49/issue4/).
For each assay in the evaluation, participants received two vials each of PRM1, PRM2, PRM3, FC, and LC, shipped together on dry ice, from the CDC. Participants were instructed to store the materials at -20 °C until they were ready to perform the evaluation.
Each participant prepared a fresh in-house serum pool (IHSP), consisting of three to nine different fresh serum samples. Participants were instructed to prepare the pool with a final CRP concentration of 4 mg/L. The IHSP was stored at 4 °C up to 1 week from collection of the individual serum samples.
laboratory measurements
Participants were instructed to use the same lots of reagents and calibrators for the entire evaluation, which was conducted in two separate analytical runs performed on 2 days. The samples were prepared in exactly the same way for both runs. For each run, one vial each of FC and PRM1 was thawed and one vial each of PRM2, PRM3, and LC was reconstituted with 1.0 mL of deionized water. CRM470 was reconstituted according to the Institute for Reference Materials and Methods procedure and diluted 1:10 with assay diluent from the reagent set to produce PRM4 or 8.5 g/L NaCl to produce PRM5.
PRM1 to PRM5 and the serum controls (IHSP, FC, and LC) contained 4 mg/L CRP. A series of dilutions were prepared and used to assess imprecision, linearity, and parallelism. The diluent from the assay reagent set was used to prepare all dilutions except those for PRM5, which were prepared with 8.5 g/L NaCl. The following dilutions were prepared in duplicate for each material and preparation, expressed as the percentage dilution: 0%, 20%, 40%, 60%, 80%, 90%, and 95% (approximate concentrations of 4.0, 3.2, 2.4, 1.6, 0.8, 0.4, and 0.2 mg/L, respectively). Participants were instructed that they must perform the reconstitution and dilution steps with pipettes calibrated according to International Organization for Standardization Technical Report (ISO/TR) 20461 (23). Participants were requested to provide documentation that their pipettes had been calibrated to within 0.5% accuracy.
The participants were instructed to calibrate the assay according to the manufacturer’s established procedure, using the manufacturer’s recommended calibrators. A new calibration curve was constructed for each analytical run. Runs were to be completed the same day that the dilutions were prepared, and a specific run order was prescribed. Each dilution series was analyzed in duplicate, producing four data points per dilution and analytical run.
data analysis
We used SAS software (SAS Institute Inc.) to process the data and perform the statistical analyses. We computed the mean, minimum, and maximum values for the undiluted samples (except for the IHSP, which was not common among the participants) for all assay systems.
We evaluated each assay on the basis of its performance when measuring the IHSP. To perform further evaluations, we selected assays that demonstrated acceptable within-day imprecision, as described below.
Computation of average imprecision across dilutions and use of conventional linear regression algorithms assume that variance is constant across the concentration range. We found that variance was not constant for all assays and materials across the concentration range; we therefore used a weighted linear regression in all further calculations, where separate weights were determined for each assay/material/dilution by taking the reciprocal of the total variance associated with a single measurement at the given dilution. Imprecision, as indicated by CV, was evaluated for each dilution separately. Weights for the regression were assigned differently for each assay and material and were based on the within-run variation at each dilution.
To assess within-day imprecision, we computed the average within-day variation at each dilution of the IHSP for each assay. From these averages we estimated the 95th and 99th percentile range limits of replicate within-day results for each dilution. For each assay, the range of values at each dilution was compared with these limits. Any assay with two ranges exceeding the 95% range limit or with one range exceeding the 99% limit for a single dilution was flagged. Assays with three or more range flags were considered to have insufficient precision and were excluded from the study.
We performed a nonparametric Wilcoxon test to determine whether the distribution of total CVs (including within- and among-day variation) differed among materials at each dilution.
To evaluate the linearity with regard to clinical relevance for each of the proposed materials across assays, we used the approach of Kroll and Emancipator (24). In this approach, both the best-fit polynomial equation and the best linear fit are determined. We used a maximum allowable CV of 10%, which means that two results could differ by 28% [(1.96)(
)(0.10)(100%)] and not be distinguished as different from one another at a 0.05 level of significance. Weighted analysis was used for both the polynomial and linear fits. We evaluated the difference between the polynomial and linear fits at seven concentrations (0.2, 0.4, 0.8, 1.6, 2.4, 3.2, and 4.0 mg/L) and determined the number of times the difference was >28%. If an assay was nonlinear at the highest or lowest concentration of the IHSP, we reevaluated linearity at a narrower concentration range. If a particular assay was nonlinear at concentrations other than the lowest or highest concentrations, then that assay was eliminated from further evaluation. If an assay was linear over a narrower concentration range with the IHSP, only the narrower concentration range was considered for the proposed materials.
We used the results of the weighted linear regression to evaluate parallelism with the IHSP. Slopes for each of the materials were compared with the slope obtained for the IHSP. The deviations of the slopes for the various materials from the IHSP slope were tested for statistical significance using F-tests to evaluate the ratio of the mean square associated with the deviation to the mean square for error. Materials with P values <0.0022 (derived by dividing a 0.05 significance level by the number of significance tests performed) were considered nonparallel compared with the IHSP.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
. The assigned CRP concentration of CRM470 was 39.2 mg/L. For the 1:10 dilutions of CRM470 (PRM4 and PRM5), the mean values were near the expected values. The range of minimum to maximum values was large for all of the materials. Minimum values were 45–53% lower than the mean; maximum values were 31–42% higher than the mean.
|
imprecision
Three assays (one chemiluminescent assay, one PEITA, and the ELISA) had within-day variation exceeding the 95% and 99% range limits for three or more dilutions and did not meet our imprecision criteria. They were therefore excluded from the study.
The imprecision for each of the proposed materials at each dilution for the 25 remaining assays is shown in Table 2
. Except for the 95% dilution for all materials and the 90% dilution for PRM1, all CVs were <10%. All CVs at the lower dilutions (60% and lower, corresponding to CRP concentrations of 1.6 mg/L or higher) were <5%. Plots of the mean total CV and the 5th and 95th percentiles for each material appear in the Data Supplement that accompanies the online version of this article (http://www.clinchem.org/content/vol49/issue4/). The nonparametric Wilcoxon test showed that the CVs for any material at any dilution were not significantly different from those for the IHSP (P = 0.2119–0.8078).
|
linearity
We included 24 assays in the evaluation of linearity; one assay (an ITA) was eliminated because it did not have results for the three highest concentrations for all materials. Sixteen assays were linear with the IHSP in the range 0.2–4.0 mg/L. Seven (three ITAs, one LITA, one PEITA, one LEITA, and one particle-enhanced immunonephelometric assay) were nonlinear at the lowest concentration (0.2 mg/L), whereas one (an ITA) was nonlinear at the highest concentration (4.0 mg/L). Details regarding linearity ranges for the IHSP and a comparison of the linearity evaluation for all materials and methods appear in Table 1
of the Data Supplement (http://www.clinchem.org/content/vol49/issue4/). Most assays showed linearity across all materials. Those assays that were nonlinear for PRM1 and PRM2 were also nonlinear for the FC and/or the LC. None of the materials showed linearity across all assays. PRM4 was nonlinear in one assay; PRM1, PRM2, and PRM5 were nonlinear in two assays; and PRM3 was nonlinear in three assays.
No difference was found between PRM1 and PRM2 or between FC and LC, indicating that lyophilization does not affect linearity. Nonlinearity could not be linked to the source of CRP (rCRP or iCRP).
parallelism
The same 24 assays were included in the evaluation of parallelism. The mean absolute values of the percentage difference between the slope of the proposed materials and the IHSP were 1.5% for PRM1, 2.3% for PRM2, 1.9% for PRM3, 2.7% for PRM4, and 2.1% for PRM5. Details regarding the slope comparison appear in Table 2
of the Data Supplement (http://www.clinchem.org/content/vol49/issue4/). None of the materials was parallel across all assays. Most of the cases of nonparallelism were with the ITAs. PRM2 and PRM3 were not parallel in two assays, PRM4 was not parallel in three assays, and PRM1 and PRM5 were not parallel in four assays. The LITAs, PEITAs, and lateral-flow immunoassay were parallel across all proposed materials. The percentage differences from the IHSP slope are shown in Fig. 1
. With three exceptions, all assays had differences <10% from the IHSP slope. The nonparallelism observed could not be linked to either the source of CRP or to lyophilization.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
4 mg/L, spanned the tertiles (or quintiles) proposed as medical decision points (11)(25). Compared with the high variability across assays, the variability within assays at this concentration was very low. Thus the high variability observed across assays appears to be caused by the different calibration protocols rather than overall high intraassay variability, confirming the need for standardization. Because all participants in this evaluation claim to calibrate their assays using CRM470, the lack of a common value-transfer protocol may be one explanation for the observed variability. One such protocol has been proposed, using CRM470 as an example (26).
imprecision
To be viable, a PRM should have performance comparable to or better than the IHSP. We observed no significant differences in the CVs between the IHSP and any of the proposed materials; we can therefore assume that the distributions (across laboratories) of the within-laboratory variations (among-day + within-day) are comparable across materials at each dilution, i.e., no material had significantly better imprecision than any other. A CV of 10% at 0.2 mg/L has been used in other studies as a maximum tolerable assay performance criterion for evaluating hsCRP assays (16)(17). For the 95% dilutions, which had concentrations of 0.2 mg/L, only PRM1 had a mean CV 
10% (Table 2
). The IHSP had a CV of 10.4% at the 95% dilution. The proposed materials with CVs similar to that for the IHSP at this concentration were PRM1, PRM2, and PRM4.
linearity
The results showed that use of rCRP (in PRM1 and PRM2) or iCRP (in PRM3) as well as lyophilization (PRM2 vs PRM1 and LC vs FC) does not affect linearity. Linearity does seem to be affected by the type of base material used, as indicated by the observation that those assays that lacked linearity with PRM1 and PRM2 also lacked linearity with either FC or LC. Linearity and nonlinearity were observed for all types of assays; thus, specific assay technologies (e.g., ITA and LITA) were not associated with lack of linearity. The base materials used for PRM1, PRM2, and PRM3 are quite similar, whereas CRM470 underwent a more extensive modification.
Currently, no performance criteria have been established for hsCRP assays, but a panel convened by CDC and the American Heart Association is in the process of establishing laboratory performance guidelines. For the purpose of this study, we used an assay CV of 10% as a linearity evaluation criterion, but this should not be construed as a recommendation for maximum assay imprecision. We performed the linearity evaluation using a CV of 5% and came to the same conclusions regarding the proposed materials (data not shown). Thus, in this study, assay imprecision did not impact the outcome of the linearity evaluation.
parallelism
On the basis of the number of assays with slopes significantly different from that for IHSP, the materials behaving most similarly to the IHSP in the parallelism evaluation were PRM2 and PRM3, with PRM4 following closely. However, the mean absolute values of the differences in slope from the IHSP for the 24 assays were on the same order of magnitude, and the distributions of the differences were similar (Fig. 1
). Interestingly, PRM4, although highly diluted, still showed most of the same characteristics as the IHSP for linearity and parallelism, probably because the manufacturers used this material and diluent to optimize and develop their assays.
Although assays may show linearity for certain materials, the slopes (instrument response to CRP concentration) may be different for each material. A good secondary reference material should provide the same slope as a fresh specimen and consequently should provide a calibration curve that is parallel to that obtained from fresh specimens. We considered that matrix effects or differences in the source of CRP might cause the materials to not be parallel to the IHSP. In this study, nonparallelism could not be associated with either differences in the source of CRP (iCRP or rCRP) or with matrix changes attributable to lyophilization. It appears that most of the nonparallelism occurred with the ITAs, but the fact that two ITAs remained parallel with all of the materials shows that nonparallelism is not an ITA technology-specific problem.
All of the PRMs investigated were prepared by methods that dramatically alter the matrix: all were prepared from serum or plasma that was, at a minimum, pooled, delipidated, and filtered. In addition, the matrix control materials (FC and LC) for this study were similarly prepared. Although each IHSP simulated a fresh sample from a patient, it was prepared uniquely by each participant and was not common among the participants. A thorough evaluation of matrix effects and commutability is thus not possible with the data obtained in this study. Commutability will be evaluated in a subsequent study.
The current approach available to manufacturers is to use diluted CRM470 to calibrate their assays. During preparation, CRP was added at the acute-phase-reactant concentration range and was value-assigned through an elaborate protocol involving three different manufacturers’ assays that were calibrated using WHO IS 85/506. CRM470 has characteristics that make it a well-documented, high-quality material, including being human serum-based, optically clear, stable, free of known infective agents, and available in sufficient quantity. This study has shown that it has behavior similar to fresh serum samples when diluted to the range of hsCRP assays.
In conclusion, all of the proposed materials evaluated in this study performed relatively well, but PRM4 performed slightly better overall than the other materials. For this reason and the fact that it has been certified as a reference material, CRM470 will be used in the next phase to harmonize hsCRP assays. On the basis of this study, materials prepared using rCRP or iCRP can be used as working calibrators, quality-control materials, or proficiency testing materials. Newer generations of CRP assays are being developed that have reportable ranges that span the high-sensitivity and acute-phase-reactant ranges; therefore, use of a common reference material for all CRP assays is preferable. All participants in this study claim to use CRM470 to calibrate their assays, but different methods of transferring the value from CRM470 to the manufacturers’ working calibrators clearly are being used. In the next phase of standardization, diluted CRM470 will be used in conjunction with a standard value-transfer protocol.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  H. W. Vesper and L. M. Thienpont Traceability in Laboratory Medicine Clin. Chem., June 1, 2009; 55(6): 1067 - 1075. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. M. Kimberly, S. P. Caudill, H. W. Vesper, E. A. Monsell, W. G. Miller, R. Rej, N. Rifai, F. Dati, and G. L. Myers Standardization of High-Sensitivity Immunoassays for Measurement of C-Reactive Protein; II: Two Approaches for Assessing Commutability of a Reference Material Clin. Chem., February 1, 2009; 55(2): 342 - 350. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. F. P. Man, L. Xing, J. E. Connett, N. R. Anthonisen, R. A. Wise, D. P. Tashkin, X. Zhang, R. Vessey, T. G. Walker, B. R. Celli, et al. Circulating fibronectin to C-reactive protein ratio and mortality: a biomarker in COPD? Eur. Respir. J., December 1, 2008; 32(6): 1451 - 1457. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Rifai, C. M. Ballantyne, M. Cushman, D. Levy, and G. L. Myers Point: High-Sensitivity C-Reactive Protein and Cardiac C-Reactive Protein Assays: Is There a Need to Differentiate? Clin. Chem., July 1, 2006; 52(7): 1254 - 1256. [Full Text] [PDF]
|
|
|
|
|
|
J. V. Callaghan and S. I. Gutman Counterpoint: food and drug administration guidance for C-reactive protein assays: matching claims with performance data. Clin. Chem., July 1, 2006; 52(7): 1256 - 1257. [Full Text] [PDF]
|
|
|
|
 |
|
 P. Ridker, N. Rifai, W. Koenig, and R. S. Blumenthal C-reactive protein and cardiovascular risk in the framingham study. Arch Intern Med, June 26, 2006; 166(12): 1327 - 1328. [Full Text] [PDF]
|
|
|
|
 |
|
 M. Di Napoli, M. Schwaninger, R. Cappelli, E. Ceccarelli, G. Di Gianfilippo, C. Donati, H. C.A. Emsley, S. Forconi, S. J. Hopkins, L. Masotti, et al. Evaluation of C-Reactive Protein Measurement for Assessing the Risk and Prognosis in Ischemic Stroke: A Statement for Health Care Professionals From the CRP Pooling Project Members Stroke, June 1, 2005; 36(6): 1316 - 1329. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. H. Wilson, G. Williams, R. Herrmann, D. Wiesner, and P. Brookhart Issues in Immunoassay Standardization: The ARCHITECT Folate Model for Intermethod Harmonization Clin. Chem., April 1, 2005; 51(4): 684 - 687. [Full Text] [PDF]
|
|
|
|
 |
|
 G. L. Myers, N. Rifai, R. P. Tracy, W. L. Roberts, R. W. Alexander, L. M. Biasucci, J. D. Catravas, T. G. Cole, G. R. Cooper, B. V. Khan, et al. CDC/AHA Workshop on Markers of Inflammation and Cardiovascular Disease: Application to Clinical and Public Health Practice: Report From the Laboratory Science Discussion Group Circulation, December 21, 2004; 110(25): e545 - e549. [Full Text] [PDF]
|
|
|
|
|
|
W. L. Roberts CDC/AHA Workshop on Markers of Inflammation and Cardiovascular Disease: Application to Clinical and Public Health Practice: Laboratory Tests Available to Assess Inflammation--Performance and Standardization: A Background Paper Circulation, December 21, 2004; 110(25): e572 - e576. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Rifai, G. R. Cooper, W. V. Brown, W. Friedewald, R. J. Havel, G. L. Myers, and G. R. Warnick Clinical Chemistry Journal Has Contributed to Progress in Lipid and Lipoprotein Testing for Fifty Years Clin. Chem., October 1, 2004; 50(10): 1861 - 1870. [Full Text] [PDF]
|
|
|
|
|
|
T. B. Ledue and N. Rifai Preanalytic and Analytic Sources of Variations in C-reactive Protein Measurement: Implications for Cardiovascular Disease Risk Assessment Clin. Chem., August 1, 2003; 49(8): 1258 - 1271. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
