|
|
||||||||
Articles |
a Address correspondence to this author at: Centers for Disease Control and Prevention, 4770 Buford Hwy., NE, MS F-18, Atlanta, GA 30341-3724. Fax 770-488-4609; e-mail cfp8{at}cdc.gov
| Abstract |
|---|
|
|
|---|
Methods: We studied 14 laboratories that used eight different method types: HPLC with electrochemical detection (HPLC-ED); HPLC with fluorescence detection (HPLC-FD) further subdivided by type of reducing/derivatizing agent; gas chromatography/mass spectrometry (GC/MS); enzyme immunoassay (EIA); and fluorescence polarization immunoassay (FPIA). Three of these laboratories used two methods. The laboratories participated in a 2-day analysis of 46 plasma samples, 4 additional plasma samples with added homocystine, and 3 plasma quality-control (QC) pools. Results were analyzed for imprecision, recovery, and methodological differences.
Results: The mean among-laboratory and among-run within-laboratory imprecision (CV) was 9.3% and 5.6% for plasma samples, 8.8% and 4.9% for samples with added homocystine, and 7.6% and 4.2% for the QC pools, respectively. Difference plots showed values systematically higher than GC/MS for HPLC-ED, HPLC-FD using sodium borohydride/monobromobimane (however, for only one laboratory), and EIA, and lower values for HPLC-FD using trialkylphosphine/4-(aminosulfonyl)-7-fluoro-2,1,3-benzoxadiazole. The two HPLC-FD methods using tris(2-carboxyethyl) phosphine/ammonium 7-fluoro-2,1,3-benzoxadiazole-4-sulfonate (SBD-F) or tributyl phosphine/SBD-F, and the FPIA method showed no detectable systematic difference from GC/MS.
Conclusions: Among-laboratory variations within one method can exceed among-method variations. Some of the methods tested could be used interchangeably, but there is an urgent need to improve analytical imprecision and to decrease differences among methods.© 1999 American Association for Clinical Chemistry
| Introduction |
|---|
|
|
|---|
With the introduction of tris(2-carboxyethyl)phosphine (TCEP) as a reducing agent (6)(7), the comparability of HPLC results obtained with different reducing/derivatizing agents has become important. Furthermore, the first fully or partially automated kits for the determination of tHcy were introduced recently: a fluorescence polarization immunoassay (FPIA) on the IMx® analyzer [Abbott Laboratories (8)], a microtiter plate enzyme immunoassay [EIA; Bio-Rad Laboratories (9)], an HPLC kit with fluorometric detection (HPLC-FD) after trialkylphosphine reduction and 4-(aminosulfonyl)-7-fluoro-2,1,3-benzoxadiazole (ABD-F) derivatization [Bio-Rad Laboratories (10)], and an HPLC kit with electrochemical detection [HPLC-ED; Bioanalytical Systems (11)].
To assess the performance and the comparability of these methods, the CDC invited national and international clinicians and laboratorians actively involved in homocysteine research, as well as manufacturers of commercially available assay kits, to participate in a round-robin interlaboratory comparison study. Each participant was asked to analyze 53 samples each on 2 separate days.
| Materials and Methods |
|---|
|
|
|---|
30 µmol/L), as well as 3 CDC in-house plasma
quality-control (QC) pools on 2 days. We attempted to include two or
more laboratories performing each of the following methodologies:
HPLC-ED, HPLC-FD [subdivided into tributylphosphine
(TBP)/ammonium 7-fluoro-2,1,3-benzoxadiazole-4-sulfonate (SBD-F),
TCEP/SBD-F, trialkylphosphine/ABD-F, and sodium borohydride
(NaBH4)/monobromobimane (MBrB)], GC/MS, EIA, and
FPIA. Of the 14 participating laboratories, 10 were in the US. The
participants included three manufacturers, two government laboratories,
eight academic laboratories, and one clinical research facility.
Each of three laboratories participated with two different
methods.
specimens
Under a CDC agreement with the Emory University Hospital Blood
Collection Service (including an omnibus informed consent and Human
Subject Review protocol), blood (~20 mL each) was collected from 50
apparently healthy subjects into EDTA-anticoagulated tubes
(Becton-Dickinson) and cooled in ice water immediately after being
drawn. The plasma was separated by centrifugation within 30 min of
venipuncture and stored for a maximum of 1 month at -70 °C before
shipment. An aliquot of each specimen was analyzed for tHcy in the CDC
NHANES laboratory by an HPLC-FD assay (7) with plasma QC
pools at three concentrations (6.9, 13.4, and 29.2 µmol/L). Five
specimens were selected for addition of homocystine and were thawed,
pooled, and redivided into five aliquots;
L-homocystine was added to four of these aliquots
to concentrations of 5.0, 9.9, 14.7, and 19.5 µmol/L free thiol. Two
random numbers were assigned for each sample. All 50 specimens were
dispensed in 0.25-mL aliquots into 2.0-mL Nalge cryovials and placed
promptly at -70 °C until shipment. Each laboratory was sent a
shipment on dry ice containing two boxes with 53 samples each. Extra
aliquots were retained by CDC.
statistical methods
We tested for outliers by calculating the all-laboratory consensus
mean ± 3 SD for each sample and comparing each individual result
with this range. One sample from laboratory 9 was outside of this range
and was excluded from all further calculations.
All evaluations of imprecision, method differences, and recovery, except for those for among-run variability, were based on the mean between the day 1 and day 2 results. The following measures of imprecision were evaluated: among-laboratory, among-method, within-method, and among-run. For each sample, we calculated the among-laboratory CV of the participating laboratories. We expressed the among-laboratory variation as the mean CV (SD) for each sample type (plasma, plasma with added homocystine, and QC pool samples). For calculation of the among-method variation, all laboratories were nested within a method group based on the method they performed. The among-method variation was expressed as the mean CV (SD) of the eight method groups over the three sample types. The within-method variation was calculated for only five of eight method groups because three groups were represented by only one laboratory each. The among-run within-laboratory and among-run within-method variability was expressed as the mean CV (SD) for each laboratory and each method, respectively. P <0.05 was considered statistically significant.
In the absence of target values for the samples analyzed, we considered the GC/MS method arbitrarily as a reference method. Difference plots were used to assess the agreement between tHcy results obtained by GC/MS and all other methods (12)(13). Possible systematic biases were assessed by computing the 95% confidence intervals (mean difference ± 2 SE) for the mean differences between GC/MS and each of the methods included. Limits of agreement were assessed by calculating the central 0.95 intervals (mean difference ± 2 SD). The mean differences and the mean between GC/MS results and the results for each test method were correlated to test for a relationship between these two variables. There was no relationship for the 46 plasma samples. To assess the mean proportional bias between GC/MS and each test method, we calculated the relative ratios of the GC/MS results and test method results.
Recoveries were calculated individually for each sample with added homocystine: recovery (%) = (plasma with added homocystine - plasma without added homocystine)/added concentration of tHcy. Recovery results were reported as the mean (SD) of the four samples with added homocystine analyzed on both day 1 and day 2.
To test for methodological differences, we grouped laboratories by method and performed a two-way ANOVA with laboratory and method as variables, using the SAS GLM procedure and the Bonferroni test.
| Results |
|---|
|
|
|---|
|
imprecision
Shown in Table 2
are the mean among-laboratory, among-method, and within-method
variations for plasma samples (n = 46), samples with added
homocystine (n = 4), and QC pools (n = 3). The mean
among-laboratory variation was 7.6% for the QC pools. There was no
difference in the CV as a function of the concentrations of tHcy: 7.9%
for QC low (6.9 µmol/L), 7.2% for QC medium (13.4 µmol/L), and
7.6% for QC high (28.6 µmol/L). The mean among-method variation was
~1% lower than the among-laboratory variation. Here, the CV was a
function of the concentration of tHcy: 7.7% for QC low (6.9 µmol/L),
6.5% for QC medium (13.3 µmol/L), and 4.9% for QC high (28.3
µmol/L).
|
The within-method variation was lowest for the FPIA assay (laboratories 4B, 8B, 13, and 14) and highest for the HPLC-FD assay using NaBH4/MBrB (laboratories 10 and 11). However, laboratory 10 reported results that were in general higher than the results of all other laboratories. Because of the small number of laboratories included in a method, the within-method variation data must be interpreted cautiously.
The among-run variation (Table 3
) was highest for laboratory 9 (>10%) and for some
laboratories (i.e., 68, 10, and 3B) came close to 10%. The mean
among-run within-laboratory variation was 5.6% for plasma samples,
4.9% for samples with added homocystine, and 4.2% for the QC pools,
respectively. Interestingly, for a~70% of the laboratories,
among-run CVs were lower for the QC pools and for the samples with
added homocystine than for the plasma samples. This indicates that it
is difficult to obtain the same kind of information when different
procedures are used to control the analysis.
|
recovery
Recoveries were 85115% for laboratories other than 10 (121.1%
± 20.9%), 9 (66.0% ± 18.8%), and 6 (75.9% ± 12.0%; Table 3
).
The high recovery of laboratory 10 and the low recovery of laboratory 6
indicate that these laboratories might have a calibration problem. The
low and inconsistent recovery of laboratory 9 paired with the highest
among-run variability among all laboratories indicates that this
laboratory might have a performance problem.
differences among methods and laboratories
Among-method and among-laboratory differences were assessed with
plasma samples (without added homocystine) to avoid unbalanced results
by a few samples with high concentrations. To visualize among-method
inaccuracy, Fig. 1
shows scatter plots of observed measurement differences between
the averaged results of each method type and GC/MS against the mean of
GC/MS and the test method (used by one or more laboratories). The mean
(SD) differences between GC/MS results (laboratory 12) and each
laboratory's results (among-laboratory differences) as well as between
GC/MS and the averaged results of each method type (among-method
differences) are reported in Table 4
. Using the standard errors of the mean differences, we computed
the 95% confidence intervals; these showed an apparent positive bias
for HPLC-ED, HPLC-FD using NaBH4/MBrB (however,
only for laboratory 10), and EIA, and an apparent negative bias for
HPLC-FD using trialkylphosphine/ABD-F. The two HPLC-FD methods using
TCEP/SBD-F or TBP/SBD-F and the FPIA method showed no apparent bias
with respect to the GC/MS method; however, TBP/SBD-F produced a
relatively wider scatter of difference data points.
|
|
The central 0.95 interval (mean difference ± 2 SD) gives an indication of the agreement between GC/MS and the other methods to measure plasma tHcy. Ninety-five percent of tHcy determinations by HPLC-FD using trialkylphosphine/ABD-F were 2.05 to 0.33 µmol/L lower than concentrations determined by GC/MS. This corresponds to a proportional bias of -16.1%. Ninety-five percent of tHcy determinations by HPLC-ED and by EIA were 0.20 and 0.76 µmol/L lower to 1.26 and 1.79 µmol/L higher than concentrations determined by GC/MS. This corresponds to a proportional bias of 7.5% and 7.4%, respectively.
When we grouped laboratories by method type and performed a two-way ANOVA with laboratory and method as variables to compare methods individually, we found no significant differences among the methods except that HPLC-FD trialkylphosphine/ABD-F (laboratory 6) gave significantly lower results than HPLC-FD NaBH4/MBrB (laboratories 10 and 11) and HPLC-ED (laboratories 13). However, the results of laboratories 1 and 11 were not significantly different from the results of laboratory 6 or any other laboratory. Only laboratories 2, 3, and 10 reported significantly higher results than laboratory 6. Because HPLC-FD trialkylphosphine/ABD-F was used by only one laboratory, we could not conclude whether their significantly lower results were method-specific or part of the among-laboratory variation, as was seen with the method HPLC-FD NaBH4/MBrB.
| Discussion |
|---|
|
|
|---|
The mean among-laboratory CV of 9.2% for the 46 plasma samples (without added homocystine) is in good agreement with the 9% among-laboratory CV obtained by Møller et al. (5) for one EDTA plasma sample analyzed by nine laboratories with GC/MS and HPLC methods. In the present study, in which each laboratory and method used its own calibrators, the results of the two laboratories performing HPLC-FD with NaBH4/MBrB demonstrated that laboratory-to-laboratory variations within one method can exceed among-method variations. Although one laboratory showed virtually no apparent bias relative to GC/MS, the other laboratory reported results that were on average 18% higher than the GC/MS results.
When compared with GC/MS, HPLC-FD using TCEP/SBD-F and FPIA showed virtually no apparent bias and narrow limits of agreement. HPLC-FD using trialkylphosphine/ABD-F and EIA showed the highest disagreement with GC/MS.
Objective analysis of whether the imprecision and bias of a method are
satisfactory is difficult to perform. Some have proposed using the
biological variation as the basis for analytical quality specifications
(14). For tHcy, the CVwithin-subject
and the CVbetween-subject were 7% and 33.5%,
respectively (15). A widely held view is that analytical
imprecision (CVA) should be <0.25
CVwithin-subject for optimum performance, <0.50
CVwithin-subject for desirable performance, and
<0.75 CVwithin-subject for minimum performance
(16). In our analysis, this would require an analytical
imprecision of <1.75%, <3.5%, and <5.3% for optimum, desirable,
and minimum performance, respectively. As shown in Table 3
, none of the
laboratories obtained among-run variations <1.75% for all three types
of samples, and only two laboratories (11 and 13) had overall among-run
variations <3.5%. The among-run variations of six laboratories (610
and 3B) exceeded the required CV for minimum performance of 5.3% for
at least two types of samples. Three methods performed best regarding
analytical imprecision: GC/MS, FPIA, and HPLC-FD using TCEP/SBD-F. They
reached the required CV for minimum performance for at least two types
of samples.
For optimum performance, the bias of a method
(BA) should be
<0.125(CVwithin-subject2 +
CVbetween-subject2)1/2;
for desirable performance, it should be
<0.25(CVwithin-subject2 +
CVbetween-subject2)1/2;
and for minimum performance, it should be
<0.375(CVwithin-subject2 +
CVbetween-subject2)1/2
(16). For our analysis, this would mean a bias of <4.3%,
<8.6%, and <12.8% for optimum, desirable, and minimum performance,
respectively. As shown in Table 4
, several laboratories met the
requirement for optimum performance (4, 5, 8, 9, 11, 13, 14, 4B, and
8B) compared with GC/MS. However, two laboratories (6 and 10)
did not meet the requirement for minimum performance. The following
three methods performed best regarding apparent analytical bias (vs
GC/MS): FPIA and HPLC-FD using TCEP/SBD-F or TBP/SBD-F. It must be
cautioned, however, that the GC/MS method may itself be biased and that
until high-level reference methods for tHcy are available, little can
be said about a laboratory's or a method's bias.
Our analysis for methodological differences in mean concentrations showed significant differences for two method comparisons: HPLC-FD trialkylphosphine/ABD-F compared with HPLC-FD NaBH4/MBrB and compared with HPLC-ED. Although we could not confirm that these differences were method specific, Dias et al. (17) reported a significant negative bias of 5.2 µmol/L when they compared the new Bio-Rad HPLC method with a SBD-F assay.
Gilfix et al. (6) introduced TCEP as a novel and more suitable reductant for the routine determination of plasma tHcy. In a small method comparison study, they found that the TCEP method yielded values ~21% higher than the TBP method. We previously directly compared results obtained with these two reducing agents and found that although the use of TCEP produced higher relative fluorescence intensities, the calculated concentrations of plasma tHcy were not significantly different if calibration was performed in plasma and cystamine was used as the internal standard (7). The 20% difference seen by Gilfix et al. (6) could have been a result of the different calibration matrices that were used in that comparison (saline for TCEP vs plasma for TBP). We showed that if calibration was performed in saline and no internal standard (which could correct for the matrix effect) was added, tHcy concentrations were ~20% higher than when calibration was performed in plasma (7). The present round-robin showed very good agreement between results obtained with TCEP as reductant and those obtained by most other methods.
The performance of the Abbott homocysteine assay is of great interest to many clinical laboratories because this FPIA is fully automated and requires no sample pretreatment step. In a large method comparison study (n = 811 plasma and serum samples), we previously found a very good correlation between the FPIA assay and our HPLC assay with fluorometric detection (y = -0.110 + 0.992x; r2 = 0.986) (18). In the present interlaboratory comparison study, all four participating laboratories (13, 14, 4B, and 8B) that used the Abbott homocysteine assay produced results that were in good agreement with each other and with results obtained by most other methods.
Although we found no correlation between the performance of the different methods and the use of an internal standard or between the use of plasma calibration vs aqueous calibration, we did find that laboratories whose results did not agree well with the GC/MS results also usually showed higher among-run imprecision and lower and more variable recoveries.
This international round-robin for plasma tHcy showed good agreement between some of the most experienced laboratories performing different methods. It answered important questions concerning the performance of TCEP as a novel reductant and the Abbott homocysteine assay. Although the results indicate that some of the methods tested could be used interchangeably, the analysis for analytical quality specifications have shown that overall there is an urgent need to improve analytical imprecision and, for some methods, an apparent need to decrease analytical bias to guarantee that laboratories throughout a homogeneous population area can use the same reference intervals. We believe that improvement can be aided by the introduction of standard reference materials and more external quality assessment programs.
| Acknowledgments |
|---|
| Footnotes |
|---|
1 Nonstandard abbreviations: tHcy, total homocysteine; GC/MS, gas chromatography/mass spectrometry; TCEP, tris(2-carboxyethyl)phosphine; FPIA, fluorescence polarization immunoassay; EIA, enzyme immunoassay; HPLC-FD, HPLC with fluorometric detection; ABD-F, 4-(aminosulfonyl)-7-fluoro-2,1,3-benzoxadiazole; HPLC-ED, HPLC with electrochemical detection; QC, quality control; TBP, tributylphosphine; SBD-F, ammonium 7-fluoro-2,1,3-benzoxadiazole-4-sulfonate; NaBH4, sodium borohydride; and MBrB, monobromobimane. ![]()
| References |
|---|
|
|
|---|
The following articles in journals at HighWire Press have cited this article:
![]() |
H.-K. Kuo, K.-C. Liao, S. G. Leveille, J. F. Bean, C.-J. Yen, J.-H. Chen, Y.-H. Yu, and T.-Y. Tai Relationship of Homocysteine Levels to Quadriceps Strength, Gait Speed, and Late-Life Disability in Older Adults J. Gerontol. A Biol. Sci. Med. Sci., April 1, 2007; 62(4): 434 - 439. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Guallar, E. K. Silbergeld, A. Navas-Acien, S. Malhotra, B. C. Astor, A. R. Sharrett, and B. S. Schwartz Confounding of the Relation between Homocysteine and Peripheral Arterial Disease by Lead, Cadmium, and Renal Function Am. J. Epidemiol., April 15, 2006; 163(8): 700 - 708. [Abstract] [Full Text] [PDF] |
||||
![]() |
H.-K. Kuo, J. F. Bean, C.-J. Yen, and S. G. Leveille Linking C-Reactive Protein to Late-Life Disability in the National Health and Nutrition Examination Survey (NHANES) 1999-2002. J. Gerontol. A Biol. Sci. Med. Sci., April 1, 2006; 61(4): 380 - 387. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. E. Mignini, P. M. Latthe, J. Villar, M. D. Kilby, G. Carroli, and K. S. Khan Mapping the Theories of Preeclampsia: The Role of Homocysteine Obstet. Gynecol., February 1, 2005; 105(2): 411 - 425. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. J. Huijgen, F. P.W. Tegelaers, C. H.H. Schoenmakers, C. J. Pronk-Admiraal, and S. Ekema Multicenter Analytical Evaluation of an Enzymatic Method for the Measurement of Plasma Homocysteine and Comparison with HPLC and Immunochemistry Clin. Chem., May 1, 2004; 50(5): 937 - 940. [Full Text] [PDF] |
||||
![]() |
H. Refsum, A. D. Smith, P. M. Ueland, E. Nexo, R. Clarke, J. McPartlin, C. Johnston, F. Engbaek, J. Schneede, C. McPartlin, et al. Facts and Recommendations about Total Homocysteine Determinations: An Expert Opinion Clin. Chem., January 1, 2004; 50(1): 3 - 32. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. Tan, X. Sun, L. Tang, N. Zhang, Q. Han, M. Xu, X. Tan, X. Tan, and R. M. Hoffman Automated Enzymatic Assay for Homocysteine Clin. Chem., June 1, 2003; 49(6): 1029 - 1030. [Full Text] [PDF] |
||||
![]() |
M. R. Fokkema, M. F. Gilissen, J. J. van Doormaal, M. Volmer, I. P. Kema, and F. A.J. Muskiet Fasting vs Nonfasting Plasma Homocysteine Concentrations for Diagnosis of Hyperhomocysteinemia Clin. Chem., May 1, 2003; 49(5): 818 - 821. [Full Text] [PDF] |
||||
![]() |
H. M. Blanck, B. A. Bowman, G. R. Cooper, G. L. Myers, and D. T. Miller Laboratory Issues: Use of Nutritional Biomarkers J. Nutr., March 1, 2003; 133(3): 888S - 894. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Q. Hanson, J. H. Eckfeldt, K. Schwichtenberg, O. Aras, and M. Y. Tsai Interlaboratory Variation of Plasma Total Homocysteine Measurements: Results of Three Successive Homocysteine Proficiency Testing Surveys Clin. Chem., September 1, 2002; 48(9): 1539 - 1545. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. Kelly, J. Rosand, J. P. Kistler, V. E. Shih, S. Silveira, A. Plomaritoglou, and K. L. Furie Homocysteine, MTHFR 677C->T polymorphism, and risk of ischemic stroke: Results of a meta-analysis Neurology, August 27, 2002; 59(4): 529 - 536. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Krijt, M. Vackova, and V. Kozich Measurement of Homocysteine and Other Aminothiols in Plasma: Advantages of Using Tris(2-carboxyethyl)phosphine as Reductant Compared with Tri-n-butylphosphine Clin. Chem., October 1, 2001; 47(10): 1821 - 1828. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Zhang, E. W. Gunter, and C. M. Pfeiffer Evaluation of the Drew Scientific DS30 Homocysteine Assay in Comparison with the Centers for Disease Control and Prevention Reference HPLC Method Clin. Chem., May 1, 2001; 47(5): 966 - 967. [Full Text] [PDF] |
||||
![]() |
C. M. Pfeiffer, S. P. Caudill, E. W. Gunter, B. A. Bowman, P. F. Jacques, J. Selhub, C. L. Johnson, D. T. Miller, and E. J. Sampson Analysis of Factors Influencing the Comparison of Homocysteine Values between the Third National Health and Nutrition Examination Survey (NHANES) and NHANES 1999+ J. Nutr., November 1, 2000; 130(11): 2850 - 2854. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. Tan, L. Tang, X. Sun, N. Zhang, Q. Han, M. Xu, E. Baranov, X. Tan, X. Tan, B. Rashidi, et al. Total-Homocysteine Enzymatic Assay, Clin. Chem., October 1, 2000; 46(10): 1686 - 1688. [Full Text] [PDF] |
||||
![]() |
E. Nexo, F. Engbaek, P. M. Ueland, C. Westby, P. O'Gorman, C. Johnston, B. F. Kase, A. B. Guttormsen, I. Alfheim, J. McPartlin, et al. Evaluation of Novel Assays in Clinical Chemistry: Quantification of Plasma Total Homocysteine Clin. Chem., August 1, 2000; 46(8): 1150 - 1156. [Abstract] [Full Text] [PDF] |
||||
![]() |
Assessment of Laboratory Tests for Plasma Homocysteine--Selected Laboratories, July-September 1998 JAMA, December 8, 1999; 282(22): 2112 - 2113. [Full Text] [PDF] |
||||
![]() |
C. M. Pfeiffer, S. J. Smith, D. T. Miller, and E. W. Gunter Comparison of Serum and Plasma Methylmalonic Acid Measurements in 13 Laboratories: An International Study Clin. Chem., December 1, 1999; 45(12): 2236 - 2242. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |