Clinical Chemistry Link to Randox Laboratories Web Site
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


Clinical Chemistry 45: 1803-1812, 1999;
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit an electronic Letter to
the Editor about this paper
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (18)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kimberly, M. M.
Right arrow Articles by Waymack, P. P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kimberly, M. M.
Right arrow Articles by Waymack, P. P.
Related Collections
Right arrow Lipids, Lipoproteins, and Cardiovascular Risk Factors
(Clinical Chemistry. 1999;45:1803-1812.)
© 1999 American Association for Clinical Chemistry, Inc.


Articles

Selection, Validation, Standardization, and Performance of a Designated Comparison Method for HDL-Cholesterol for Use in the Cholesterol Reference Method Laboratory Network

Mary M. Kimberly1,a, Elizabeth T. Leary2, Thomas G. Cole3, Parvin P. Waymack1 and for the Cholesterol Reference Method Laboratory Network,4

1 Division of Laboratory Sciences, National Center for Environmental Health, Centers for Disease Control and Prevention, Atlanta, GA 30341-3724.

2 Pacific Biometrics Research Foundation, Seattle, WA 98119.

3 Core Laboratory for Clinical Studies, Washington University School of Medicine, St. Louis, MO 63110.

4 David Hassemer (State Laboratory of Hygiene, Madison, WI); Santica Marcovina (Northwest Lipid Research Laboratories, Department of Medicine, University of Washington, Seattle, WA); Robert Rej (Wadsworth Center for Laboratories and Research, New York State Department of Health, Albany, NY); Thomas G. Cole (Core Laboratory for Clinical Studies, Washington University School of Medicine, St. Louis, MO); Elizabeth T. Leary (Pacific Biometrics Research Foundation, Seattle, WA); Christa M. Boersma-Cobbaert (Lipid Reference Laboratory, Department of Clinical Chemistry, University of Rotterdam, Rotterdam, The Netherlands); Masakazu Nakamura (Osaka Medical Center for Cancer and Cardiovascular Diseases, Osaka, Japan); Chris J. Packard (Institute of Biochemistry, Glasgow Royal Infirmary, Glasgow, Scotland); David W. Seccombe [Canadian Reference Laboratory (1996) Ltd., Vancouver, Canada]; Ferruccio Ceriotti (Laboratorio Analisi Cliniche, H.S. Raffaele, Milan, Italy). Inactive laboratories that also participated in the study: John H. Eckfeldt (Department of Laboratory Medicine and Pathology, University of Minnesota Hospital and Clinic, Minneapolis, MN); Joan A. Waletzky (Department of Biochemistry, The Cleveland Clinic Foundation, Cleveland, OH); Judith R. McNamara (Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA).
a Address correspondence to this author at: Centers for Disease Control and Prevention, Mailstop F25, 4770 Buford Hwy. NE, Atlanta, GA 30341-3724. Fax 770-488-4192; e-mail mmk1{at}cdc.gov


   Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 
Background: Accurate and precise HDL-cholesterol (HDL-C) measurements are essential for effective application of National Cholesterol Education Program treatment guidelines. The Cholesterol Reference Method Laboratory Network (CRMLN) assists manufacturers of in vitro diagnostic products to establish traceability to the accuracy base. CRMLN sought to implement a designated comparison method (DCM) that overcomes the impracticalities of the expensive and labor-intensive reference method for HDL-C.

Methods: CRMLN evaluated candidate DCMs and selected one that uses 50-kDa dextran sulfate with magnesium ions as the precipitation reagent followed by measurement of cholesterol by the CDC reference method. After validating the method, we transferred it to all CRMLN laboratories and successfully standardized it using CDC frozen serum reference materials. CRMLN laboratories participate in monthly performance evaluations.

Results: CRMLN laboratories were able to meet a precision goal, as indicated by SD, of <=0.03 mmol/L (1 mg/dL) 94.4% of the time. They were able to meet a bias goal of <=0.05 mmol/L (2 mg/dL) for HDL-C <1.09 mmol/L (42 mg/dL) 97.3% of the time and a goal of <=3% for HDL-C >=1.09 mmol/L (42 mg/dL) 95.6% of the time. CRMLN is working to further improve its performance by implementing a bias criterion of 0.03 mmol/L (1 mg/dL) for all HDL-C concentrations.

Conclusions: CRMLN selected, validated, standardized, and implemented a DCM for HDL-C that is accurate, robust, transferable, and practical. The DCM is being used to assist manufacturers in calibrating their products so that ultimately, clinical laboratories using the products will more accurately measure HDL-C.


   Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 
HDL-cholesterol (HDL-C)1 has been established as an independent risk factor for coronary heart disease (CHD) (1)(2), leading to the increased measurement of HDL-C by clinical laboratories. With total cholesterol (TC), LDL-cholesterol, and triglycerides (TGs), HDL-C is part of the lipoprotein profile used to estimate an individual's risk for developing CHD (3). The National Cholesterol Education Program (NCEP) Adult Treatment Panel II has associated an increased risk of developing CHD with concentrations of HDL-C <0.91 mmol/L (35 mg/dL); HDL-C >=1.55 mmol/L (60 mg/dL) is considered protective against CHD (2).

Because accurate and reproducible HDL-C measurements are essential for effective application of NCEP treatment guidelines, NCEP established analytical performance goals for HDL-C measurement (4)(5). These goals were stated in terms of total analytical error, which takes into account inaccuracy and imprecision. The goal is total error <=13%. However, for the purposes of standardization, the CDC's Cholesterol Reference Method Laboratory Network (CRMLN) encourages manufacturers to strive to meet separate accuracy and precision goals. The 1998 performance goals recommended by the NCEP that are consistent with the total error goal are a bias vs the accuracy base <=5% and a precision of CV <=4% at HDL-C >=1.09 mmol/L (42 mg/dL) and SD <=0.044 mmol/L (1.7 mg/dL) at HDL-C <1.09 mmol/L (42 mg/dL) (5).

In 1990, CRMLN established a process by which manufacturers can establish traceability to the National Reference System for Cholesterol of their diagnostics products used to measure TC (Myers GL, Kimberly MM, Waymack PP, Smith SJ, Cooper GR, Sampson EJ, manuscript in preparation). Traceability is accomplished by performing a fresh-sample method comparison, which is based on NCCLS EP9-A protocol (6), between the diagnostic product and the reference method for cholesterol by analyzing a minimum of 40 fresh serum specimens. When a product meets the NCEP performance goals, the manufacturer is issued a dated Certificate of Traceability. NCEP's Working Group on Lipoprotein Measurement recommended that CRMLN provide a similar program for HDL-C (5). We report here on the process that CRMLN used to select, validate, standardize, and implement a designated comparison method (DCM) for HDL-C, which has been used since 1994 to evaluate the accuracy of diagnostic products for measuring HDL-C.

The CDC reference method for HDL-C has served as the reference point for the CDC-National Heart, Lung, and Blood Institute (NHLBI) Lipid Standardization Program (LSP) since 1981 (7). Because NCEP used studies in which laboratory measurements were standardized by the LSP to establish the medical decision points for assessing an individual's risk of developing CHD, NCEP recommended that the CDC reference method for HDL-C continue to serve as the accuracy base for HDL-C measurements (5).

In the CDC reference method for HDL-C, 5-mL serum specimens are ultracentrifuged to separate HDL, LDL, intermediate-density lipoprotein, and lipoprotein a (d >1.006 kg/L bottom fraction) from VLDL and chylomicrons (d <1.006 kg/L top fraction) to remove TG interference. After LDL, intermediate-density lipoprotein, and lipoprotein (a) in the bottom fraction are precipitated with a heparin-manganese (HM) reagent, HDL-C in the supernatant is measured directly with the CDC reference method for cholesterol (8)(9).

CRMLN carefully considered using the CDC reference method for HDL-C, but it was deemed impractical for accuracy transfer via method comparisons that require 40 or more fresh serum specimens. Primary issues were (a) the large specimen volume required (5 mL), and (b) the high cost in time, skilled labor, and equipment because of the involvement of ultracentrifugation. CRMLN therefore looked for a DCM for HDL-C that would address each of these issues and be a practical way for manufacturers to establish traceability to the accuracy base. The ideal DCM would be accurate, robust, transferable, and practical for use in the CRMLN. It would also be traceable to the CDC reference method for HDL-C by standardization and regular surveillance of CRMLN laboratories. These criteria should be met by applying precipitation directly to serum without prior ultracentrifugation if maximum TG concentrations were specified.


   Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 
calibrators and controls
Pure cholesterol (NIST SRM 911b) was used to prepare the cholesterol standards for the reference method for cholesterol. CDC prepared frozen human serum-based quality-control (QC) materials: a HDL-C control (AQ14) and two cholesterol controls—one in the HDL-C concentration range (MQ8) and one in the TC concentration range. The method for preparing these materials is described elsewhere (10). CDC shipped the QC materials to the CRMLN laboratories on dry ice. The laboratories stored the QC materials at -70 °C.

phase i: selection of a dcm for hdl-c
Selection of candidate DCMs.
A task force, consisting of representatives of the CDC and two CRMLN laboratories, Pacific Biometrics Research Foundation (PBRF) and Washington University School of Medicine (WUSM), was appointed to evaluate the candidate methods. All methods were two-step procedures that involved removing the apolipoprotein B-containing lipoproteins (chylomicrons, VLDL, intermediate-density lipoprotein, LDL, and lipoprotein a) by precipitation and measuring the remaining lipoprotein cholesterol, HDL-C, in the supernatant using the CDC reference method for cholesterol. The task force initially considered four precipitation reagents: HM; 50-kDa dextran sulfate-magnesium (DS); phosphotungstic acid (PTA); and polyethylene glycol. The task force eliminated polyethylene glycol because of the difficulty of pipetting this viscous reagent. It ranked the remaining reagents using nine criteria based on the experience of the CRMLN laboratories with each reagent and a literature review. As a result, the task force eliminated PTA because of a lack of consistency in the literature of the specific reagent concentrations and methodologies used and concern about the stability of the assay, which is highly sensitive to reagent pH and magnesium concentrations. There was not a specific PTA method that was readily apparent as agreeing with the reference method for HDL-C. The large dilution factor that is typical of both the polyethylene glycol and PTA methods also contraindicates use of these reagents with the reference method for cholesterol because samples with low HDL-C concentrations are diluted below the analytical range.

Because accuracy was the most important criterion, the task force performed a study to assess accuracy of the two candidate methods (DS and HM) relative to the reference method for HDL-C (11). The same precipitation reagent was used in the reference method for HDL-C and the HM candidate DCM. The DS candidate DCM was modified from the method of Warnick et al. (12) by reducing the magnesium chloride (MgCl2) concentration by 30% to improve accuracy vs the reference method.

Comparison specimens.
The Transfusion Services Unit of Emory University Clinic, Atlanta, GA, obtained informed consent and collected blood from 20 fasting donors. The procedures followed for collection were approved by CDC's institutional review board. Whole blood was collected in glass containers without any additive and allowed to clot for 1 h at room temperature before the serum was separated by centrifugation. Within 8 h after collection, the serum was dispensed by 2.3-mL aliquots into sterile vials and frozen at -70 °C. HDL-C concentration were between 0.78 and 1.81 mmol/L (30 and 70 mg/dL); all TG concentrations were <2.26 mmol/L (200 mg/dL). The fresh-frozen samples were shipped to the participating laboratories on dry ice and stored at -70 °C.

Candidate method DS: dextran sulfate-magnesium chloride method.
Dextran sulfate (Dextralip 50) was obtained from Genzyme. Magnesium chloride (MgCl2 · 6H2O) and sodium azide (NaN3) were obtained from Sigma Chemical. The DS reagent was prepared by mixing equal volumes of a stock solution containing 20 g/L DS and 0.5 g/L NaN3 and a stock solution containing 0.7 mol/L MgCl2 and 0.5 g/L NaN3. This working reagent contained 10 g/L DS and 0.35 mol/L MgCl2. The solution was stored between 2 and 8 °C. The three task force laboratories used the same working reagent solution prepared at PBRF and shipped to CDC and WUSM at 4 °C.

The samples and working reagent were equilibrated to room temperature and mixed at a ratio of 10 volumes specimen to 1 volume reagent. Final reagent concentrations were 0.0091 g/L DS and 0.032 mol/L MgCl2. The samples were then incubated at room temperature for 10–30 min and centrifuged for 30 min at 4 °C and 1500g (or equivalent "g-minutes"). Clear supernatants were analyzed using the reference method for cholesterol. Supernates that could not be analyzed immediately were stored at -20 °C for up to 72 h.

Candidate method HM: heparin-manganese method.
A 5000 USP units/mL heparin solution, derived from porcine intestinal mucosa, was obtained from Wyeth-Ayerst Laboratories and used as is. It was stored at 4 °C. Manganese chloride (MnCl2) was obtained from Fisher Chemical. All three task force laboratories used the same working reagent solution prepared at CDC and shipped to PBRF and WUSM at 4 °C.

The samples and working reagent were equilibrated to room temperature. To 2 mL of serum were sequentially added 80 µL of heparin reagent and 100 µL of 1 mol/L MnCl2. Final reagent concentrations were 183 USP units/mL heparin and 0.046 mol/L MnCl2. Samples were incubated in an ice water bath for 30 min and centrifuged for 30 min at 4 °C and 1500g (or equivalent g-minutes). Clear supernatants were analyzed using the reference method for cholesterol. Supernates that could not be analyzed immediately were stored at 4 °C for up to 72 h.

Test run composition.
The laboratories analyzed individual samples by each of the candidate methods in the same analytical run. They also analyzed the three CDC QC materials in every analytical run. CDC analyzed all samples in quadruplicate; WUSM and PBRF analyzed all samples in duplicate. In addition to the candidate DCMs, CDC analyzed the samples using the reference method for HDL-C (9). The throughput was three to four fresh-frozen samples per analytical run in all three laboratories.

Statistics.
The number of samples required to have adequate power to detect a difference between two methods depends on the CV of the method being tested. The lower the CV, the fewer the samples required. Preliminary studies performed with similar methods showed that CRMLN laboratories had CVs between 2% and 3%. Assuming that a CV of 2% could be achieved, the three participating laboratories would have 80% statistical power to detect differences of 1.25% from the reference method when each laboratory analyzed 21 samples in duplicate by each of the methods.

phase ii: validation of ds method as the dcm
Frozen vs fresh serum comparison.
Eleven CRMLN laboratories participated in this part of the study. Each of the participating laboratories analyzed between 14 and 20 samples, for a total of 209 samples. Samples were analyzed in three to six analytical runs in each laboratory. Only samples with TGs <2.26 mmol/L (200 mg/dL) were used.

The serum was divided into two aliquots. Within 8 h of collection, analysis of one aliquot by the DCM and freezing of the other aliquot at -70 °C was begun at the same time. Between 11 and 21 days, the frozen samples were thawed and analyzed by the DCM. The three CDC QC materials were analyzed in every analytical run. All analyses were performed in duplicate.

Statistics.
It was desirable to have adequate power to detect a 1% difference between the two sample types. At the time of this study, the CRMLN laboratories were required to maintain a CV of 3% for the DCM. The CRMLN set a goal of obtaining 240 samples for the entire study, with the sample load divided equally among the participating laboratories. This sample number provided 95% power of detecting a 1% difference between results for fresh and frozen sera. The laboratories performed three to six test runs to minimize run-specific bias.

The data set was evaluated to determine whether any samples exceeded the 99% range for duplicates. If results from fresh or frozen samples were outside the 99% range for duplicates, based on the QC range limits for the HDL-C control for the laboratory, they were omitted from the data set. The data set was also evaluated for outliers. The bias and relative bias (frozen vs fresh) were determined for each sample. The mean, SD, and an outlier test statistic (four times the SD) were calculated for each type of bias. Samples with both bias and relative bias that exceeded the test statistic were eligible outliers.

Linearity.
Blood from two fasting donors was collected and processed as described in phase I. The HDL-C concentration of the serum from these two donors was 0.80 and 2.47 mmol/L (31.0 and 95.5 mg/dL); both had TG concentrations <2.26 mmol/L (200 mg/dL.) The linearity study was performed according to the NCCLS EP6-P protocol (13). Briefly, this involves mixing the two serum samples in proportions to give five equally distributed HDL-C concentrations, analyzing each for HDL-C in quadruplicate and carrying out the EP6-P statistical analysis.

Working reagent stability.
Each of five participating laboratories collected three to six fresh sera. Separate aliquots of each specimen and the CDC QC material were precipitated by reagent that was 18 or 24 months old and by reagent that was less than 12 months old. The resulting supernatants were analyzed in duplicate in the same analytical run. The participating laboratories analyzed a total of 32 fresh serum samples in eight analytical runs over a 2-year period. The total number of measurements for the 18- and 24-month-old reagents were 40 and 54, respectively. One CDC HDL-C QC material was included in every analytical run.

phase iii: standardization of the dcm in the crmln
Reference materials used for standardization.
The materials used for standardization of the CRMLN laboratories were prepared by CDC (10) and had reference values (RVs) assigned by the CDC's reference method for HDL-C. The RVs for these materials are the means of 12 analytical runs in quadruplicate over a period of 2-3 months.

Standardization of CRMLN laboratories.
Previous studies with TC have shown that accuracy is the most important variable in CRMLN's ability to assess performance (14). Therefore, CRMLN laboratories are required to meet more stringent performance goals than those established by NCEP for clinical laboratories. At the time CRMLN laboratories were initially standardized, NCEP performance goals for clinical laboratories were bias vs the accuracy base <=10% and precision of CV <=6% at HDL-C >=1.09 mmol/L (42 mg/dL) and SD <=0.064 mmol/L (2.5 mg/dL) at HDL-C <1.09 mmol/L (5). Initially, CRMLN set as its goal to achieve accuracy and precision that was one-half the NCEP goals for clinical laboratories.

To help minimize potential lot-to-lot variations in the precipitation reagent, all laboratories used the same lot of DS. The first step in the standardization process was a transfer challenge in April 1994. Each laboratory analyzed three pooled, frozen CDC reference materials in duplicate in four analytical runs, and their results were compared to the CDC RVs. In the next standardization step, the laboratories were sent a CDC-NHLBI LSP Part II challenge—four pooled, frozen CDC reference materials analyzed randomly in eight analytical runs (7). The three CDC QC materials were analyzed in every analytical run for both the transfer challenge and the LSP Part II.

phase iv: performance of the dcm in the crmln
Reference materials used for performance monitoring.
Standardization of the DCM in CRMLN laboratories was maintained through monthly surveys. The laboratories were sent CDC frozen reference materials for two monthly surveys per quarter and fresh-frozen materials once per quarter. The RVs for the CDC reference materials were obtained as described above; those for the fresh-frozen materials were the means of four analytical runs in quadruplicate with the reference method for HDL-C performed at CDC.

Performance monitoring schemes.
Three different schemes have been used and three different performance criteria have been applied since the surveys began. Between June 1994 and November 1995, laboratories analyzed three samples in duplicate in four analytical runs for each survey (scheme 1). The performance criteria were as follows: for HDL-C <=1.09 mmol/L (42 mg/dL), a bias of <=0.05 mmol/L (2 mg/dL) vs CDC and a SD of <= 0.04 mmol/L (1.7 mg/dL); for HDL-C >1.09 mmol/L (42 mg/dL), a bias of <=5.0% vs CDC and a CV of <=3.0% (A criteria.) At this time, CRMLN had an unofficial goal for bias of <=3.0% vs CDC for HDL-C >1.09 mmol/L (42 mg/dL; A' criteria).

Between December 1995 and August 1998, the monthly surveys contained four samples that were analyzed in duplicate in two analytical runs (scheme 2). During part of this time (December 1995 to July 1997), the performance was evaluated with the A and A' criteria, as in scheme 1. In August 1997, CRMLN tightened the performance criteria to a bias vs CDC of <=0.03 mmol/L (1 mg/dL) and SD <=0.03 mmol/L (1 mg/dL) across the concentration range (B criteria).

In response to the more stringent 1998 NCEP performance goal for clinical laboratories, CRMLN modified the survey scheme to provide greater statistical power to assess its performance. In September 1998, CRMLN changed the survey scheme so that the laboratories now analyze four samples in duplicate in four analytical runs.

QC materials.
CDC provides all CRMLN laboratories with QC materials as described above. When old QC materials are replaced, each CRMLN laboratory performs a minimum of 20 overlapping analytical runs with the old and new material. CDC establishes network QC limits based on the performance of all laboratories. CRMLN laboratories apply both their own limits and the network limits to all analytical runs.


   Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 
phase i: selection of a dcm for hdl-c
Precision.
A summary of the precision of the task force laboratories is shown in Table 1 . Paired SD, calculated from the method comparison results, as well as SD and CV calculated from the HDL-C control are shown in Table 1 . The paired SD, an indicator of within-run precision, was calculated as follows:

where D is the difference between duplicate measurements of the fresh-frozen samples and n is the number of samples (15). Although the paired SD values tend to show that the DS method has slightly better precision for fresh-frozen specimens, there is not a clear difference between the DS and HM methods when HDL-C QC data are considered.


View this table:
[in this window]
[in a new window]
 
Table 1. Precision of HDL-C measurement in task force laboratories.1

Bias vs the reference method For HDL-C.
The mean percentage of bias, the mean absolute percentage of bias, and regression statistics for the comparison between the candidate methods performed in all three task force laboratories and the reference method for HDL-C performed at CDC are shown in Table 2 .


View this table:
[in this window]
[in a new window]
 
Table 2. Summary of statistics of candidate DCMs vs reference method for HDL-C.1

phase ii: validation of ds method as the dcm
Frozen vs fresh serum comparison.
Manufacturers are required to analyze fresh samples (samples <8 h old) in the method comparisons conducted with the CRMLN, but for practical reasons, CRMLN laboratories use frozen samples. CRMLN undertook a study to compare results from fresh and frozen serum (16).

Five samples (four from one laboratory) were omitted from the data set because they exceeded the 99% range for duplicates. Five outliers were also omitted, leaving 199 samples for the final analysis. The HDL-C concentration range for the samples was 0.28–2.46 mmol/L (11–95 mg/dL), with a mean concentration (± SD) of 1.33 ± 0.38 mmol/L (51.5 ± 14.6 mg/dL). The sample distribution was gaussian.

The mean CV for the 11 laboratories during this study was 1.2% (range, 0.49–2.2%) as determined from results of the HDL-C QC material. At the observed CV, there was 95% power to detect a difference of 0.5% with 199 samples.

Regression analysis of frozen sample results vs fresh sample results yielded a linear relationship, with a correlation coefficient of 0.997, a slope of 0.992, and an intercept of -0.0005 mmol/L (-0.021 mg/dL). The bias was not dependent on HDL-C concentration (Fig. 1 ). Both the bias and relative bias were analyzed by univariate analysis. As a further test of concentration dependence, the sample set was divided into approximately equal parts, one greater than and the other less than the median concentration [1.32 mmol/L (51 mg/dL)]. These two sets of data were also subjected to univariate analysis. The results of all the univariate analyses are shown in Table 3 . The univariate analysis showed a small but statistically significant difference of 0.010 mmol/L (-0.4 mg/dL), or -0.81%, between the frozen and fresh samples analyzed by the DCM.



View larger version (17K):
[in this window]
[in a new window]
 
Figure 1. Bias plot of HDL-C in frozen serum vs fresh serum, plotted as a function of the fresh serum concentration.

To convert mg/dL to mmol/L, multiply mg/dL by 0.0259.


View this table:
[in this window]
[in a new window]
 
Table 3. Frozen vs fresh sample validation of DCM for HDL-C: Univariate analysis1 of bias and percentage of bias.

Linearity.
The linearity study showed a deviation from the estimated line of less than 0.018 mmol/L (0.68 mg/dL) with 95% confidence over the analytical range of 0.80–2.46 mmol/L (31–95 mg/dL) HDL-C.

Working reagent stability.
Initially, the limit on the shelf life of the working reagent was set at 12 months. Because preparation of a new reagent batch requires expensive, time-consuming cross-over studies between old and new batches, CRMLN evaluated the shelf life of batches at 18 and 24 months.

The mean concentration difference between samples analyzed using working reagent 18 or 24 months old and working reagent <12 months old was 0.0026 mmol/L (0.1 mg/dL) HDL-C. The SD of the means was 0.0181 mmol/L (0.7 mg/dL) for the 18-month-old reagent and 0.0207 mmol/L (0.8 mg/dL) for the 24-month-old reagent. The Student t-test showed that there was no significant difference between the results. The P values were 0.35 for the 18-month reagent and 0.34 for the 24-month reagent; therefore, CRMLN extended the acceptable shelf life of the working reagent to 24 months.

phase iii: standardization of the dcm in the crmln
During the transfer challenge, all but two of the laboratories met the <=5% criterion for bias (Fig. 2 A) for all of the materials analyzed; all but three met the <=3% criterion for CV. During the LSP Part II standardization, all of the laboratories met the <=5% criteria for bias (Fig. 2B ); all but three met the <=3% criterion for CV.



View larger version (20K):
[in this window]
[in a new window]
 
Figure 2. Bias plots of HDL-C in CDC reference materials during transfer (A) and standardization (B) of DCM.

(A), plot of percentage of bias of HDL-C in CDC reference materials during transfer of DCM to CRMLN laboratories, plotted by laboratory. {blacktriangledown}, RV = 0.70 mmol/L (27.2 mg/dL); {blacksquare}, RV = 1.30 mmol/L (50.3 mg/dL); •, RV = 2.05 mmol/L (79.0 mg/dL). (B), plot of percentage of bias of HDL-C in CDC reference materials during standardization of DCM in CRMLN laboratories, plotted by laboratory. {blacktriangledown}, RV = 0.70 mmol/L (27.2 mg/dL); {blacksquare}, RV = 1.12 mmol/L (43.3 mg/dL); •, RV = 1.12 mmol/L (43.1 mg/dL); {blacktriangleup}, RV = 1.53 mmol/L (59.1 mg/dL).

CRMLN laboratories that had a bias vs CDC of <=5% and CVs <=3% were considered standardized and could begin providing reference services to manufacturers. CRMLN completed standardization of the DCM for HDL-C by July 1994.

phase iv: performance of the dcm in the crmln
Precision.
The ability of CRMLN laboratories to meet the precision criteria are shown in terms of pass rates in Table 4 . The pass rates in Tables 4 and 6 are the percentage of events, where an event is defined as performance of one laboratory on one material in one survey, that meet the criteria for a particular scheme. CRMLN laboratories have consistently met all precision criteria. Before the B criteria were implemented, data collected between December 1995 and June 1997 were evaluated statistically and it was determined that the SD had no concentration dependence across the range. Based on these results, CRMLN decided that a single criterion for precision, as indicated by a SD of 0.03 mmol/L (1 mg/dL), without regard for HDL-C concentration is appropriate (B criteria).


View this table:
[in this window]
[in a new window]
 
Table 4. Percentage of pass rates for CRMLN precision goals.


View this table:
[in this window]
[in a new window]
 
Table 6. Percentage of pass rates for CRMLN accuracy goals.

Table 5 lists the QC results for each laboratory between July 1994 and April 1997. These data clearly demonstrate the long-term precision for CRMLN laboratories in performing the DCM. The SDs observed are well below the 0.03 mmol/L (1 mg/dL) criterion currently applied to CRMLN laboratories.


View this table:
[in this window]
[in a new window]
 
Table 5. Quality-control results for CRMLN laboratories participating in the evaluation of frozen vs fresh serum.

Accuracy vs the reference method for HDL-C.
The ability of CRMLN laboratories to meet the accuracy criteria are shown in terms of pass rates in Table 6Up . The CRMLN laboratories had little difficulty meeting the initial criterion of bias <=5%. Between July 1994 and August 1998, CRMLN laboratories also improved their accuracy (the pass rate for the unofficial goal of bias <=3% improved from 87.1% to 95.6%); however, the ability of the CRMLN laboratories to meet new criterion of bias <=0.03 mmol/L (1 mg/dL) has been difficult to document. CRMLN will continue striving to meet this criterion and expects that the new survey scheme (four samples analyzed in duplicate in four analytical runs) implemented in September 1998 will improve its ability to assess its members' accuracy.


   Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 
phase i: selection of a dcm for hdl-c
Although the bias of the DS method appeared to be slightly better than that of the HM method, neither the accuracy nor the precision observed for the two methods was significantly different. Practical issues—ease of use, availability of reagent components, and method robustness—became the determining factors in selecting a DCM for HDL-C. Preceding the analysis of cholesterol in the supernatants by the reference method for cholesterol, the reference method for HDL-C requires 5 h of technician time and a total time of 23 h to perform, the HM candidate DCM requires 30 min of technician time and total time of 1.5 h, and the DS candidate DCM requires 20 min of technician time and total time of 1 h. Availability of the major precipitation reagents was a serious consideration. No variability has been found in the properties of available heparin, which is produced for pharmaceutical rather than chemical use; however, its supply may be unreliable because the market is changing and fewer suppliers are available. Because of the potential for variability in the production of biological commercial heparin, the selection of a chemically stable precipitating reagent was considered an arguable advantage. In 1994, based on these considerations, the task force recommended the DS method as the DCM for HDL-C in the CRMLN.

phase ii: validation of ds method as the dcm
CRMLN decided that the small but statistically significant difference between the frozen and fresh samples analyzed by the DCM would not greatly impact the CRMLN's services. We also believed that the practical advantages of the DCM outweigh the disadvantage of this minor difference. The need for improving the accuracy of clinical methods was very great, and this small difference would not hinder CRMLN's goal to improve the accuracy of HDL-C measurement by providing a stable accuracy point for HDL-C.

phase iii: standardization of the dcm in the crmln
The results of the initial transfer challenge showed that the DS method could be transferred easily and that it met CRMLN's performance criteria. The standardization challenge showed that it could be performed consistently and that most of the CRMLN laboratories were also able to meet an informal bias goal of <=3%.

phase iv: performance of the dcm in the crmln
Currently, CRMLN provides manufacturers with evaluation rather than certification. In an evaluation, manufacturers can establish traceability to the reference method for HDL-C, and the CRMLN does not make pass or fail judgments about the product; CRMLN is providing a reference point for manufacturers. CRMLN must be able to document that its accuracy, as indicated by bias vs the reference method for HDL-C at CDC, is <=0.03 mmol/L (1 mg/dL) before it can certify that a manufacturer's performance meets NCEP performance goals. CRMLN believes that by offering a stable target for which manufacturers of clinical diagnostic products can aim improves accuracy in clinical laboratory measurement of HDL-C (17)(18)(19)(20)(21)(22)(23)(24).


   Conclusions
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 
CRMLN developed, evaluated, and standardized a DCM for HDL-C. The DCM is accurate, robust, and practical. CRMLN evaluated and applied the DCM only with samples containing <2.26 mmol/L (200 mg/dL) TGs. This approach is comparable to that used in the CDC-NHLBI LSP, where the TG concentration is similarly restricted in HDL-C reference materials. The DCM overcomes the shortcomings of the reference method for HDL-C in the types of fresh-sample method comparisons performed with manufacturers of clinical diagnostic products.

By making the DCM available to manufacturers and clinical laboratories for formal and informal method comparisons, CRMLN has provided a method to impact the accuracy of HDL-C measurements. The DCM has been used successfully in a program allowing manufacturers to establish traceability of their methods to the HDL-C accuracy base (17)(18)(19)(20)(21)(22)(23)(24). The DCM has several practical advantages over the use of the reference method for HDL-C: the cost of running the DCM is lower, and it requires less than one-half the volume of serum (it requires 2 mL of serum rather than 5 mL).

Standardization of HDL-C measurement is especially relevant because homogeneous methods that do not require a sample pretreatment step are being introduced. These methods present new calibration challenges because they use different principles to quantify HDL-C. Accuracy depends on providing calibration with set points traceable to the accuracy base. CRMLN has been able to offer a valuable service by standardizing these new products as well as those manufacturers that produce traditional products.

Manufacturers of clinical diagnostic products can contact any CRMLN member to obtain the protocol for collecting and analyzing samples. A list of all current members can be found on the World Wide Web at www.aacc.org/standards/cholesterolinfo.html. All manufacturers who follow the protocol to establish traceability to the accuracy base are issued a dated "Document of Comparison" that is valid for 2 years. After 2 years, manufacturers are encouraged to repeat the process to maintain traceability. Manufacturers who have a current Document of Comparison from CRMLN are listed on the World Wide Web site listed above.


   Acknowledgments
 
We thank S. Jay Smith (CDC) for performing the statistical power calculations and assisting in the statistical evaluations; G. Russell Warnick and Gerald R. Cooper for valuable input and advice; Stephen Ethridge (CDC), Charlene Griffin (CDC), Dave Gibson (WUSM), Connie Ferguson (WUSM), Barbara Decepida (WUSM), Janice Goetsch (PBRF), and Edward Balle (PBRF) for able technical assistance; and Merle Holstun (CDC) for valuable assistance in data analysis. We also thank all current and past members of the CRMLN who participated in the various parts of this study.


   Footnotes
 
1 Nonstandard abbreviations: HDL-C, HDL-cholesterol; CHD, coronary heart disease; TC, total cholesterol; TG, triglyceride; NCEP, National Cholesterol Education Program; CRMLN, Cholesterol Reference Method Laboratory Network; DCM, designated comparison method; NHLBI, National Heart, Lung, and Blood Institute; LSP, Lipid Standardization Program; HM, heparin-manganese; QC, quality control; PBRF, Pacific Biometrics Research Foundation; WUSM, Washington University School of Medicine; DS, dextran sulfate; PTA, phosphotungstic acid; and RV, reference value.


   References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
Conclusions
References
 

  1. National Cholesterol Education Program. Second report of the Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel II). NIH Publication No. 93-3095. Bethesda, MD: National Heart, Lung, and Blood Institute, 1993..
  2. National Cholesterol Education Program. The Expert Panel. Summary of the second report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel II). JAMA 1993;269:3015–23..
  3. Friedewald WT, Levy RI, Frederickson DS. Estimation of the concentration of low-density lipoprotein cholesterol in plasma, without use of the preparation ultracentrifuge. Clin Chem 1972;18:499-502. [Abstract]
  4. National Cholesterol Education Program. Working Group on Lipoprotein Measurement. Recommendations on lipoprotein measurement. NIH Publication No. 95-3044. Bethesda, MD: National Heart, Lung, and Blood Institute, 1995..
  5. Warnick GR, Wood PD. National Cholesterol Education Program recommendations for measurement of high-density lipoprotein cholesterol: executive summary. Clin Chem 1995;41:1427-1433. [Free Full Text]
  6. National Committee for Clinical Laboratory Standards. Method comparison and bias estimation using patient samples; approved guideline. NCCLS Document EP9-A. Villanova, PA: NCCLS, 1995..
  7. Myers GL, Cooper GR, Winn CL, Smith SJ. The Centers for Disease Control-National Heart, Lung, and Blood Institute Lipid Standardization Program. An approach to accurate and precise lipid measurements. Clin Lab Med 1989;9:105-135. [ISI][Medline] [Order article via Infotrieve]
  8. Abell LL, Levy BB, Brodie BB, Kendall FE. Simplified methods for the estimation of total cholesterol in serum and demonstration of specificity. J Biol Chem 1951;195:357-366.
  9. Cooper GR, Myers GL, Henderson LO. Establishment of reference methods for lipids, lipoproteins and apolipoproteins. Eur J Clin Chem Clin Biochem 1991;29:269-275. [ISI][Medline] [Order article via Infotrieve]
  10. Kuchmak M, Hazelhurst JS, Olansky AS, Taylor L. Reference sera with graded levels of high-density lipoprotein cholesterol. Clin Chim Acta 1984;144:237-243. [ISI][Medline] [Order article via Infotrieve]
  11. Kimberly MM, Waymack PP, Myers GL, Cole TG, Ferguson C, Gibson D, et al. Comparison of precipitation methods for determination of HDL cholesterol in the Cholesterol Reference Method Laboratory Network [Abstract]. Clin Chem 1994;40:1105.
  12. Warnick GR, Benderson J, Albers JJ. Dextran sulfate-Mg2+ precipitation procedure for quantitation of high-density-lipoprotein cholesterol. Clin Chem 1982;28:1379-1388. [Free Full Text]
  13. National Committee for Clinical Laboratory Standards. Evaluation of the linearity of quantitative analytical methods; proposed guideline. NCCLS Document EP6-P. Villanova, PA: NCCLS, 1986..
  14. Bennett ST, Eckfeldt JH, Belcher JD, Connelly DP. Certification of cholesterol measurements by the National Reference Method Laboratory Network with routine clinical specimens: effects of network laboratory bias and imprecision. Clin Chem 1992;38:651-657. [Abstract/Free Full Text]
  15. Koch DD, Peters T. Selection and evaluation of methods with an introduction to statistical techniques. Burtis CA Ashwood ER eds. Tietz fundamentals of clinical chemistry 4th ed. 1996:170-181 WB Saunders Philadelphia, PA. .
  16. Kimberly MM, Waymack PP, Smith SJ. Evaluation of frozen vs fresh serum samples using the designated comparison method for HDL cholesterol in the Cholesterol Reference Method Laboratory Network [Abstract]. Clin Chem 1995;41:S136.
  17. Modenese A, Baldo L, De Giorgi E, Carobene A, Ferrero C, Franzin M, Ceriotti F. Comparison among three precipitation agents for HDL cholesterol using Abell Kendall cholesterol method [Abstract]. Clin Chem 1995;41:S136.
  18. Tjersland G, Warnick GR, Cole TG. SPINPRO®-HDL qualifies for certification from the CDC Cholesterol Reference Method Laboratory Network [Abstract]. Clin Chem 1996;42:S289.
  19. Person N, Washington S. Evaluation of HDL cholesterol separation methods using NCEP goals [Abstract]. Clin Chem 1996;42:S290.
  20. Miki Y, Hatanaka S, Hanada T, Matsuura S, Mallory T. A homogeneous assay for the selective measurement of HDL-cholesterol in serum [Abstract]. Clin Chem 1997;43:S256.
  21. Nakamura M, Sato S, Iida M. Performance evaluation of homogeneous HDL-cholesterol assays in 48 Japanese clinical laboratories [Abstract]. Clin Chem 1997;43:S259-S260.
  22. Blutman J, Rose R, Fisher V, Laaker L, Gilbert W. Evaluation of a fully automated, homogeneous immuno-inhibition method for HDL-cholesterol [Abstract]. Clin Chem 1997;43:S265-S266.
  23. Cobbaert C, Zwang L, Ceriotti F, Modenese A, Cremer P, Herrmann W, et al. Reference standardization and triglyceride interference of a new homogeneous HDL-cholesterol assay compared with a former chemical precipitation assay. Clin Chem 1998;44:779-789. [Abstract/Free Full Text]
  24. Rifai N, Cole TG, Iannotti E, Law T, Macke M, Miller R, et al. Assessment of interlaboratory performance in external proficiency testing programs with a direct HDL-cholesterol assay. Clin Chem 1998;44:1452-1458. [Abstract/Free Full Text]



The following articles in journals at HighWire Press have cited this article:


Home page
J. Lipid Res.Home page
M. S. Yee, D. V. Pavitt, T. Tan, S. Venkatesan, I. F. Godsland, W. Richmond, and D. G. Johnston
Lipoprotein separation in a novel iodixanol density gradient, for composition, density, and phenotype analysis
J. Lipid Res., June 1, 2008; 49(6): 1364 - 1371.
[Abstract] [Full Text] [PDF]


Home page
Arch NeurolHome page
G. L. Vega, M. F. Weiner, A. M. Lipton, K. von Bergmann, D. Lutjohann, C. Moore, and D. Svetlik
Reduction in Levels of 24S-Hydroxycholesterol by Statin Treatment in Patients With Alzheimer Disease
Arch Neurol, April 1, 2003; 60(4): 510 - 515.
[Abstract] [Full Text] [PDF]


Home page
Int J EpidemiolHome page
S Houterman, W. Verschuren, C. Oomen, C. Boersma-Cobbaert, and D Kromhout
Trends in total and high density lipoprotein cholesterol and their determinants in The Netherlands between 1993 and 1997
Int. J. Epidemiol., October 1, 2001; 30(5): 1063 - 1070.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
G. R. Warnick, M. Nauck, and N. Rifai
Evolution of Methods for Measurement of HDL-Cholesterol: From Ultracentrifugation to Homogeneous Assays
Clin. Chem., September 1, 2001; 47(9): 1579 - 1596.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
M. M. Muller
Implementation of Reference Systems in Laboratory Medicine
Clin. Chem., December 1, 2000; 46(12): 1907 - 1909.
[Full Text] [PDF]


Home page
Clin. Chem.Home page
G. L. Myers, M. M. Kimberly, P. P. Waymack, S. J. Smith, G. R. Cooper, and E. J. Sampson
A Reference Method Laboratory Network for Cholesterol: A Model for Standardization and Improvement of Clinical Laboratory Measurements
Clin. Chem., November 1, 2000; 46(11): 1762 - 1772.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
C. M. Cobbaert and T. K. J. Luderer
Total Error Evaluation of Roche Direct HDL-Cholesterol Reagent and Calibrator across 31 Lot Combinations: A 2-Year Experience
Clin. Chem., January 1, 2000; 46(1): 133 - 134.
[Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit an electronic Letter to
the Editor about this paper
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (18)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kimberly, M. M.
Right arrow Articles by Waymack, P. P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kimberly, M. M.
Right arrow Articles by Waymack, P. P.
Related Collections
Right arrow Lipids, Lipoproteins, and Cardiovascular Risk Factors


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS