| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Dutch Foundation for Quality Assessment in Clinical Laboratories (SKZL), University Hospital Nijmegen, NL 6500 HB Nijmegen, The Netherlands.
2 Amphia Hospital, 4819 EV Breda, The Netherlands.
3 Lipid Reference Laboratory, University Hospital Rotterdam, 3000 CA Rotterdam, The Netherlands.
4 Queen Beatrix Hospital, 7100 GG Winterswijk, The Netherlands.
5 St. Anna Hospital, 5660 AB Geldrop, The Netherlands.
aAddress correspondence to this author at: University Medical Center Nijmegen, Department of Clinical Chemistry/116 SKZL, PO Box 9101, NL 6500 HB Nijmegen, The Netherlands. Fax 31-24-356-0686; e-mail hbaadenhuijsen{at}skzl.nl.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: The study consisted of the simultaneous analysis of fresh patient sera and potential reference materials (PRMs) for HDL-cholesterol (HDL-C) by 86 laboratories forming 43 laboratory couples. Six subgroups of method combinations were formed. The patient sera were selected and interchanged by each laboratory couple. The PRMs consisted of three types: C37, prepared according to the NCCLS C37 protocol; Fro, frozen selectively pooled human serum; and Lyo, which was the same serum pool as Fro but lyophilized in the presence of sucrose. All PRMs were provided in three HDL-C concentrations. The regression line residuals for the PRMs were normalized by expressing them as multiples of the state-of-the-art within laboratory SD (SDSA). In addition, the extra contribution of each PRM to the total measurement uncertainty, CVNetto, was calculated.
Results: Averaged over the three PRM concentrations, 1.6% of the C37 residuals were outside the 3 SDSA limit. For the Fro and Lyo PRMs, these values were 2.4% and 11.1%. CVNetto values for C37, Fro, and Lyo were 2.9%, 4.3%, and 5.3%, respectively.
Conclusions: The present twin-study design, as a practical alternative to the NCCLS EP14 protocol, is a viable way of studying commutability characteristics of PRMs. The study suggests that the C37 PRMs are the best candidates for a future reference material.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In view of these considerations, reference systems are needed to substantiate the claims of accurate results (1)(2). The introduction of the Directive for In Vitro Diagnostic Medical Devices (3) in the European Union has had a major impact on the further development of such systems.
In a reference system, analytical results should be traceable to the international system, SI. This traceability consists of an unbroken chain of comparisons, each with its stated uncertainties. Part of this chain is formed by the role of Certified Reference Materials (CRMs). Ideally, CRMs help to produce results that are commutable, e.g., numerically the same when different measurement procedures are applied, for all kinds of clinical conditions (2). The proposed ISO/CEN metrology standard (4) gives details for metrologic traceability and asks that the manufacturers of calibrators play a prominent role. We are convinced that a prominent role must also be played by the profession. The profession assesses, controls, and if possible, harmonizes the commercial systems between as well as within laboratories. For this activity, the profession should make use of reference materials that are commutable rather than system specific. This concept forms the basis for the Dutch project "Calibration 2000" (5)(6)(7). This project aims at harmonization of laboratory data via calibration by development of commutable, matrix-based secondary reference materials.
The NCCLS EP14 protocol (8) for evaluating possible matrix effects of processed samples or possible calibrators requires the simultaneous analysis of, preferably, 20 selected fresh patient sera together with the candidate calibrators to be studied by both a particular field method and, preferably, a reference method. The results obtained for the patient sera and the candidate calibrators with the comparative method are plotted on the x axis and the evaluated method on the y axis. The scatter of the results of the patient sera around the regression line, expressed as the prediction interval for the standardized residuals of the patient results, will be the measure for evaluating the characteristics of the preparations under investigation.
We considered implementation of the NCCLS EP14 protocol to be very demanding and costly, especially when several analytes and several field methods are involved. A practical alternative is presented, the so-called twin-study design, which in essence is a multicenter, split-patient-sample, between-field-methods protocol. The procedure is illustrated with the commutability assessment of potential reference materials (PRMs) in the analysis of HDL-cholesterol (HDL-C).
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Laboratories usually participating in the Dutch EQAS for general clinical chemistry were first asked about their interest in participating in the study. They were also asked for details of their measurement methodologies. Eighty-six laboratories were thus included. The study protocol consisted of the exchange of 12 fresh patient sera between each of two laboratories forming a laboratory couple; 43 laboratory couples were formed. Each laboratory was asked to select six fresh patient sera on the basis of various HDL-C concentrations, preferably spanning the relevant concentration interval for HDL-C. After these samples were split into two portions, one portion from each sample was transported the same day to the partner laboratory, which in turn proceeded in the same way for its patient specimens. The interchanged fresh patient samples were then analyzed (within 24 h of the initial analysis) in the same analytical run with the PRMs, which were sent beforehand to each participant on dry ice.
For the patient samples, only the results of the second day analysis were reported to the coordinating center. The laboratories acting as laboratory couples were selected on the basis of a modest geographic distance between each so that reanalysis could be carried out within 24 h and on the basis of differences in analytical techniques for the analysis of HDL-C.
analytical methods used
Of the study participants, 84% used one of the three direct HDL-C methods: 41% used a 
-cyclodextrin sulfate (-Cyclo) method (Roche); 41% used the N-Geneous method (Roche); and 2% used an immunoinhibition method (Wako). The remaining 16% used a precipitation method: 8% used a dextran sulfate–magnesium (PrDexMg) method; 6% used a phosphotungstic acid–magnesium (PrPTA) method; and 2% used a polyethylene glycol–dextran sulfate method. The selection procedure produced the following analytical combinations and numbers of laboratory couples: N-Geneous/-Cyclo (n = 15 couples); N-Geneous/N-Geneous (n = 8); -Cyclo/-Cyclo (n = 6); N-Geneous/PrDexMg (n = 4); -Cyclo/PrPTA (n = 4); and other combinations (n = 6).
PRMS
The PRMs, described in detail in the companion by Cobbaert et al. (9) in this issue of the Journal, were as follows: three frozen human serum pools (low, medium, high) prepared exactly according to the NCCLS C37 protocol (pools C37L, C37M, and C37H) (10); three pooled frozen human serum preparations originating from residuals of patient sera and selected on the basis of HDL-C concentration (FroL, FroM, and FroH); and three lyophilized human serum preparations (LyoL, LyoM, and LyoH). Selection, preparation, and lyophilization (for the Lyo PRMs) of the Fro and Lyo PRMs were carried out as described previously (11). Lyophilization took place in the presence of sucrose (200 g/L final concentration). All PRMs were stored centrally at -80 °C until dispatch on dry ice to the participants. Nominal HDL-C concentrations were as follows: 1.07, 1.25, and 1.83 mmol/L for C37L, C37M, and C37H, respectively; 0.93, 1.13, 1.55 mmol/L for FroL, FroM, and FroH, respectively; and 1.09, 1.70, and 1.89 mmol/L for LyoL, LyoM, and LyoH, respectively. All PRMs were analyzed for value assignment according to the procedure described by Cobbaert et al. (9).
statistical data analysis
The 43 sets of returned results were first screened for the presence of possible gross errors. In these cases, the respective results were excluded from further statistical evaluation.
Whereas the NCCLS EP14 protocol in most cases uses univariate linear regression analysis to study the behavior of patient and test samples, it was reasoned that in our case application of a bivariate distribution-free statistical approach was more appropriate because of the absence of an error-free reference method. Therefore, bivariate regression analysis according to Passing and Bablok (12)(13) was used throughout.
The regression residuals of the PRMs were expressed as the absolute values D of the perpendicular distances of each PRM to the respective patient regression line and were normalized by expressing them as multiples of the state-of-the-art within-laboratory SD (SDSA). Because of the design of the Dutch EQAS, this SDSA is one of the statistical outcomes for each of the analytes covered in this scheme and is targeted on a value of 0.04 mmol/L at 1.0 mmol/L HDL-C. A concentration-dependent correction of this SDSA was carried out by use of a square root approximation of the precision profile of the relevant within-laboratory variation (14). The decision limit for accepting a test material as commutable was set at 3 SDSA. The data set was also treated by ANOVA, in which the variation components that were attributable to the measurement of the PRMs were computed. The aggregated variance of the patient samples was computed by the formula:
 |
are the squared standard errors of regression in the regression analysis of the patient samples, summed over n laboratory couples.
The total variation for each respective PRM aggregated over all laboratory couples was calculated by:
 |
Finally, the extra contribution to the total uncertainty by each PRM, CVNetto, was calculated by:
 |
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
are examples of the results for HDL-C returned by some laboratory couples. These examples give an indication of some of the errors that can be expected in this kind of experiment with participating laboratories operating on a voluntary basis and under relatively unsupervised experimental field conditions. It was observed that, despite guidelines on sample handling, some participants diluted the frozen, ready-to-use samples with the amount of water to be used for reconstitution of the lyophilized samples. This is seen, for example, with the Fro samples assayed by laboratory 129 (Fig. 1
, middle panel in the right-hand column). Obvious aberrant cases were excluded from further data treatment. In all, the results of 42 laboratory couples were included. Visual inspection revealed the inferior behavior for some PRMs in various laboratory couples, as illustrated, for example, by laboratory couple 93/107 with the Lyo PRMs (Fig. 1
, center panel). Fig. 2
shows the distribution of the SDSA-normalized regression line residuals for each PRM in relation to the various method combinations. As seen in Fig. 2
, the Lyo PRM samples in general performed worse, primarily because of the outlying behavior of the analytical combination N-Geneous/PrDexMg. Averaged over the three concentrations and the 126 residuals concerned, 1.6% of the C37 residuals were outside the 3 SDSA limit. For the Fro and Lyo PRMs, these values were 2.4% and 11.1%.
|
|
The CVNetto values for each PRM are shown in Table 1
, both for the total data set and for the various analytical method combinations. In general, the CVNetto appeared to be most favorable for the C37 PRMs and to a lesser extent for the Fro PRMs, whereas the Lyo PRMs performed the worst, especially for the method combination N-Geneous/PrDexMg. A comparison of the information in Fig. 2
with that in Table 1
revealed the qualitative agreement of both approaches. Averaged over all method combinations and over all three PRM concentrations, mean CVNetto values for the C37, Fro, and Lyo PRMs were 2.9%, 4.3% and 5.3%, respectively; the C37 PRM thus was the most promising candidate for a future secondary reference material for the analysis of HDL-C.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Possible matrix effects in PRMs can be evaluated by comparison of the scatter of the results for these samples with the scatter of results for patient specimens around the regression line. The regression method applied in the NCCLS EP14 protocol uses the assumption that, because a reference method (x axis) is used, there is no error in the comparative method. Therefore, the residual scatter for patient results, taken in the y axis direction, is influenced by two factors: imprecision and nonspecificity of the method under evaluation. The use of replicate measurements can reduce the contribution of imprecision, and the remaining scatter points primarily to the influence of nonspecificity attributable to interference from known or unknown substances (matrix effect). In view of our primary aim to evaluate not only the characteristics of a PRM in combination with the various known methods for analyzing HDL-C, but also to study the possibility of using the same reference material for all other lipid and lipoprotein analytes (9), we realized that proper application of the NCCLS EP14 protocol would require a high investment in time and money. As organizers of an EQAS, we have intensive contact with its participants. Making use of the existing logistic environment, we thought that a concerted action, such as the twin-study design described here, might demonstrate the possibility for a practical alternative to the NCCLS EP14 protocol. Instead of performing replicate measurements in one analytical setting for the methods to be evaluated, we used the replicates formed by the aggregated results of the participating laboratories. It may be realized that evaluation of individual laboratory couple cases eventually will show larger result scattering compared with a situation in which each laboratory is instructed to report replicate results. This implies that both the imprecision and a potential matrix effect are being "seen" to the maximum degree, which we consider an additional advantage of the approach used in this study.
The absence of a reference method with presumed minimal error led us to use the bivariate regression analysis of Passing and Bablok (12)(13). In addition to being bivariate, this regression technique is rather insensitive to extreme outlying data points. In view of these considerations, we think that the present approach, which involves a sufficiently large population of participating laboratories with different analytical methods, is a viable way of getting information on the commutability characteristics of PRMs and, in that sense, may possibly be regarded as a practical addendum to the NCCLS EP14 protocol. We realize that our multicenter approach, because of its relatively unsupervised experimental conditions, inevitably introduces clerical and/or logistic errors for which the data set has to be screened and corrected before data analysis can be carried out.
The NCCLS EP14 protocol uses a 95% confidence interval around the patient regression line, depending on the inherent scatter of the patient results. In view of the multicenter approach used in this study, we thought it better not to use this individual measure and introduced a more general criterion, SDSA, for normalization of the residuals of the PRM results. In this way all included laboratory couple data sets were evaluated against a common standard. In cases with a relatively large patient scatter, this approach will lead to less liberal weighting of the particular PRM, as illustrated in Fig. 1
, in which one of the data points for the C37M PRM that exceeded the 3 SDSA limit was caused by the results for the laboratory couple 240/33, as depicted in Fig. 2
. In addition, the C37H data point for this laboratory couple was the highest data point in the population of normalized regression residuals for the method combination N-Geneous/-Cyclo in Fig. 1
.
In a previous study (11) in which different materials for use in an EQAS were evaluated for their effects on the accuracy of the total cholesterol assay, it was concluded that the detrimental effect of lyophilization on the serum matrix could be minimized by suitable cryoprotection with sucrose. In the case of HDL-C, used in the present study, we have to conclude that sucrose does not provide this protecting effect. Taken over all three concentrations of the Lyo PRMs, in most of the cases in which the normalized residuals exceeded the 3 SDSA limit, a precipitation method was involved. During recent years, we have seen a large increase in the use of direct assays for HDL-C at the expense of the precipitation methods, from 10% in 1996 to 85% at present. In light of the preceding discussion, this is a promising development with respect to further harmonization of HDL-C analysis results.
We had two reasons for introducing overall descriptive statistics. The first reason is that the expression CVNetto gives quantitative insight into the density distribution of the normalized residuals, which can only be deduced from Fig. 2
in a qualitative way. It is easy, for example, to grasp the overall worse performance of the Lyo PRMs, for which the mean CVNetto value was 5.3% compared with 2.9% for the C37 PRMs. However, it is much more difficult to visualize from Fig. 2
the difference between the performance of C37L and C37H, with CVNetto values of 3.6% and 1.9%, respectively. The second reason is that CVNetto allows extrapolation from the study of commutability characteristics to the situation in which a population of laboratories effectively uses a common calibrator. CVNetto may be interpreted as the extra contribution by the PRM to total measurement uncertainty. The basic assumption is that a perfect calibrator does not contribute to the intrinsic measurement uncertainty. In the ideal case, the CVNetto value should therefore be zero. Alternatively, a CVNetto value of, e.g., 2.5% implicates that the respective PRM introduces an extra measurement error of 2.5%. In the imaginary case of any other inherent errors being absent, this consequently may be translated to an expected value for the between-laboratory CV of 2.5%. If the state-of-the-art within-laboratory CV is 4%, as seen for HDL-C in The Netherlands, then an additional 2.5% between-laboratory variation component contributes to a total variation of 
= 4.7%.
We realize that the identification of a material as a potential candidate for a successful secondary reference material does not imply that the material is already fit to be used as such. Validation of the stability, value assignment with traceability to the relevant accuracy base accompanied by stated uncertainty levels, and the guarantee that future production lots have the same quality are a few of the prerequisites needed to meet the specifications of an accepted reference material. We think that we have taken the first step in this process by the characterization of candidate PRMs. In the meantime, we think that it is already possible to use the available C37 material in routine exchanges in the Dutch EQA system. The present twin-study approach has been used in the total setting of a commutability and harmonization study of PRMs by Cobbaert et al. (9) to be used for the standardization not only for HDL-C, but also across the other lipid and lipoprotein analyses in The Netherlands.
|
Footnotes |
|---|
-Cyclo, -cyclodextrin sulfate; PrDexMg, dextran sulfate-magnesium precipitation; PrPTA, phosphotungstic acid-magnesium precipitation; and SDSA, state-of-the-art within-laboratory SD. 
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  S. Branford, L. Fletcher, N. C. P. Cross, M. C. Muller, A. Hochhaus, D.-W. Kim, J. P. Radich, G. Saglio, F. Pane, S. Kamel-Reid, et al. Desirable performance characteristics for BCR-ABL measurement on an international reporting scale to allow consistent interpretation of individual patient response and comparison of response rates between clinical trials Blood, October 15, 2008; 112(8): 3330 - 3338. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Cobbaert, C. Weykamp, H. Baadenhuijsen, A. Kuypers, J. Lindemans, and R. Jansen Selection, Preparation, and Characterization of Commutable Frozen Human Serum Pools as Potential Secondary Reference Materials for Lipid and Apolipoprotein Measurements: Study within the Framework of the Dutch Project "Calibration 2000" Clin. Chem., September 1, 2002; 48(9): 1526 - 1538. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
, patients; 
, C37 PRM; 
, Fro PRM; •, Lyo PRM
, N-Geneous/
, N-Geneous/PrDexMg; 
, 
, other combinations.