|
|
||||||||
Special Reports |
1 Queen Beatrix Hospital, Winterswijk, The Netherlands (IFCC-network coordinator); 2 Norfolk and Norwich University Hospital, and School of Medicine, Health Policy and Practice, UEA, Norwich, UK; 3 Centro Interdipartimentale per la Riferibilità Metrologica in Medicina di Laboratorio (CIRME), Università degli Studi di Milano, Milano, Italy; 4 Institute of Biopathological Medicine, Kanagawa, Japan (JDS/JSCC network coordinator); 5 University of Missouri School of Medicine, Columbia, MO (NGSP network coordinator); 6 Malmoe University Hospital, Malmoe, Sweden (Coordinator Reference System Sweden); 7 Austin Pathology, Austin Health, Heidelberg, Australia; 8 Isala Klinieken, Zwolle, the Netherlands; 9 Center for Disease Control and Prevention, Atlanta, GA; 10 INSTAND e.V., Duesseldorf, Germany; 11 Brigham and Womens Hospital and Harvard Medical School, Boston, MA.
aAddress correspondence to this author at: Queen Beatrix Hospital, Beatrixpark 1, 7101 BN Winterswijk, the Netherlands. Fax +31 543 524265; e-mail c.w.weykamp{at}skbwinterswijk.nl.
| Abstract |
|---|
|
|
|---|
Methods: Each study included: unknown samples (to test individual network laboratories); known samples (controls); recently manufactured calibrators (to check calculated assigned value); stored calibrators (to test stability) and a calibration-set (to calibrate the IFCC-RM). The unknown samples are measured by use of the IFCC-RM and the designated comparison methods [DCMs; the National Glycohemoglobin Standardization Program (NGSP) in the US, Japanese Diabetes Society/Japanese Society for Clinical Chemistry (JDS/JSCC) in Japan, and Mono-S in Sweden] are used to investigate the stability of the Master Equation (ME), the relationship between IFCC-RM and DCMs.
Results: A total of 105 IFCC-RM data sets were evaluated: 95 were approved, 5 were not, and for 5 no data were submitted. Trend analysis of the MEs, expressed as change in percentage HbA1c per year, revealed 0.000% (NGSP, not significant), –0.030%, (JDS/JSCC; significant) and –0.016% (Mono-S; not significant). Evaluation of long-term performance revealed no systematic change over time; 2 laboratories showed significant bias, 1 poor reproducibility. The mean HbA1c determined by laboratories performing mass spectrometry (MS) was the same as the mean determined by laboratories using capillary electrophoresis (CE), but the reproducibility at laboratories using CE was better. One batch of new calibrators was not approved. All stored calibrators were stable.
Conclusion: A sound reference system is in place to ensure continuity and stability of the analytical anchor for HbA1c.
| Introduction |
|---|
|
|
|---|
| Materials and Methods |
|---|
|
|
|---|
|
Materials and Methods and Units
The IFCC Network Laboratories used the IFCC-RM (12)(13). DCMs used the national reference methods: 3 JDS/JSCC network laboratories in Japan (16), 8 NGSP network laboratories in the US (17), and 1 Mono-S reference laboratory in Sweden (5). According to the Consensus Statement (15), HbA1c is expressed as mmol HbA1c/mol Hb for the IFCC-RM and as percentage HbA1c (%HbA1c) for the DCMs.
statistics and network rules
The performances of laboratories in each intercomparison study were evaluated according to statistical rules for networks of reference laboratories (18). In essence these rules are derived from a robust approach with all of the intercomparison samples examined together, allowing proportional and constant bias to be calculated. Mathematically, the differences between the results of an individual laboratory and the overall median are plotted against the overall median of the network laboratories, and the slope (proportional bias) and intercept (constant bias) are calculated from the linear relation. Slope and intercept together determine a criterion that is expressed as a graph (Fig. 1
). The criterion for approval is fixed and set empirically: a laboratory is approved when the joint CI of constant and proportional bias is within the 95% CIs as derived from the first 6 intercomparison studies.
|
The relations between IFCC-RM and DCMs were evaluated using a method based on linear regression analysis, using the formula: y = ax + b, in which x is the concentration HbA1c of the IFCC-RM and y = the %HbA1c of the respective DCM. The uncertainty in the relation is determined by the uncertainty in the ME, and the uncertainties in the outcome of the IFCC-network and DCM networks in the individual studies. This protocol implies that the CI is variable from study to study. In addition, the CI varies with the HbA1c concentration. To examine the trend in the relationship between IFCC-RM and DCMs, a Shewhart chart with variable limits can be constructed (unpublished data). This report includes a chart for the defined target for diabetic control (53 mmol/mol).
Assigned values and the expanded uncertainty of specimens are calculated according to (18).
To evaluate long-term performance of individual network laboratories, we included data from the last 8 intercomparison studies (4 laboratories were not approved at the time of the first 4 studies).
The criterion to approve a new batch of calibrators, and stability of calibrators of stored batches is the maximum allowable difference between calculated and measured HbA1c, defined as the combined uncertainty in the calculated and measured HbA1c concentrations at the 95% confidence level (19). The criterion to approve stability of controls is the maximum allowable difference between assigned value and HbA1c measured after 1 and 2 years of storage, defined as the uncertainty in the assigned value at the 95% confidence level (19).
For purpose of the evaluation of the network, specific statistics have been developed (18, 19, and unpublished data). The statistics are implemented in R, a software environment for statistical computing and graphics, and an Excel add-in to import the results submitted by the network laboratories, as well as an R add-on package that contains functions specially defined for the analysis of the network.
It is the policy of the IFCC Working Group to have 8–15 approved network laboratories distributed in a number of countries. Network laboratories participate twice a year in an intercomparison study to demonstrate their competence. A laboratory wishing to be a reference laboratory first becomes a candidate reference laboratory and has to meet the defined analytical criteria in 2 consecutive intercomparison studies to gain the status of approved network laboratory. An approved network laboratory loses the status of approved network laboratory when it fails (or does not submit results) in 2 consecutive studies. A meeting is organized once a year for the network laboratories to exchange knowledge and discuss problems. The educational part of the network statistics helps laboratories that fail these criteria to identify the source of their problems.
| Results |
|---|
|
|
|---|
reproducibility and trend in the relation between ifcc-rm and dcms
The relation between IFCC-RM and DCMs is expressed by the equation y = ax + b, in which y is the HbA1c value of a DCM, x is the HbA1c value measured by the IFCC-RM, a is the slope, and b is the intercept. The median r value in the 12 studies was 0.9990 (NGSP), 0.9984 (JDS/JSCC), and 0.9985 (Mono-S). The "Equations and individual studies" section of Table 2
shows slope, intercept, and %HbA1c (calculated at 53 mmol/mol, the defined target for diabetic control) for the 12 intercomparison studies for the respective DCMs.
|
The "Master Equations " section (Table 2
) shows ME4 and ME12. ME4 is the mean relation between IFCC-RM and DCMs based on results of the first 4 intercomparison studies (14). ME12 is calculated from all 12 intercomparison studies.
The "Trend Analysis " section deals with the trend in the IFCC-RM DCM relation and is expressed as %HbA1c/year over the period 2001–2006.
The Shewhart chart in Fig. 2
allows inspection of whether the relation of IFCC-RM and DCMs in each of the individual intercomparison studies is significantly different from the published ME. Observations within the limits (NGSP) imply compliance of those studies with the ME. Observations just outside or close to the limits (JDS studies 2003–2, 2005–2, 2006–1, 2006–2 and Mono-S 2002–2) suggest non- and borderline compliance, respectively.
|
value assignment and expanded uncertainty
In 8 intercomparison studies, values have been assigned to 40 samples with HbA1c concentrations from 32–121 mmol/mol. At the lower HbA1c concentrations (30–40 mmol/mol HbA1c), the mean (range) expanded uncertainty (k = 2) was 0.6 (0.4–0.8) mmol/mol; in the middle HbA1c concentrations (50–70 mmol/mol) it was 1.0 (0.8–1.2) mmol/mol; and at higher concentrations (80–120 mmol/mol) it was 1.5 (1.0–2.0 mmol/mol).
long-term trend, bias, and reproducibility results for network laboratories
Table 3
shows the results of long-term evaluation of the network laboratories. Trend is expressed as the HbA1c change (mmol/mol) per year of an individual laboratory. Bias is expressed as the differences between the mean HbA1c value of a laboratory and that of the network mean in 8 intercomparison studies. Reproducibility is expressed as the SD of differences between a laboratory and the network-mean in the 8 intercomparison studies. Statistical significance is evaluated with linear regression analysis (trend), t-test (bias), and F-test (reproducibility). An evaluation of 8 intercomparison studies conducted between 2003 and 2006 (at an HbA1c concentration of 53 mmol/mol), revealed that the trend over time (Table 3
, column 2) ranges from +0.5 to –0.4 mmol/mol HbA1c /year. Regression analysis shows that there is no significant trend for any of the network laboratories. Bias (Table 3
, column 3) of network laboratory 2 (+0.8 mmol/mol) is statistically significantly higher than the network-mean, whereas the bias of network laboratory 4 (–0.9 mmol/mol) has a significantly lower outcome. Reproducibility (Table 3
, column 4) ranges from 0.1–1.4 mmol/mol. Seven network laboratories have a very low variation (SD <0.4 mmol/mol) in their difference from the network-mean. Laboratory 3 has a statistically significant higher variation (SD 1.4 mmol/mol).
|
reproducibility and stability calibrators and controls
The upper part of Table 1
shows essential data describing the reproducibility test of new batches of calibrators and summarizes the stability test of old calibrators. From 2000–2006, 10 batches of 6-concentration sets of calibrators were manufactured. To check the assigned value, new calibrators are measured as samples, and outcome is compared with the HbA1c calculated from the weighed amounts pure HbA1c and pure HbA0. The mean difference between measured and weighed HbA1c ranges from –0.4 to +0.5 mmol/mol in the respective levels. The maximum allowable difference was exceeded only once (3.6 mmol/mol observed vs 1.2 mmol/mol allowed).
To test stability of the calibrators, a limited number of the moderate concentrations (B to E) of stored calibrators were assayed as samples 2–3 years after their manufacture. No significant difference between measured and weighed HbA1c was observed in any of these "old calibrators".
In the lower part of Table 1
the outcomes for 12 batches of control samples, assayed 1 and 2 years after manufacture, are summarized. Control specimens are spare samples of previous intercomparison studies. In each intercomparison study, 4 controls are included: 2 (1 low and 1 high HbA1c level) from an intercomparison study performed the previous year, and 2 from a study conducted 2 years earlier. On 1 occasion, for the low concentrations after 2 years of storage, the difference was 1.0 mmol/mol and exceeded the maximum allowable difference of 0.7 mmol/mol.
| Discussion |
|---|
|
|
|---|
The criteria for approval of network laboratories are empirically based rather than determined on the basis of predefined performance goals (20). One reason for this protocol is the lack of consensus on approval criteria: in a recent review Goodall (21) refers to 7 published statements, with a proposed CV of 2% to 5% for routine methods. The other reason is that the development of the RM started with the qualitative aim to be "as precise as possible." With 12 intercomparison studies completed, we now can quantify the performance of the network. According to criteria for approval of reference laboratories, the maximum CV of assigned values is <0.9% (assigned by the network) or <3% (assigned by an individual network laboratory). In light of the most stringent performance goal of 2% for routine laboratories (21), the uncertainty of value assignment by one single reference laboratory (in general) is too high to be acceptable, but suitable when performed by the whole network. However these are maximum CVs. The CV of 0.5% seen over 6 years in relation to the NGSP (Table 2
) suggests that the actual CV of the network is substantially lower than 0.9%. From the performance data of 12 intercomparison studies, it can be concluded that the IFCC-RM is suitable for the intended purpose in the top of the traceability chain of HbA1c. The data also suggest that, to limit uncertainty, it is preferable for values to be assigned by the network rather than by individual network laboratories.
stability of the relationship between ifcc-rm and dcms
The stability of the relationship between IFCC-RM and DCMs is of the utmost importance for clinical studies. In 2004 the relationship was calculated on the basis of the 4 completed intercomparison studies and published as the ME (14). To date, the outcome of 12 intercomparison studies is known and allows evaluation of compliance with the published ME for each of the intercomparison studies and trend over time in the relationship of IFCC-RM and DCMs. As demonstrated by the r values, the relationship between IFCC-RM and DCMs is very consistent. Slope and intercept are not independently related, as demonstrated by the CVs (CV %HbA1c <CV slope and intercept). Therefore the % HbA1c is the best parameter to evaluate the stability of the MEs. For the NGSP this relationship is very stable: at the 53 mmol/mol level, the calculated %HbA1c is 7.00% whether ME4 or ME12 is used. There is also no trend (<0.001%/year), and all 12 studies are in compliance with the published ME. The same is true for the Mono-S relationship with the IFCC-RM. For the JDS/JSCC a borderline trend (Fig. 2
) is seen that is also reflected by a difference of 0.05% between ME4 and ME12, a finding that will be investigated in future intercomparison studies.
value assignment and expanded uncertainty
The assigned values and uncertainty of specimens derives from (i) the uncertainty in the calibrator sets used to calibrate the IFCC-RM, (ii) the uncertainty due to the measurement error of the reference method, (iii) the number of network laboratories involved in the value assignment, and (iv) the number of assays performed by each network laboratory. When values are assigned with the whole network, an expanded uncertainty of 1.0 mmol/mol (0.9% CV) can be achieved in the middle range of HbA1c concentrations, which is acceptable in view of the performance goals for routine laboratories as discussed above.
long-term trend, bias, and reproducibility results for network laboratories
A single intercomparison study is the forum for approval of network laboratories at a given point of time. Evaluation of multiple intercomparison studies discloses small phenomena and trends over time. From Table 3
it can be seen that none of the laboratories has shown a trend over time, that 2 laboratories have a consistent low or high bias, and that 1 laboratory has high variation, indicating lack of reproducibility. From the bottom lines of Table 3
, it can be seen that there is no difference in HbA1c outcome between laboratories that use MS vs CE methods (+0.1 mmol/mol vs 0.0 mmol/mol) but that the MS-group has a significantly higher variation (0.7 mmol/mol vs 0.4 mmol/mol). The difference in performance between MS and CE might be explained by nonoptimal HPLC circumstances for the MS method, and a modification leading to improvement (22) is under investigation, to be implemented in the IFCC-RM.
reproducibility and stability of calibrators and controls
Long-term reproducibility is the cornerstone of the management of the network. Calibrators and controls play key-roles, and their reproducibility and stability are systematically monitored.
After manufacture, a batch of calibrators is measured as a sample by the whole network and the mean measured concentration HbA1c is compared with the HbA1c concentration calculated from the weighed amounts of pure HbA1c and HbA0. If the difference exceeds the criterion, the calibrator set is rejected. This situation occurred once in our studies. When a batch of calibrators meets the criterion it is approved and used as the calibrator set in the next intercomparison study (1 year after manufacture). Two or 3 years after manufacture, spares are included in an intercomparison study as "old calibrators" to check their long-term stability. Throughout the 12 studies we report, calibrators always proved to be stable.
Spare samples of intercomparison studies are stored at –84 °C and after 1 and 2 years of storage are systematically included as controls in intercomparison studies. This protocol allows comparison 0, 1, and 2 years after manufacture and is a parameter for stability. As can be seen from Table 1
, changes in the 12 batches of controls have been negligible. The maximum allowable difference was exceeded in only 1 of 48 studies (1.0 mmol/mol vs 0.7 mmol/mol). The data indicate that this type of material can be (and is) used as long-term quality control. A possible objection to this conclusion is that the trend in laboratories may have been compensated by the instability in the controls, but this theory does not hold given that 12 batches showed the same performance.
In conclusion, the results of the 12 intercomparison studies performed during 2001–2006 reported here confirm the robustness of this system to guarantee stability and continuity of the analytical reference method for HbA1c. The results also demonstrate that the concept of a network of reference laboratories as the foundation to develop, implement, and maintain a reference system is very effective and efficient.
| Acknowledgments |
|---|
Financial Disclosures: None declared.
Acknowledgments: We thank the members of the IFCC Scientific Division for reviewing the manuscript before submission. We also thank the following persons of the network laboratories: Masao Umemoto (Institute of Biopathological Medicine, Kanagawa, Japan), Izumi Takei (School of Medicine Keio University, Tokyo, Japan), Maria Ospina (CDC, Atlanta, USA), Yuanfang Deng (Siemens Medical Solution Diagnostics, Norwood, USA), Veronica Luzzi (Washington University, St. Louis, USA), Jean Bucksa (Fairview University, Minneapolis, USA), Uwe Kobold (Roche Diagnostics, Penzberg, Germany), Renata Paleari and Donatella Caruso (Universita degli Studi di Milano, Milano, Italy), Erna Lenters and Janine Slootstra (Isala Klinieken, Zwolle, The Netherlands), Willeke Dekkers and Jeffrey Sigger (Queen Beatrix Hospital, Winterswijk, The Netherlands), Franciscus Susanto (Deutsches Diabetes Zentrum, Duesseldorf, Germany), Patricia Kaiser (INSTAND, Duesseldorf, Germany).
| Footnotes |
|---|
| References |
|---|
|
|
|---|
The following articles in journals at HighWire Press have cited this article:
![]() |
H. Garcia-Alcala, A. Ruiz-Arguelles, and B. Cedillo-Carvallo Effect of the Method to Measure Levels of Glycated Hemoglobin on Individual Clinical Decisions: Comparison of an Immunoassay With High-Performance Liquid Chromatography Am J Clin Pathol, September 1, 2009; 132(3): 332 - 335. [Abstract] [Full Text] [PDF] |
||||
![]() |
The International Expert Committee International Expert Committee Report on the Role of the A1C Assay in the Diagnosis of Diabetes Diabetes Care, July 1, 2009; 32(7): 1327 - 1334. [Full Text] [PDF] |
||||
![]() |
R. R. Little and C. L. Rohlfing HbA1c Standardization: Background, Progress and Current Issues Lab Med, June 1, 2009; 40(6): 368 - 373. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Geistanger, S. Arends, C. Berding, T. Hoshino, J.-O. Jeppsson, R. Little, C. Siebelder, C. Weykamp, and on behalf of the IFCC Working Group on Standardiza Statistical Methods for Monitoring the Relationship between the IFCC Reference Measurement Procedure for Hemoglobin A1c and the Designated Comparison Methods in the United States, Japan, and Sweden Clin. Chem., August 1, 2008; 54(8): 1379 - 1385. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |