|
|
||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Informatics and Statistics |
1 Roche Diagnostics GmbH, Department of Biostatistics, Penzberg, Germany; 2 Institute of Biopathological Medicine, Kanagawa, Japan; 3 Malmoe University Hospital, Malmoe, Sweden; 4 University of Missouri School of Medicine, Columbia, MO; 5 Queen Beatrix Hospital, Winterswijk, The Netherlands.
aAddress correspondence to this author at: Roche Diagnostics GmbH, Department of Biostatistics, Nonnenwald 2, 82377 Penzberg, Germany. Fax +49 8856 604152; e-mail andrea.geistanger{at}roche.com.
| Abstract |
|---|
|
|
|---|
Methods: We describe statistical methods to evaluate and monitor the relationships as expressed in master equations (MEs) between the IFCC Reference Measurement procedure (IFCC-RM) and designated comparison methods (DCMs) [US National Glycohemoglobin Standardization Program (NGSP), Japanese Diabetes Society/Japanese Society for Clinical Chemistry (JDS/JSCC), and Mono-S in Sweden]. We applied these statistics, including uncertainty calculations, to 12 studies in which networks of reference laboratories participated, operating the IFCC-RM and DCMs.
Results: For NGSP and Mono-S, slope, intercept, and derived percentage HbA1c at the therapeutic target show compliance with the respective MEs in all 12 studies. For JDS/JSCC, a slight deviation is seen in slope and derived percentage HbA1c in 2 of the 12 studies. Using the MEs, the uncertainty in an assigned value increases from 0.42 mmol/mol HbA1c (IFCC-RM) to 0.47 (NGSP), 0.49 (JDS/JSCC), and 0.51 (Mono-S).
Conclusions: We describe sound statistical methods for the investigation of relations between networks of reference laboratories. Application of these statistical methods to the relationship between the IFCC-RM and DCMs in the US, Japan, and Sweden shows that they are suitable for the purpose, and the results support the applicability of the ADA/EASD/IDF/IFCC Consensus Statement on HbA1c measurement.
| Introduction |
|---|
|
|
|---|
Given the increasing prevalence of diabetes, HbA1c is a key analyte in the medical laboratory, and the importance of result harmonization has been well recognized as reflected by national designated comparison methods (DCMs) in place in the US (3), Japan(4), and Sweden (5). DCMs are, however, based on arbitrarily chosen analytical methods with method-dependent results. There is a need to replace the national systems by one global, scientifically sound reference system or at least to link the respective national systems to the same analytical anchor.
The IFCC Working Group on Standardization of HbA1c developed a reference measurement procedure that has been approved by all IFCC member societies (6). The relationships between the IFCC Reference Measurement procedure (IFCC-RM) and DCMs are of utmost importance, especially now that there is a consensus signed by the IFCC, the European Association for the Study of Diabetes (EASD), the International Diabetes Federation (IDF), and the American Diabetes Association (ADA) stating that "the new IFCC reference system for [HbA1c] ... represents the only valid anchor to implement standardization of the measurement" and "... [HbA1c] results are to be reported worldwide in IFCC units (mmol/mol) and [italics added] derived NGSP [National Glycohemoglobin Standardization Program] units (%), using the IFCC-NGSP master equation [ME]" (7). These relationships have been investigated in 4 independent comparison studies that formed the basis for deriving MEs (8). Since the publication of these MEs, 8 additional comparison studies have been performed to assess the validity of the each published ME. This article reports the statistical tools developed to monitor the ME over the 6-year period and the outcome of the application of these tools.
| Materials and Methods |
|---|
|
|
|---|
the systems
The IFCC-RM as well as the DCMs in the US, Japan, and Sweden are executed by networks of laboratories. A short description of the respective systems is given below.
The IFCC Laboratory Network uses the reference measurement procedure developed by Kobold et al. (9) on behalf of the IFCC Working Group on Standardization of HbA1c. The method was published in 2002 as the IFCC approved recommended method (6). According to the approved IFCC recommendation, HbA1c as measured with the IFCC-RM is expressed in SI units (mmol/mol) (7). At the moment there are 13 IFCC network laboratories: 3 in Japan, 3 in the US, and 7 in Europe (10).
National calibrators are the basis of the harmonization scheme of the Japanese Diabetes Society and the Japanese Society for Clinical Chemistry (JDS/JSCC). The JSCC developed a high-resolution ion-exchange HPLC method, named KO500 (11), and a set of national calibrators (frozen below –70 °C) called JDS/JSCC Calibrator Lot 2. For the measurements in the intercomparison studies, the KO500 HPLC method was calibrated with JDS Calibrator Lot 2 and the measurements were performed by 3 network laboratories.
The NGSP network was established after the completion of the Diabetes Control and Complications Trial (DCCT). Implementation of treatment goals based on results of the DCCT in clinical settings made it necessary to harmonize HbA1c results among laboratories (12). A network of "secondary reference laboratories" was established to both assist manufacturers with calibration to the DCCT method and serve as comparison methods for NGSP certification (3). The measurements in the intercomparison studies were performed by 8 network laboratories in the US and Europe.
The Swedish scheme uses the Mono-S method (strong methylsulfonate cation exchanger on monobeds) as DCM for the harmonization of HbA1c results (5). The measurements in the intercomparison studies were performed in 1 to 2 laboratories in Sweden.
For the DCMs, HbA1c is expressed as the percentage of total hemoglobin (%).
statistical methods
The MEs between the IFCC standardization network and the national standardization networks should serve to recalculate HbA1c measurements made in IFCC units (mmol/mol) into the national HbA1c units. When this transformation is applied, however, the uncertainty of the recalculated value will increase, as the MEs also carry some uncertainty, i.e., the estimates of intercept and slope of the MEs carry uncertainty that is transferred to the recalculated values.
Denote a ME formula with: yME = b0ME + b1MEx, where x is the measured HbA1c value in mmol/mol, standardized to IFCC values and
x its uncertainty and yME the recalculated %HbA1c value in national units. Denote further with
b0 the uncertainty of the intercept of this ME,
b1 the uncertainty of the slope, and finally
b0b1 the correlation between these 2 values.
According to the formulas for uncertainty calculation (13), the uncertainty of the recalculated values is given by:
![]() |
There is a large amount of literature on the best regression method for method comparison in analytical chemistry (14)(15)(16). Because the values of the reference method in method comparison experiments are also subject to measurement error, it is often suggested that regression procedures be used that take these circumstances into account to avoid a biased estimation of intercept and slope. One common regression procedure in this context is Demings regression (14). However, the better the linear relationship between the measured values, the smaller is the bias introduced by the estimation via simple linear regression. For example the CLSI (formerly NCCLS) guideline EP9-A2 Method Comparison and Bias Estimation Using Patient Samples (17) recommends the use of simple linear regression estimation if the correlation coefficient is >0.975. For our data, the correlation coefficient in all studies was >0.99, such that the bias introduced by using simple linear regression is negligible.
To examine whether a certain relationship is still valid, most regression procedures concentrate on the estimation of slope and intercept and the derivation of CIs for these estimates. But the estimates of slope and intercept are correlated, such that only a 2-dimensional acceptance region correctly addresses this issue. Further, it is difficult to incorporate the uncertainty in the reference relationship in these approaches.
Therefore the comparison of the ME and the new regression line does not focus on the estimates of slope and intercept; instead, we constructed simultaneous confidence bands of the difference in prediction over the measurement range of interest, according to the ideas of Liu et al. (18). The range of interest extends from 0 to 150 mmol/mol HbA1c.
Given a value x within the measurement range of interest, based on the ME this value is transformed to:
![]() |
![]() |
![]() |
A visualization of the comparison is given by plotting the calculated differences as well as the confidence bands centered at zero against the x values in the measurement range. If the calculated difference exceeds the centered confidence bands at any point within the measurement range, the ME and the regression line are significantly different.
Equivalence between 2 regression lines also can be shown by the simultaneous CIs approach. Liu et al. (18) discuss this issue in detail in their second example. The main idea is that one specifies, in advance, an acceptable distance between the predictions over the whole measurement range. If the confidence band stays within this predefined distance, the 2 regression lines are equivalent. The width of the CIs is influenced by the number of sample values and their distribution over the measurement range used for the estimation of the regression lines.
In Results, we show the graphical outcome of these statistics as applied to 12 intercomparison studies from 2001 to 2006.
| Results |
|---|
|
|
|---|
|
Based on these MEs, the IFCC HbA1c results in mmol/mol can be transformed into the results of the respective DCMs. The uncertainty of the transformed values includes the uncertainty of the measured IFCC value as well as the uncertainties of the respective ME. Table 2
shows the standard uncertainties in assigned HbA1c values with the IFCC-RM as well as in the derived DCM values when the ME is used for conversion for 3 HbA1c concentrations.
|
For example, assuming a 0.8% measurement CV for a HbA1c value measured within the IFCC reference system, a sample of 53 mmol/mol HbA1c has a standard uncertainty of 0.42 mmol/mol HbA1c. If this value is transformed into a NGSP %HbA1c value based on the respective ME, the uncertainty increases by 0.05 mmol/mol HbA1c. Table 2
also shows the 95% CIs of assigned values in units of the respective systems. The transformation based on the MEs adds only a small amount of uncertainty to the derived values.
stability of the mes
The results of the comparison of the 12 regression lines with the respective MEs are given in the next sections. Within the studies of 2001-1 and 2001-2, we analyzed 8 samples; for the other studies, we analyzed 5 samples distributed over the measurement range of interest. The number of laboratories within the NGSP network was 8 for all studies; the JDS/JSCC network consisted of 2 laboratories in the 2001-1 study and 3 laboratories in the other studies. The Swedish standardization scheme is performed in 1 laboratory, except in the 2001-2 study (2 laboratories).
The number of laboratories within the IFCC standardization network differs slightly from study to study between 9 and 14 laboratories (9 laboratories in studies 2002-1, 2002-2, 2003-1, and 2003-2; 10 laboratories in studies 2001-2, 2004-1, and 2004-2; 12 laboratories in study 2001-1; 13 laboratories in studies 2005-1 and 2006-2; and 14 laboratories in studies 2005-2 and 2006-1).
Within each laboratory each sample was measured 4 times.
For this comparison, we set
= 0.05 and adjusted this confidence level within each network comparison to the number of comparisons (n = 12). The measurement range of interest for the comparison was set from 0 to 150 mmol/mol.
ifcc-rm and ngsp (dcm us)
Fig. 1
shows slope, intercept, and calculated HbA1c percentages at the 53 mmol/mol concentration for the 12 individual intercomparison studies. The 95% CI of the ME, as published by Hoelzel et al. (8), is represented by the shaded area. One can clearly see the interdependence of slope and intercept: an intercept shifted to the left corresponds to a shift to the right of the slope. One can also see that the widths of the CIs vary from study to study; this is due to variation in the sample dispersion. The CIs for slopes, intercepts, and HbA1c percentages of all 12 studies overlap with the gray zone of the ME, which implies that none of the results obtained in the studies is significantly different from the ME.
|
Fig. 1
shows the derived NGSP HbA1c percentage at a single IFCC HbA1c concentration, i.e., 53 mmol/mol (the therapy target). However, agreement at 1 HbA1c concentration does not automatically mean that there is agreement over the whole HbA1c concentration range. Fig. 2
shows that in all 12 studies, the straight line (difference in derived HbA1c percentage) is within the 2 curved lines (the CI of nonsignificant difference), confirming that the relationship found in each of the 12 individual studies is in agreement with the ME at any HbA1c concentration.
|
At the edges of the measurement range, the maximum half-width of the CIs ranges between 0.27 (2003-2 study) and 0.60 (2006-2 study) NGSP %HbA1c. The half-width of the CIs at the decision limit of 53 mmol/mol HbA1c is between 0.07 and 0.17 NGSP %HbA1c.
ifcc-rm and jds/jscc (dcm japan)
As with the NGSP-derived results, we investigated the results for the JDS/JSCC system. Supplemental Fig. 1 in the online Data Supplement shows that the 95% CIs for all intercepts overlap with the ME CI. The same is true for the slopes, except in the 2006-1 and 2004-1 studies. Again, the interdependency of slope and intercept is clear. Looking at the JDS/JSCC-derived HbA1c percentage, results from the 2006-1 study are outside the 95% CI of the ME, and those from 2004-2 and 2003-2 are borderline. Supplemental Fig. 2 in the online Data Supplement shows a significant deviation of the derived HbA1c percentage at higher HbA1c concentrations in 2 studies (2004-1 and 2006-1), with a similar but not significant tendency in studies 2003-2, 2004-2, and 2005-1.
With respect to the width of the CIs, the maximum half-width at the extremes of the measurement ranges lies between 0.32 and 0.74 JDS/JSCC %HbA1c, whereas at the decision limit of 53 mmol/mol HbA1c, it is between 0.08 and 0.23 JDS/JSCC %HbA1c.
ifcc-rm and mono-s (dcm sweden)
Supplemental Fig. 1 shows that the CIs for all experimental slopes and intercepts overlap with the CI of the ME. In 1 study (2002-2), the derived Mono-S HbA1c percentage is outside the CI of the ME. This is confirmed with regard to the overall HbA1c range by Supplemental Fig. 3 in the online Data Supplement, in which one can also see that the widest dispersion occurred in the 2001 and 2002 studies.
The maximum half-width of the CIs at the edges of the measurement range is between 0.38 and 0.85 Sweden %HbA1c. At 53 mmol/mol HbA1c (medical decision limit), the half-width lies between 0.11 and 0.24 Sweden %HbA1c.
| Discussion |
|---|
|
|
|---|
The aim of this work was to develop statistics to judge the stability of the ME by comparing the regression line of each new intercomparison study with the published ME. As both ME and new regression lines are only estimates of the true relationship, uncertainties of both had to be included in the relationship calculations. The comparison of the ME and the new regression lines was not based on the simple comparison of slopes and intercepts, as these estimates are highly correlated, which implies that only a 2-dimensional acceptance region correctly addresses the issue. Further, it is difficult to incorporate the uncertainty in the comparison relationship of these 2 values. To address this issue, we constructed simultaneous confidence bands of the difference in prediction of the calculated HbA1c over the range of interest. These statistics represent a suitable and powerful instrument to evaluate the consistency of relations between methods.
The recent consensus (statement #3) states that " ... [HbA1c] results are to be reported in IFCC units (mmol/mol) and derived NGSP units (%), using the IFCC-NGSP master equation" (7). This implies that the IFCC-RM is the analytical reference to which HbA1c values should be directly traceable and that NGSP values are derived in an additional step using the respective ME. This additional step adds further uncertainty, which was quantified in this study. At the therapy target concentration of 53 mmol/mol HbA1c, the uncertainty in a value assigned by the IFCC network is 0.42 mmol/mol. When NGSP values are derived using the ME, the uncertainty increases by 0.05 mmol/mol to 0.47 mmol/mol, demonstrating that the uncertainty in DCM calculated values is higher than in the IFCC-RM assigned values, although this increase is relatively small.
In conclusion, the small uncertainty in assigned values by the IFCC-RM, the consistency of the MEs to convert IFCC-RM values to DCM values, and the relatively small additional uncertainty due to this conversion demonstrate the practical applicability of the IFCC-RM as the analytical anchor and the MEs as tools for conversion and make practically feasible the global implementation of the recommendations included in the ADA/EASD/IDF/IFCC Consensus Statement (7).
| Acknowledgments |
|---|
Financial Disclosures: None declared.
Acknowledgments: We thank the members of the IFCC Scientific Division for reviewing the manuscript before submission. We also thank the following persons of the network laboratories: Masao Umemoto (Reference Material Institute for Clinical Chemistry Standards, Kanagawa, Japan), Izumi Takei (School of Medicine Keio University, Tokyo, Japan), Maria Ospina (CDC, Atlanta, GA), Yuanfang Deng (Siemens Medical Solution Diagnostics, Norwood, MA), Veronica Luzzi (Washington University, St. Louis, MO), Jean Bucksa (Fairview University, Minneapolis, MN), Uwe Kobold (Roche Diagnostics, Penzberg, Germany), Renata Paleari and Donatella Caruso (Universita degli Studi di Milano, Milano, Italy), Erna Lenters and Janine Slootstra (Isala Klinieken, Zwolle, The Netherlands), Willeke Dekkers and Jeffrey Sigger (Queen Beatrix Hospital, Winterswijk, The Netherlands), Franciscus Susanto (Deutsches Diabetes Zentrum, Duesseldorf, Germany), Patricia Kaiser (INSTAND, Duesseldorf, Germany).
| Footnotes |
|---|
| References |
|---|
|
|
|---|
The following articles in journals at HighWire Press have cited this article:
![]() |
S.-C. Shiesh, H.-M. Wiedmeyer, J.-T. Kao, S. D. Vasikaran, J. B. Lopez, and the Laboratory Management Committee for the Asian- Proficiency Testing of Hb A1c: A 4-Year Experience in Taiwan and the Asian Pacific Region Clin. Chem., October 1, 2009; 55(10): 1876 - 1880. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |