| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letters |
1 Laboratorium voor Analytische Chemie Universiteit Gent Harelbekestraat 72 9000 Gent, Belgium
aAuthor for correspondence. Fax 32-9-264-81-98; e-mail Linda.thienpont{at}rug.ac.be.
To the Editor:
Current guidelines for the combined graphical/statistical interpretation of method-comparison studies (1) include a scatter plot combined with correlation and regression analysis (2) and/or a difference plot combined with calculation of the 2s limits of the differences between the methods (the so-called 95% limits of agreement) (3)(4). The former approach has a long tradition in clinical chemistry, and its advantages and pitfalls are well known (5). The latter approach, however, which was deemed "simple both to do and to interpret" and was propagated as a substitute for regression analysis (4)(5), became available only in recent years and has increased in popularity. The general features of the Bland–Altman plot have been well described (4) (see also Fig. 1A
). The x axis shows the mean of the results of the two methods ([A + B]/2), whereas the y axis represents the absolute difference between the two methods ([B - A]). When the standard deviation increases with concentration, Bland and Altman recommend a logarithmic y scale, whereas others propose a percent y scale (6). Although generally there is not much difference in effect between using percentages and using a log transformation of the data, we prefer the percent plot (except when data extend over several orders of magnitude) because numbers can be read directly from the plot without the need for back-transformation. Additionally, the plot includes the line for the mean difference and the experimentally observed 2s limits of the differences between the methods. Often forgotten, the Bland–Altman approach consists of a comparison of the 2s limits with a clinically acceptable difference between the two methods.
|
We reviewed difference plots published in this journal and discuss here the key aspects associated with their use. We screened all articles in this journal, starting from the first issue of 1995 up to May 2001. We observed increasing use of the Bland–Altman plot over the years, from 8% in 1995 to 14% in 1996, and 31–36% in more recent years. In addition to the Bland–Altman method, method comparisons were performed using correlation and regression analysis and the concordance plot. In total, we found 96 uses of difference plots [listed in the Data Supplement that accompanies the online version of this letter at Clinical Chemistry Online (clinchem.org/content/vol48/issue5)]. Most authors also used correlation and regression analysis, suggesting that difference plots are viewed as complementary to, rather than substitutes for, regression analysis. Among 96 references (in total, 98 plots) with Bland–Altman plots, 75 used the absolute difference plot, 20 applied a percent y-scale version, and 3 a logarithmic version of the plot. In total, 50 presented the results in an additional scatter plot.
The following general problems were observed. In 13 cases, the x axis was constructed using only the values of the comparison method (see Data Supplement, Addendum 2, for listing). By doing so, however, the plot may falsely show a concentration-dependent difference even when there is none (7). The 2s limits were presented in only 67 cases, and most importantly, only 2 authors compared the 2s limits with a clinically acceptable difference between the two methods. The 2s limits were more generally used in absolute (59) and logarithmic (3) difference plots, but rarely in percent (5) difference plots.
A similar search was performed in two other laboratory medicine journals for the period 1996–2001. We found in Clinical Chemistry and Laboratory Medicine and Annals of Clinical Biochemistry, respectively, 29 and 43 difference plots (17 and 34 absolute, 10 and 7 percent, and 2 and 2 logarithmic difference plots). We found that the characteristics of the plots in Clinical Chemistry and Laboratory Medicine were similar to those reported for this journal (see Data Supplement, Addenda 3a, 3b, and 3c). However, in Annals of Clinical Biochemistry, additional scatter plots were very seldom presented. This apparently results from the fact that the "Instructions for Authors" deprecate the use of regression analysis, which traditionally is accompanied by a scatter plot.
Bland and Altman (4) show method comparisons that cover a small concentration range and data sets without proportional differences between the methods. In this situation, a constant standard deviation may be assumed, and parallel 2s limits and a mean bias are justified (Fig. 1A
). However, this case is rather unusual in clinical chemistry. In the 75 examined references with absolute difference plots (showing 103 figures), we found, by eye, 57 data sets with a standard deviation increasing with concentration and/or with a proportional difference (see Fig. 1B
). In these cases, Bland and Altman recommend the use of a log transformation of the data points. Neither a mean bias (in Fig. 1B
suggested by the horizontal line at 0.6 mmol/L) nor constant and parallel 2s limits are justified. Rather, the 2s limits should be "V-shaped" around the regression line of the differences (8)(9) (see Fig. 1C
). Alternatively, to use parallel 2s limits, a percent difference plot can be used (Fig. 1D
). Overall, we found that 87% of plots had technical flaws, similar to data reported by Mantha et al. (10), who made an analogous survey in the field of anesthesia. Most striking, in both surveys, interpretation of the data by comparison of the actually observed limits of agreement with a priori ones was missing in >90% of the cases.
In summary, difference plots are useful for the presentation and interpretation of method-comparison studies, but most authors in this journal use them as supplements to regression analysis and the scatter plot, a practice that is also recommended by the NCCLS (1). Unfortunately, many authors uncritically apply the classical absolute difference plot in method-comparison studies that cover a wider concentration range, where they would better use a percent (or log) difference plot. Last but not least, the main objective of the Bland–Altman approach, namely, comparison of the experimentally observed deviations with a preset clinical acceptance limit, is seldom followed despite recommendations for doing so that were given earlier in this journal (11).
To emphasize, the key aspects of the appropriate construction and use of the Bland–Altman plot are the following. The x axis should be constructed by the mean of the methods and the y axis in a way that is most sensible to the concentration range of the x data (absolute: small range; percentage: medium range; log-scale: large range). The 95% limits of agreement should reflect the actually observed nature of the differences (whether or not there is a relationship between difference and magnitude) (9). Most important, interpretation of the data should be done by comparison of the observed limits of agreement with a priori ones.
As a final note, we want to remark that in this journal, already in 1981, a similar plot (with the y axis constructed as a ratio) was proposed for the evaluation of method-comparison data (12). Strange to say, this report has been overlooked.
Acknowledgments
This work was supported by the Research Fund of the University Ghent (Grant BOF 011109000).
References
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  N. V. Dawson, M. E. Singer, L. Lenert, M. B. Patterson, S. A. Sami, I. Gonsenhouser, H. A. Lindstrom, K. A. Smyth, M. J. Barber, and P. J. Whitehouse Health State Valuation in Mild to Moderate Cognitive Impairment: Feasibility of Computer-Based, Direct Patient Utility Assessment Med Decis Making, April 1, 2008; 28(2): 220 - 232. [Abstract] [PDF]
|
|
|
|
 |
|
 N. Janzen, M. Peter, S. Sander, U. Steuerwald, M. Terhardt, U. Holtkamp, and J. Sander Newborn Screening for Congenital Adrenal Hyperplasia: Additional Steroid Profile using Liquid Chromatography-Tandem Mass Spectrometry J. Clin. Endocrinol. Metab., July 1, 2007; 92(7): 2581 - 2589. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Boodhan, A. M. Maloney, and L. L. Dupuis Extent of Agreement in Gentamicin Concentration Between Serum That Is Drawn Peripherally and From Central Venous Catheters Pediatrics, December 1, 2006; 118(6): e1650 - e1656. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C.-C. Lin, C.-F. Chang, H.-J. Chiou, Y.-C. Sun, S.-S. Chiang, M.-W. Lin, P.-C. Lee, and W.-C. Yang Variable Pump Flow-Based Doppler Ultrasound Method: A Novel Approach to the Measurement of Access Flow in Hemodialysis Patients J. Am. Soc. Nephrol., January 1, 2005; 16(1): 229 - 236. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Stockl, D. Rodriguez Cabaleiro, K. Van Uytfanghe, and L. M. Thienpont Interpreting Method Comparison Studies by Use of the Bland-Altman Plot: Reflecting the Importance of Sample Size by Incorporating Confidence Limits and Predefined Error Limits in the Graphic Clin. Chem., November 1, 2004; 50(11): 2216 - 2218. [Full Text] [PDF]
|
|
|
|
|
|
C. Wang, D. H. Catlin, L. M. Demers, B. Starcevic, and R. S. Swerdloff Measurement of Total Serum Testosterone in Adult Men: Comparison of Current Laboratory Methods Versus Liquid Chromatography-Tandem Mass Spectrometry J. Clin. Endocrinol. Metab., February 1, 2004; 89(2): 534 - 543. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. T. Olesberg, M. A. Arnold, and M. J. Flanigan Online Measurement of Urea Concentration in Spent Dialysate during Hemodialysis Clin. Chem., January 1, 2004; 50(1): 175 - 181. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. B. Ledue and N. Rifai Preanalytic and Analytic Sources of Variations in C-reactive Protein Measurement: Implications for Cardiovascular Disease Risk Assessment Clin. Chem., August 1, 2003; 49(8): 1258 - 1271. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Tan, X. Sun, L. Tang, N. Zhang, Q. Han, M. Xu, X. Tan, X. Tan, and R. M. Hoffman Automated Enzymatic Assay for Homocysteine Clin. Chem., June 1, 2003; 49(6): 1029 - 1030. [Full Text] [PDF]
|
|
|
|
|
|
M. Panteghini and F. Pagani On the Comparison of Serum and Plasma Samples in Troponin Assays Clin. Chem., May 1, 2003; 49(5): 835 - 836. [Full Text]
|
|
|
|
|
|
R. M. Dorizzi, M. Caputo, A. Ferrari, L. Lippa, and P. Rizzotti Comparison of Serum and Heparin-Plasma Samples in Different Generations of Dimension Troponin I Assay Clin. Chem., December 1, 2002; 48(12): 2294 - 2296. [Full Text] [PDF]
|
|
Read all eLetters
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
