| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letters to the Editor |
1 STT Consulting, Horebeke, Belgium
2 Laboratorium voor Analytische Chemie, Faculteit Farmaceutische Wetenschappen, Universiteit Gent, Gent, Belgium
aAddress correspondence to this author at: Laboratorium voor Analytische Chemie, Faculteit Farmaceutische Wetenschappen, Universiteit Gent, Harelbekestraat 72, B-9000 Gent, Belgium. Fax 32-9-264-81-98; e-mail linda.thienpont{at}ugent.be.
To the Editor:
The difference, or Bland–Altman plot (1)(2)(3)(4) has become a popular tool for the presentation of method-comparison studies (5)(6), but the plot has rarely been used for making decisions about the quality of a method (5)(6). Bland and Altman expressed this in the terms "we want to know by how much the new method is likely to differ from the old; if this is not enough to cause problems in clinical interpretation we can replace the old method by the new or use the two interchangeably. How far apart measurements can be without causing difficulties will be a question of judgment. Ideally, it should be defined in advance to help in the interpretation of the method comparison and to choose the sample size" (2). The tool for doing so was to investigate whether the upper (UCL) or lower (LCL) 95% confidence limit of 1.96 SD of the differences between the methods (UCL1.96 SD,diff, LCL1.96 SD,diff) was equal to or smaller than a predefined limit for total error (TE; acceptance, UCL1.96 SD,diff or LCL1.96 SD,diff 
TE). The reason that this strategy is seldom applied may be that Bland and Altman presented neither the acceptance limits nor the confidence intervals (CIs) in the graphic (1)(2)(3)(4), whereas in fact, they discussed the "precision of the estimates" (CId, CI1.96 SD,diff, where d is the mean difference) and their relationship with the sample size in detail (2). Moreover, based on UCL1.96 SD,diff or LCL1.96 SD,diff and a predefined medical limit, Bland and Altman made the decision that "the degree of agreement between the two [Wright] meters was not acceptable" (2). As an expansion of the concept, we propose to additionally investigate the question of whether the 95% confidence limits of d between the methods is equal to or smaller than a predefined limit for systematic error (SE; acceptance, UCLd or LCLd SE). Note that CId = ± t(95%, n – 1) x (SD,diff/
n) and CI1.96 SD,diff = ± 1.71x t(95%, n – 1) x (SD,diff/n). Note also that we use the one-sided t values for the calculation of the CI because the question is whether the UCL or LCL is equal to or less than a predefined error limit.
Because of this unrecognized but important purpose of the difference plot, we present here a "model Bland–Altman plot" to be used for the interpretation of a method-comparison study vs predefined error limits (Fig. 1
). The plot shows ±SE, ±TE, CId, and CI1.96 SD,diff in addition to the commonly presented lines representing d and d ± 1.96 SD,diff. At the same time, we emphasize the importance of the sample size (n) on the "acceptance decision" by simulations with sample sizes of 80 (Fig. 1A
), 40 (Fig. 1B
), and 20 (Fig. 1C
). We chose as example a simulated method comparison for serum cholesterol between an accuracy-based reference method (applying isotope-dilution gas chromatography/mass spectrometry) and a routine method. However, the simulation truly mimics a "real world" comparison that has been done before (7). The y axis represents the differences of the routine method from the reference method, expressed in percentage of the values of the reference method. This approach is recommended for data that span a "medium" range, where a more or less constant CV can be expected (8). Moreover, the 1.96 SD,diff deviations can directly be related to the CV of the routine method. The bias of the routine method was assumed to be 2.3% and the CV to be 3%. We used as acceptance limits for the routine method SE = 3% (9) and TE = 10%(10). For an overview about strategies for setting quality specifications, the reader is referred to a recent conference report (11).
|
Visual interpretation of Fig. 1A
(n = 80) easily allows one to conclude that the routine method satisfies the limits for SE as well as TE (UCLd SE and UCL1.96 SD,diff TE). From Fig. 1B
(n = 40), on the other hand, the conclusion would be that the routine method does not satisfy the SE limit (UCLd > SE), but does satisfy the TE limit (UCL1.96 SD,diff TE). From Fig. 1C (n = 20), one would conclude that the routine method satisfies neither the SE nor the TE limit (UCLd > SE and UCL1.96 SD,diff > TE).
In summary, the example demonstrates that the incorporation of confidence limits and predefined error limits in a Bland–Altman plot allows easy visual interpretation of a method-comparison study. Moreover, the confidence limits directly show the importance of the sample size for decisions about method acceptance, a fact that is usually not considered.
Finally, we want to remark that the confidence intervals (and indeed, limits of agreement) are by convention set at 95% but that other values might be used. Most obviously, one might in some situations require 99% limits of agreement to meet a predefined specification.
References
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  J. Daniel, H. Ziaee, P. B. Pynsent, and D. J. W. McMinn The validity of serum levels as a surrogate measure of systemic exposure to metal ions in hip replacement J Bone Joint Surg Br, June 1, 2007; 89-B(6): 736 - 741. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
