| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letters to the Editor |
1 St. Paul’s Hospital, Department of Laboratory Medicine, and the Department of Pathology, University of British Columbia, Vancouver, BC, Canada
2 Department of Mathematics, University of British Columbia, Vancouver, BC, Canada
aAddress correspondence to this author at: St. Paul’s Hospital Department of Laboratory Medicine and the University of British Columbia Department of Pathology. 1081 Burrard Street, Vancouver, BC, Canada, V6Z 1Y6. Fax 604-806h8815; e-mail dtholmes{at}interchange.ubc.ca.
To the Editor:
Apple et al. (1) recently presented an analysis of the effect of imprecision (CV) on false positive cardiac troponin I (cTnI) results. After examining their methodology, we have identified problems with the model and arguments.
Key mathematical details are omitted, and some assumptions appear unrealistic. The authors model the plasma cTnI concentrations in a healthy reference population by an exponential distribution and the analytical error as gaussian using 2 assumed CV profiles: a baseline profile and the same scaled by a factor of 0.40. We observe 2 difficulties.
First, the baseline profile is incompletely specified, and the points provided (CVs of 37.5%, 25.0%, and 9.4% at true cTnIs of 0.05, 0.07, and 0.14 µg/L) imply SDs that decrease with increasing cTnI, which does not reflect reality (2)(3). Further, the "SD of the blank" (cTnI = 0 µg/L) is entirely unspecified. To contrast, we constructed empirical profiles using mean concentration data and CVs from a recent paper comparing 15 cTnI assays (4). Fig. 1a
compares the 5 cTnI assays whose data overlap the concentration range 0.05–0.14 µg/L to those used by Apple et al. (1). For the experimentally derived profiles, the SD function has both a positive slope and a "y-intercept", features assumed in theoretical models (5). Unfortunately the authors’ profile biases the results in favor of their conclusions.
|
Second, simulation with gaussian measurement error gives negative concentrations. If infrequent, such values might be dropped or taken as zero, but our attempts to duplicate the authors’ analysis give many negative values. Specifically, noting that the 3 supplied profile points are perfectly collinear when plotted as SD vs mean (in itself unrealistic), and assuming this collinearity holds everywhere, the resulting CV profile gives 
160 000 (32%) negative measured values in a run of 500 000 simulated values (Fig. 1b
). Fig. 1
of the technical brief by Apple et al. does not lend clarity as it apparently shows true, not measured, values. How did the authors handle negative simulated values? Did they discard them, possibly biasing their assumed reference population toward higher cTnI concentrations?
Yet, if we agree that a suitable modification of their model might lead to results such as those presented, albeit not numerically identical, we fail to see how they lead to the conclusions drawn. Specifically, the authors first intend to show that, if 2 assays are identical except that one CV profile is 0.4 times that of the other, for 3 sequential measurements on 1000 healthy individuals, the more precise assay will result in 3–5 fewer patients exceeding the 99th percentile in at least one measurement. On this basis, the authors suggest that the analysis is relevant to clinical practice, as when patients present with chest pain. Second, they conclude that only the 99th percentile cutoff (and not the concentration at CV = 10%) should be used.
With respect to the first point, data collected at our institution indicate that emergency room patients have a very different cTnI distribution than healthy individuals, with a larger fraction of patients near the 99th percentile cutoff. Attempts to assess the error rates of serial measurements by use of a model based on healthy people will therefore have little bearing on real clinical practice. To appropriately compare more and less precise assays, with or without serial measurements, the model must simulate cTnI distributions appropriate to an emergency room setting; additionally, it must consider both false positives and false negatives.
With respect to the second point, we feel the results simply do not address it. For the 2 hypothetical assays, the CV = 10% concentrations of 0.07 µg/L and 0.14 µg/L exceed the respective 99th percentile cutoffs of 0.063 µg/L and 0.07 µg/L, but no results are presented for the CV = 10% cutoffs. How can conclusions be drawn about the relative merits of 2 alternate cutoffs when only 1 is considered?
To use the 99th percentile irrespective of assay imprecision is an attractive suggestion because this would be simpler and less arbitrary than a 10% CV cutoff (which corresponds to a different cTnI concentration on every platform). Although we applaud the authors for attempting to evaluate this problem in a systematic manner, this conclusion, however attractive, is not supported by the evidence provided; and the issue is certainly not closed, as was asserted with reference to the results of this technical brief (6). Mathematical models are only as good as the assumptions that underpin them. If the assumptions are incorrect or not explicitly stated, then the results will be misleading and difficult to asses.
References
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
exp(–
88.6 L/µg, and x is the concentration of cTnI in µg/L. The 2 measured value PDFs were generated using the convolution product (
(x)=–0.0625x+0.021875 ("Baseline"), or the same profile scaled by exactly 0.4 ("Scaled"— i.e., the precision profiles described by Apple et al.).