| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Editorials |
Memorial Sloan-Kettering Cancer Center, New York, NY
aAddress correspondence to this author at: Memorial Sloan-Kettering Cancer Center, 1275 York Avenue Box 213, New York, NY 10021, Fax 1-646-422-2379, E-mail liljah{at}mskcc.org
When prostate-specific antigen (PSA) testing started to become widespread in the late 1980s, discrepancies between values yielded by different assays became apparent. Elucidation of the various molecular forms of PSA in the blood and other bodily fluids in the early 1990s contributed critical information as to how to avoid these assay-related biases (1). Another important advance was Stamey and coworkers’ design of a novel PSA calibrator (2), which later was endorsed by the WHO. In the last issue of Clinical Chemistry, Jansen and colleagues reported their use of the WHO-endorsed calibrator to examination the recalibration of one widely used PSA assay (3). Jansen et al. (3) report that calibration affects both the likelihood that a man will undergo a biopsy procedure and the likelihood that the biopsy will reveal cancer. This report is a timely reminder that the PSA concentration, as given on a laboratory report, is not a simple statement of a true biological state but is affected by subtle details of laboratory techniques. A man whose PSA increases from 1.9 to 2.4 µg/L over the course of a year would have cause for concern because this sudden increase in PSA exceeds a widely discussed "PSA velocity" cutpoint of 0.35 µg/L per year (4) and thus may be indicative of cancer. As Jansen and coworkers demonstrated, however, such an increase may also be attributable to a change in the assay used to measure PSA.
The main results of the Jansen study focused on a different and more widely used cutpoint as an indication for biopsy, a PSA concentration of 3 or 4 µg/L. The use of this cutpoint reflects contemporary urologic practice, and the authors correctly state that their analyses have important practical implications. We suggest an additional possibility, to abandon the cutpoint of 3 or 4 µg/L altogether. This suggestion reflects our view that cutpoints in clinical chemistry are problematic. We believe that the need exists for a new way of thinking about the relationship between biomarker measures and clinical decision-making.
The suggestion to abandon cutpoints is not trivial—cutpoints could not be more central to routine clinical chemistry. Almost all of the many millions of clinical laboratory reports produced yearly include both the absolute concentration of each analyte and an indication of whether this concentration is above or below a cutpoint. A PSA above 4 µg/L is flagged, bolded, or marked in red, as is, for example, a fasting blood glucose above 5.8 mmol/L or hemoglobin below 130 g/L. We see numerous problems with the uncritical use of such cutpoints.
For Many Cutpoints the Rationale Is Unknown
Much as we would like to think that medicine is based solely on clear evidence, the origin of many cutpoints is unclear. The cutpoint of 4 µg/L for PSA is one example. In the past 2 years, we have conducted numerous enquiries as to the origin of this cutpoint, contacting clinical chemists, urologists who were in practice when the PSA test was first introduced, and industry personnel. Most of the individuals we contacted had no idea of where 4 µg/L came from; others referred us to early landmark studies (5)(6) or reports that we could not find. The studies we did find, however, did not provide empirical validation of this cutpoint, because they used 4 µg/L as the criterion for biopsy. The best source appears to be an internal industry report selecting 4 µg/L as the basis for a reference interval.
Cutpoints Are Often Chosen with Irrational Methods
Reference intervals are the most common source of clinical chemistry cutpoints: a blood marker is measured in a group of individuals who report no disease; the range of values that includes 95% of the population is defined as "normal," and the values in the top or bottom 2.5% are defined as "abnormal." The essential problem with reference intervals is that they are a statistical construct, entirely disconnected from any consideration of health or illness. For a start, a reference interval defines a fixed proportion of the population (e.g., top or bottom 2.5%) as abnormal, irrespective of the incidence of disease. The use of a reference interval for a laboratory screening test, e.g., sarcoma in children, would indicate that about 2 million children in the US per year are worthy of additional work-up, when fewer than 1000 cases are diagnosed each year. Moreover, reference intervals are naturally highly dependent on the population studied. As a simple thought experiment, consider a 95% reference interval for body mass index based on the contemporary US population, compared to the US population in 1975 or an African population.
We propose that a viable alternative to reference intervals must consider the relationship(s) between the laboratory value and a clinical outcome. This has been done with body mass index, which was correlated with longevity (thus explaining why obesity cutpoints are not increasing over time). Given a relationship between a laboratory measure and a clinical outcome, however, we still need a statistical method to obtain a cutpoint. The most common approach is to choose a cutpoint that maximizes sensitivity plus specificity, a process often performed by choosing the point closest to the top left of the ROC curve (7). This approach weights sensitivity and specificity equally and is thus, in our view, not optimal. Consider, for example, a test for infection for which the disease is often fatal and the treatment is a highly effective, safe, and well-tolerated antibiotic. A cutpoint associated with 70% sensitivity and 70% specificity would be closer to the top left of the ROC curve than a cutpoint with 100% sensitivity and 30% specificity, but we would choose the latter cutpoint. We do so because the clinical consequences of a missed diagnosis are grave compared to the minimal risk of harm to an individual patient who receives an unnecessary antibiotic treatment. Other clinical settings may involve an entirely different balance between disease severity vs treatment efficacy and side effects, leading to different choices for the importance of sensitivity compared to specificity.
Cutpoints Are Invariant to Patient Preference
Imagine that a man tells us that he is very anxious about invasive medical procedures and tolerates them poorly. Accordingly, he is willing to undergo prostate biopsy only when the doctor is quite convinced that he may need it. One might imagine that in such a case a doctor might forego biopsy if the man’s PSA was only slightly higher than the commonly used cutpoint of 4 µg/L (e.g., 4.5 µg/L) but be more insistent if the PSA showed a larger increase (such as 17 µg/L). A similar approach might be used with respect to increased cholesterol in a patient who experiences side effects from a statin: whether the patient is advised to continue medication despite the side effects will depend on the degree of the cholesterol increase.
The problem is that a doctor has no rational way to reset a cutpoint that takes patient preference into consideration. A PSA cutpoint of 4 µg/L might be too low for the anxious patient, but who is to say whether 6, 10, or 15 µg/L should be the new cutpoint? Conversely, this cutpoint may be far too high for very risk-adverse younger men. Similarly, if 240 mg/dL (6.2 mmol/L) cholesterol is an insufficiently high threshold for a patient with low statin tolerance, how much higher would be "high enough" for a doctor to insist on treatment?
Cutpoints Cannot Include Multiple Pieces of Information
Medical decisions are rarely based on a single clinical finding, symptom, or laboratory measure. With respect to our statin-intolerant patient, the physician’s advice is likely to take into account other risk factors such as blood pressure, smoking, diabetes, and family history. In the case of prostate cancer, some urologists would consider ordering both a total PSA concentration and a free-to-total PSA ratio (8), which tends to be lower in men with cancer. Thus the question arises of how the doctor should integrate these 2 measures. Is biopsy indicated if PSA is increased but the free-to-total PSA ratio is normal? If not, would there be some very high concentration of PSA that would override the normal free-to-total PSA ratio?
Use of Risk Prediction in Place of Cutpoints
A simple, rational alternative to the traditional use of clinical chemistry cutpoints is to use laboratory measures to calculate the probability of clinically relevant states or events. For example, instead of the laboratory report stating whether total PSA and free-to-total PSA ratio are in the reference interval, a probability of prostate cancer would be given on the basis of a statistical model including age, total PSA, and free-to-total PSA ratio. Similarly, instead of marking cholesterol as out of range, the laboratory report could incorporate other laboratory values, such as hemoglobin A1C, and clinical information, such as blood pressure, to give an estimate of a patient’s risk of a cardiovascular event within 10 years.
The advantages of such an approach mirror each of the problems described above. First, creation of a statistical risk prediction model is a complex scientific procedure that would require careful documentation in the peer-reviewed literature. Such documentation will likely include reference to the sort of calibration issues reported by Jansen et al. (3). Second, and importantly, the choice of any cutpoint would be rationally based on clinical consequences. To return to two of our previous examples, patients at risk of the fatal infection would likely be willing to take an antibiotic even if they had only a 1% risk of disease; in contrast, many men are likely to require a 20% or higher risk of prostate cancer to be willing to undergo an uncomfortable procedure such as a prostate biopsy. Third, use of risk prediction can easily incorporate patient preference, in part because information is presented in intuitive terms (compare "your cholesterol is 240 mg/dL " to "you have a 1 in 5 chance of a heart attack in the next 10 years"). Fourth, risk prediction can easily incorporate multiple items of information.
In our view, the use of clinical chemistry cutpoints is a holdover from a time when bedside calculation of probabilities was impractical for the practicing clinician. Defining a patient’s laboratory results as normal or abnormal and treating accordingly used to be easier than calculating a probability on the basis of a statistical prediction model. With the widespread availability of information technology, this is no longer true: one can envisage computer systems that would seamlessly integrate multiple laboratory reports, imaging, and clinical data to provide the treating physicians and the patient with clinically relevant and properly validated risk predictions. However, such systems would require both clinical research—to develop and validate by independent replication appropriate risk prediction models—and new platforms for bioinformatics; and thus their availability likely remains several years in the future. In the meantime, we must not forget the very real limitations of our current approaches: cutpoints certainly make life simpler, but rarely reflect complex biological systems adequately. In addition, as pointed out by Janssen et al. (3), changes in calibration can importantly influence widely accepted cutpoints.
Acknowledgments
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors’ Disclosures of Potential Conflicts of Interest: Upon manuscript submission, all authors completed the Disclosures of Potential Conflict of Interest form. Potential conflicts of interest:
Employment or Leadership: None declared.
Consultant or Advisory Role: None declared.
Stock Ownership: None declared.
Honoraria: None declared.
Research Funding: H. Lilja, the National Cancer Institute.
Expert Testimony: None declared.
Other: H. Lilja holds patents for free PSA and hK2 assays.
Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, preparation or approval of manuscript.
References
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  P. A. Kavsak, X. Wang, D. T. Ko, A. R. MacRae, and A. S. Jaffe Short- and Long-Term Risk Stratification Using a Next-Generation, High-Sensitivity Research Cardiac Troponin I (hs-cTnI) Assay in an Emergency Department Chest Pain Population Clin. Chem., October 1, 2009; 55(10): 1809 - 1815. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
