| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Cancer Diagnostics |
1 Cancer Research UK Epidemiology Unit, University of Oxford, Radcliffe Infirmary, Oxford OX2 6HE, United Kingdom. 2 Diagnostics Division, Bayer HealthCare, Bayer House, Strawberry Hill, Newbury, Berks, and Department of Clinical Biochemistry, University of Oxford, Oxford, United Kingdom. 3 19 Kerwin Drive, Dore, Sheffield, United Kingdom.
aAuthor for correspondence. Fax 44-1865-310545; e-mail andrew.roddam{at}cancer.org.uk.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: A simulation model, calibrated to the distribution of PSA values in the United Kingdom, was developed to estimate the effects of bias, nonequimolarity, and analytical imprecision in terms of the rates of men who are recommended to have a biopsy on the basis of their assay-reported PSA values when their true PSA values are below the threshold (false positives) or vice versa (false negatives).
Results: False recommendation rates for a calibrated equimolar assay are 0.5–0.9% for analytical imprecision between 5% and 10%. Positive bias leads to significant increases in false positives and significant decreases in false negatives, whereas negative bias has the opposite effect. False-positive rates for nonequimolar assays increase from 0.5% to 13% in the worst-case scenario, whereas false-negative rates are almost always 0%.
Conclusions: Biased and nonequimolar assays can have major detrimental effects on both false-negative and false-positive rates for recommending biopsy. PSA assays should therefore be calibrated to the International Standards and be unbiased and equimolar in response to minimize the likelihood of incorrect clinical decisions, which are potentially detrimental for both patient and healthcare provider.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Assays for the measurement of PSA used by diagnostic laboratories, the majority of which are now commercially available assays, have considerable differences in their epitope recognition, which can lead to significantly different tPSA values for the same serum sample (6)(7). Therefore, samples containing effectively the same amount of tPSA but with different proportions of fPSA can produce very different values for both tPSA and fPSA. These differential response assays are usually referred to as skewed-response or nonequimolar assays. This has led to the development of International Standards for the purposes of assay calibration for both fPSA and tPSA (8).
The clinical relevance of using an equimolar assay for tPSA remains controversial (9), although two recent studies (10)(11) have shown the superior diagnostic ability of equimolar response assays in differentiating between benign prostatic hyperplasia and prostate cancer. However, there has been no investigation on the impact of assays that are either nonequimolar or biased (i.e., not numerically calibrated to the International Standards) on the decision to recommend a patient to undergo a prostate biopsy for cancer detection. This is a relevant clinical question because a biopsy is a costly and invasive procedure that is often associated with negative psychologic consequences.
Whereas the majority of commercial assays are equimolar and unbiased (12), there remain some that are not. In this study we used a computer-simulation-based approach to investigate the effects of bias, nonequimolarity, and analytical imprecision on the decision to recommend a patient for a prostate biopsy.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
population-based psa concentrations
The study population consisted of males in the United Kingdom 45–84 years of age because this age group represents the population most likely to undergo a PSA test. On the basis of a previous report of PSA values recorded during 1999 in the population of the United Kingdom (14), it was possible to construct approximate age-specific log-normal PSA distributions for men 45–54, 55–69, and 70–84 years of age. Each simulation sample size of 30 000 was achieved by equally sampling 10 000 men from the three age-specific PSA distributions.
laboratory variation
For each simulation the impact of laboratory and sample-handling variability was modeled by assuming that the overall assay CV was 5%, 7.5%, or 10%. Including this variation in the simulated age-specific PSA distributions required the use of a simple statistical model, where:
 |
assay inaccuracy (bias)
It was assumed that a biased assay will proportionately over- or underreport the tPSA concentration depending on the underlying true tPSA concentration in the sample. The amount of bias can be calculated using the following equation:
 |
nonequimolar assay
The impact of a nonequimolar assay, as demonstrated by Wians et al. (15), is that the assay does not respond proportionately to the amount of tPSA in a sample, but is biased by a rate proportional to the amount of fPSA. The value of the assay-reported PSA can be calculated according to the following equation:
 |
outcome measurements
Because age-specific population distributions of PSA were calibrated according to the population of the United Kingdom, age-related PSA reference intervals in the United Kingdom were adapted, such that men 45–54, 55–69, and 70–84 years of age were recommended for a biopsy if their PSA was >3, 4, or 5 µg/L, respectively.
To measure the effect of bias and nonequimolarity on the recommendation to carry out a prostate biopsy, for each of the 1000 simulations the following were recorded for each age group separately:
To minimize the effects of sampling errors, these estimated rates were averaged over the 1000 simulations in each age group for each combination of assay variability, bias, and nonequimolarity. The effect of differing amounts of bias and nonequimolarity on the average estimated false-positive and -negative biopsy recommendation rates was compared with that of an unbiased equimolar response assay. These were translated into proportions of men undergoing a PSA test who would have incorrect biopsy recommendations based on the size of the population of the United Kingdom in 2001 (16) and national PSA testing rates, estimated in 1999 (14). All PSA tests were assumed to be performed with the same assay.
sensitivity analysis
To assess the sensitivity of the results to changes in the age-specific PSA distributions, we performed several sensitivity analyses in which the relative proportions of men in each of the three age groups who would be considered test-positive were changed. To assess the impact of using age-specific PSA cutoffs, we performed sensitivity analyses using single PSA cutoffs of 4 and 3 µg/L for each age group. In each sensitivity analysis, all combinations of nonequimolar and biased responses were used, and outcomes were recalculated.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
70 years, respectively, in 1999, we determined that 
325 000 PSA tests are carried out annually in the United Kingdom. On the basis of the assumed age-specific distributions, 9%, 16%, and 26% of the PSA tests in the 45–54, 59–60, and 70–84 age groups, respectively, will be over the age-specific PSA cutoffs, leading to 60 000 biopsy recommendations per year (an overall biopsy rate of 18.5%).
If a calibrated and equimolar assay was used, laboratory and sample handling errors of 5%, 7.5%, and 10% produced false-positive and -negative biopsy recommendation rates of 0.5%, 0.7%, and 0.9%, respectively, which is equivalent to 1600, 2300, and 3000 false-positive biopsies and an equivalent number of false-negative biopsies.
The effects of biased and nonequimolar assays on PSA values in terms of the proportions of men who would be wrongly recommended or failed to be recommended for a biopsy for a constant laboratory and sample handling CV of 7.5% are shown in Table 1
. Other values of laboratory CV (5% or 10%) produce similar results when some degree of bias or nonequimolarity is present (data not shown). In the case of an equimolar response assay, as the proportion of bias increases, the false-negative rate falls to 0% very rapidly, whereas the false-positive rate steadily increases to 5.6% with a positive bias of 25%. Negative amounts of bias have the opposite effect, such that an assay that systematically underreports PSA values leads to large increases in the false-negative rates and decreases in the false-positive rates. However, a constant bias across all PSA concentrations rather than a bias proportionate to PSA concentration has little impact on the estimated false-negative or -positive rates in Table 1
(results not shown).
|
In the case of an unbiased but nonequimolar assay, the false-positive rates increase to 11.8% and 13.3% for assay response rates of 10% and 25%, respectively, as the fraction of fPSA in the sample increases to 50% (Table 1
). Nonequimolar assays will always overestimate the amount of tPSA in a sample and therefore reduce the number of false-negative recommendations; in most cases examined in this analysis, the false-negative recommendation rates were 0%.
Use of both a biased and a nonequimolar assay greatly inflates the rate of false-positive biopsy recommendation rates from 0.7% to 18% in the most extreme case (Table 1
). In all scenarios, the false-negative rate rapidly decreases to 0%, meaning that biased and nonequimolar assays rarely underestimate the tPSA in a sample and will tend to overestimate the observed tPSA concentrations, leading to a false recommendation for biopsy.
Sensitivity analyses were conducted in which 7%, 10%, and 20% of the PSA tests in the 45–54, 59–60, and 70–84 age groups, respectively, were assumed to be over the age-specific PSA cutoffs. Even with this considerable change to the proportions of men regarded as test-positive, the false-positive and -negative biopsy recommendation rates remained very similar (within 1–2% of those shown in Table 1
; results not shown). Although use of a single PSA cutoff (either 3 or 4 µg/L) rather than age-specific cutoffs produced different estimates of the false-positive and -negative rates, the patterns seen in Table 1
remained (data not shown).
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
was shown to be robust to changes in key model assumptions by several sensitivity analyses. Although the simulations were carried out under the assumption that all PSA assays would use the same assay method, the estimated rates can be applied to the number of tests carried out with a particular assay to estimate the numbers of false-positive and -negative recommendations that it would produce. The results of this study are consistent with two previous reports that showed that the use of an equimolar assay can reduce the number of patients having an unnecessary biopsy by up to 10% (10)(11). Moreover, it has been shown that reported estimates of diagnostic ability are dependent on assay choice (17); thus, combining results from different assays can make generalizations difficult (18). From a clinical viewpoint it is important to minimize the effects of positive bias and nonequimolar responses to reduce the number of false-positive recommendations because this will lead to a lower cost burden for the healthcare provider and will reduce unnecessary stress and discomfort for patients. Assays that are negatively biased will substantially increase the number of false-negative biopsy recommendations and will lead to many men not being recommended for a biopsy when their true PSA is above the cutoff; in this case, cancers will potentially be missed.
Even when an unbiased, equimolar assay is used, sample handling and laboratory variability of 5–10% can still lead to misclassification errors. Such random variability will mean that some men whose true values are close to the cutoff will be incorrectly classified. Although we were able to quantify the effects of this source of random variation on biopsy recommendation rates, other sources of variation may also have significant effects on both true and observed PSA values. PSA values are known to vary within an individual over time, and a recent metaanalysis suggested that intraindividual variation is associated with a CV of 12% (19). Although this additional source of variation is clearly significant, to isolate the effects of using biased or nonequimolar assays, we assumed that the simulated value of PSA represents an individual’s true PSA at the time of sampling. For the same reasons, this study did not attempt to account for the effects of other potential diagnostic tools in reaching a final recommendation for biopsy. In practice, results of a digital rectal exam, symptomatology of the patient, or a strong family history of disease may provide additional evidence to support or not support the recommendation for a biopsy.
This study used PSA testing rates based on a survey conducted in general practice in the United Kingdom in 1999 (14). However, there was no information about the nature of the population undergoing tests, and it is possible that these men were more likely than the general population to be symptomatic. Indeed, 16% of men 55–69 years of age had a PSA >4 µg/L compared with a mean of 11% (range, 7–15%) in the worldwide prostate cancer screening trials, which recruited men >55 years of age (1). In the general population in the United Kingdom, with a higher proportion of asymptomatic men, the PSA distribution would certainly be expected to be lower. However, a more recent survey of PSA testing in general practices in the United Kingdom suggests that the proportions of men 50–59 and 60–69 years of age with a PSA concentration >4 µg/L are 9.7% and 18.8%, respectively; therefore, the PSA distribution in the 55–69 age group is unlikely to have changed significantly (20). These revised estimates fall within the bounds of the sensitivity analyses considered here and suggest that any changes in the numbers of PSA tests performed will lead to an almost proportionate increase in the absolute numbers of false-positive and -negative recommendations.
This work was commissioned by the PSA Isoform Working Group, which reports to the Scientific Reference Group of the NHS Prostate Cancer Risk Management Programme. The PSA Isoform Working Group comprises:
In conclusion, this analysis has shown the effect that using biased and nonequimolar PSA assays can have on false-positive and -negative biopsy recommendation rates. It strongly supports the assertion that assays for PSA and its fractions must be calibrated to the International Standards and be unbiased and equimolar in their response. These results highlight the importance of minimizing laboratory variation; although the false-recommendation rates are low in the case of an unbiased equimolar assay, even small increases in assay CV can lead to significant increases in false-negative and -positive rates. Although this analysis is specific to PSA testing, these conclusions are relevant to other diagnostic tests. Efforts to reduce assay CV and to ensure accurate calibration against relevant International Standards are essential to minimize the likelihood of making incorrect clinical decisions that are potentially detrimental for both the patient and healthcare provider.
|
Footnotes |
|---|
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
1-antichymotrypsin. Clin Chem 1991;37:1618-1625.The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  C. Stephan, M. Klaas, C. Muller, D. Schnorr, S. A. Loening, and K. Jung Interchangeability of Measurements of Total and Free Prostate-Specific Antigen in Serum with 5 Frequently Used Assay Combinations: An Update Clin. Chem., January 1, 2006; 52(1): 59 - 64. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
