Received on September 18, 2007
Accepted on May 6, 2008

Informatics and Statistics

How to Measure the Diagnostic Accuracy of Noninvasive Liver Fibrosis Indices: The Area Under the ROC Curve Revisited

¹ Université Pierre et Marie Curie-Paris; INSERM, UMR-S 707; unité de santé publique, Assistance Publique Hôpitaux de Paris, Hôpital Saint-Antoine, Paris, France
² Laboratoire Alphabio, Hôpital Ambroise Paré, Marseille, France
³ Département de Biostatistiques, CDL Pharma, Marseille, France
⁴ Service d'Anatomie Pathologique, Assistance Publique Hôpitaux de Paris, Hôpital Beaujon, Clichy, France
⁵ Service de Médecine Interne, Hôpital Pitié-Salpêtrière, Assistance Publique Hôpitaux de Paris, Paris, France

^* To whom correspondence should be addressed. E-mail: carrat{at}u707.jussieu.fr.

BACKGROUND: The area under the receiver operating characteristics curve (AUC) is widely used to measure the diagnostic accuracy of noninvasive fibrosis indices. However, use of the AUC assumes a binary gold standard, whereas fibrosis staging is based on an ordinal scale and also depends on the distribution of fibrosis stages in the study sample. We explored other fibrosis staging accuracy measures designed for ordinal gold standards, the C-statistic and the Obuchowski measure.

METHODS: We performed a simulation study to assess the bias in estimating the accuracy measures when the distribution of fibrosis stages in the study sample do not fit the reference distribution in the population to which the indices are applied. We also estimated the type I error of the tests comparing these measures in 2 samples with different distributions of fibrosis stages. We illustrated the practical use of these measures by reanalyzing real data.

RESULTS: Compared with the AUC or the C-statistic, the Obuchowski measure showed limited bias when the distribution of fibrosis stages in the study sample differed from the reference distribution. The type I error was strongly inflated with the AUC or the C-statistic but was preserved in the Obuchowski measure. When we compared noninvasive indices on real data, AUC analysis led to discordant results depending on how the fibrosis stages were grouped together. One single conclusion was drawn from the analysis based on the Obuchowski measure.

CONCLUSIONS: We recommend using the Obuchowski measure for assessing the diagnostic accuracy of noninvasive indices of fibrosis.