Received on January 14, 2009
Accepted on February 3, 2009

Evidence-based Medicine and Test Utilization

Dealing with Missing Predictor Values When Applying Clinical Prediction Models

¹ Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, the Netherlands
² Department of Epidemiology, Biostatistics and Health Technology Assessment, Radboud University Nijmegen Medical Center, the Netherlands
³ Department of Biostatistics, Vanderbilt University Medical School, Nashville, TN

^* To whom correspondence should be addressed. E-mail: k.j.m.janssen{at}umcutrecht.nl.

BACKGROUND: Prediction models combine patient characteristics and test results to predict the presence of a disease or the occurrence of an event in the future. In the event that test results (predictor) are unavailable, a strategy is needed to help users applying a prediction model to deal with such missing values. We evaluated 6 strategies to deal with missing values.

METHODS: We developed and validated (in 1295 and 532 primary care patients, respectively) a prediction model to predict the risk of deep venous thrombosis. In an application set (259 patients), we mimicked 3 situations in which (1) an important predictor (D-dimer test), (2) a weaker predictor (difference in calf circumference), and (3) both predictors simultaneously were made missing. The 6 strategies to deal with missing values were (1) ignoring the predictor, (2) overall mean imputation, (3) subgroup mean imputation, (4) multiple imputation, (5) applying a submodel including only the observed predictors as derived from the development set, or (6) the "1-step sweep" method. We compared the model's discriminative ability (expressed by the ROC area) with the true ROC area (no missing values) and the model's estimated calibration slope and intercept with the ideal values of 1 and 0, respectively.

RESULTS: Ignoring the predictor led to the worst and multiple imputation to the best discrimination. Multiple imputation led to calibration intercepts closest to the true value. The effect of the strategies on the slope differed between the 3 scenarios.

CONCLUSIONS: Multiple imputation is preferred if a predictor value is missing.