Explainable machine learning prediction of ICU mortality

Abstract

Background: There is a variety of mortality prediction models for patients in intensive care units (ICU) to guide appropriate clinical management. Advances in machine learning methodologies typically employ classifiers such as Neural Network and Random Forest which are often regarded by healthcare professionals as black boxes. These models often do not provide clear links between the input model features and output clinical event. We investigate whether features identified by Cox-Proportional Hazards (CPH) model can be used for ICU mortality prediction. Methods: We employ the PhysioNet Challenge 2012 dataset, a subset of MIMIC-II Clinical Database data of ICU patients admitted to Boston's Beth Israel Deaconess Medical Center from 2001 to 2008. The dataset is split into train set A, test set B and unseen set C, with 4000 patients each. Python is the programming language used alongside scikit-learn, and lifelines packages. Besides white-box feature selection methods (logistic regression and decision tree), we also explore using Cox-Proportional Hazards model for feature selection. We then trained the machine learning model using classifiers such as logistic regression and variants of decision tree. Extreme gradient boosted trees models performed better than other classifiers. The model is validated using 5-fold cross-validation and evaluated against unseen set C. The model performance is assessed using area under the precision-recall curve (AUC-PR) as the main metric. Findings: The data of about 12,000 patients is used, providing a high degree of generalizability. The number of statistically significant features identified by CPH (n = 16) is significantly smaller than logistic regression (n = 36), decision tree (n = 26) and all features (n = 42). With only 16 features used, the model achieves a performance of AUC-PR 0·438 on test set B, which is close to decision tree (AUC-PR 0·442) and logistic regression (AUC-PR 0·446) and all features (AUC-PR 0·446). Interpretation: The significantly fewer features identified by CPH allows the building of a model that is easily interpretable by clinicians whilst still achieving comparable results to other models. This finding allows clinicians to use CPH as an alternative method to determine and act on features that need to be closely monitored for ICU patients. © 2021 The Authors