(1368) Machine Learning-Based Prediction of ICU Complications Using Medication Data: A Validation Study

Introduction: Medications may influence intensive care unit (ICU) patient outcomes but are rarely included in machine learning (ML)-based prediction algorithms. To test the hypothesis that medication-related variables are important predictors of patient outcomes, the aim of this study was to validate two ML models incorporating medication administration data for predicting fluid overload (FO) and prolonged mechanical ventilation (PMV).

Methods: Previously, 28-variable traditional regression and supervised ML (XGBoost, Random Forest, Support Vector Machine [SVM]) models were developed for predicting FO at 72 hours (n=391) and PMV (n=318) using a cohort of adults admitted to any ICU at an academic medical center for longer than 24 hours between October 2015-2020. FO was defined as a positive fluid balance of at least 7% of the patient’s admission body weight and PMV was defined as greater than 5 days. Using a separately developed dataset of a larger population with similar characteristics (FO: n=4273, PMV: n=1721), both sets of models were validated. The primary outcome was area under the receiver operating characteristic (AUROC) of prediction models. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were also calculated.

Results: For predicting FO in the new dataset, the strongest-performing logistic model was the 28-variable model (AUROC 0.67, sensitivity 0.45, specificity 0.78, PPV 0.17, NPV 0.93) and the strongest ML model was Random Forest (AUROC 0.81, sensitivity 0.86, specificity 0.59, PPV 0.16, NPV 0.97). For predicting PMV in the new dataset, the strongest-performing logistic model was the base model with medication data added (AUROC 0.70, sensitivity 0.75, specificity 0.62, PPV 0.66, NPV 0.71) and the strongest ML model was SVM (AUROC 0.61, sensitivity 0.92, specificity 0.13, PPV 0.53, NPV 0.48). Feature importance analysis using SVM and Random Forest revealed severity of illness scores and medication data were the most important predictors.

Conclusions: Prediction model performance was similar between the original and validation datasets. Incorporation of medication information led to only modest improvement. Validation is an important step when implementing ML-based models at the ICU bedside.