(1302) APACHE 5: An Updated Classic

Introduction: The discrimination and calibration of benchmarking models such as APACHE drift over time, requiring periodic updates. When a model is developed, by definition the standardized mortality ratio (SMR; observed/predicted) will be trained to 1.00. APACHE IVb, released in 2016, currently overpredicts mortality (SMR=0.91) potentially causing overconfidence in quality of care. Similar issues are encountered with predicting eventual active therapy or length of stay at the hospital or ICU level, and for specific subgroups such as CABG patients.

Methods: The APACHE Outcomes database has detailed data and acute physiology scores (APS) on 864,881 patients from 112 hospitals between 2016 to 2023. Calibration curves (APS vs ICU mortality) suggested March 2020-April 2022 data was negatively impacted by the COVID-19 pandemic, and this data was censored leaving 615,588 that was split between a training and test set. Variables from APACHE IV (age, co-morbidity and physiology on ICU day 1) were examined against numerous outcomes including hospital mortality. The discrimination and calibration of the logistic models were assessed by AUROC and Brier’s score, respectively.

Results: Observed ICU mortality was 7.2% for non-CABG, and 0.8% for CABG patients. 52 equations were updated; 42 for binary outcomes (ICU and hospital mortality, active treatment) at various times for patient subgroups, and 10 for ICU and hospital length of stay and ventilator patients. The SMR for the ICU test set is 1.02 with AUROC of 0.91 and Brier score of 0.05 on day 1. Corresponding values for the CABG mortality model are 1.00; 0.88 and 0.01.

Conclusions: APACHE 5 corrects the gradual calibration drift APACHE IVb with excellent discrimination and calibration on a contemporary sample, once the distorting effects of pandemic-era data is censored.