(882) Performance of the Global Open Source Severity of Illness Score at a Quaternary Children’s Hospital

Introduction: The Global Open Source Severity of Illness Score (GOSSIS) has emerged as a generalizable mortality model to perform risk adjustment in cohorts of critically ill adults, using data routinely recorded in the electronic health record (EHR). We evaluate the ability of GOSSIS to predict mortality in children admitted to the pediatric intensive care unit (PICU).

Methods: This is a retrospective cohort study of diagnostically diverse admissions to a quaternary PICU from July 2016-December 2022. EHR data were curated from the first 24 hours of PICU admission. The GOSSIS imputation and prediction algorithms were used to calculate a probability of mortality for each encounter. Model performance was assessed by examining area under the receiver operating curve (AUROC), area under the precision recall curve (AURPC), standardized mortality ratios (SMR), calibration plots, and Brier scores before and after spline recalibration. Data are presented with counts (%), medians (interquartile range), and 95% confidence intervals.

Results: There were 16071 encounters and 285 (1.8%) deaths. 7197 (44.5%) patients were female. Median age was 6 (1-13). There were 8308 (51.4%) admissions from the emergency department, 3060 (18.9%) from other hospitals, 2514 (15.5%) from acute care, 1901 (11.8%) post-operatively, and 280 (1.7%) from other ICUs. The most common reason for admission was respiratory (6206, 38.4%), neurologic (2074,12.8%), metabolic (1907, 11.8%), and sepsis (1339, 8.3%). The original GOSSIS model demonstrated excellent discrimination with an AUROC of 0.89 [0.87-0.91] and AUPRC of 0.38 [0.32-0.44]. Visualized calibration was poor, SMR was 0.11, and Brier score was 0.07. Discrimination performance was similar after recalibration with an AUROC of 0.88 [0.86-0.91] and AUPRC of 0.38 [0.32-0.44]. Both visualized calibration and associated metrics improved, with Brier score of 0.01 and SMR of 1.0.

Conclusions: In this diagnostically diverse cohort of critically ill children cared for in a quaternary PICU, the GOSSIS demonstrated excellent performance and required recalibration. Open-source severity of illness models coupled with interoperable EHR data transfer can be expected to facilitate population risk adjustment in support of multicenter performance improvement efforts in the field of pediatric critical care.