(891) Performance of PIM-2 and PRISM-III Mortality Prediction Scores in Children With Sepsis

Introduction: Pediatric sepsis studies frequently use mortality prediction scores, such as Paediatric Index of Mortality (PIM) and Pediatric Risk of Mortality (PRISM), to adjust for illness severity. These scores were derived in general PICU cohorts in which sepsis drives a large proportion of PICU mortalities. We hypothesized that PIM-2 and PRISM-III would have poor discrimination and calibration when selectively applied to two cohorts of children with sepsis.

Methods: We identified two pediatric sepsis cohorts: a prospective cohort through the Children’s Hospital of Philadelphia (CHOP) sepsis registry, and a retrospective multicenter cohort through the Virtual Pediatric Systems (VPS) registry. CHOP cohort patients were identified using the IPSCC 2005 criteria, while VPS cohort patients were identified using ICD-9 codes for severe sepsis (995.92) or septic shock (785.52). The primary outcome was all-cause PICU mortality. We assessed discrimination by the area under the receiver operating characteristic (AUROC) curve and calibration using the Hosmer-Lemeshow goodness-of-fit test.

Results: We identified 401 patients in the CHOP cohort and 10,768 patients in the VPS cohort. Discrimination of mortality by PIM-2 was poor in the CHOP cohort (AUROC 0.58 [0.46-0.69]) and fair in the multicenter VPS cohort (AUROC 0.72 [0.71-0.74]). PIM-2 model calibration for mortality was poor in both the CHOP cohort (χ2 181.2, p< 0.0001) and VPS cohort (χ2 2076.6, p< 0.0001), and the observed mortality rate was higher than predicted by PIM-2 in the VPS cohort (12.6% vs 5.9%). Discrimination of mortality by PRISM-III was poor in the CHOP cohort (AUROC 0.61 [0.50-0.72]) and fair in the VPS cohort (AUROC 0.76 [0.74-0.77]). PRISM-III model calibration for mortality was poor in both the CHOP cohort (χ2 87.9, p< 0.0001) and the VPS cohort (χ2 1196.3, p< 0.0001) and the observed mortality rate was higher than predicted by PRISM-III in the VPS cohort (12.6% vs 10.8%). In a sensitivity analysis of CHOP patients excluding those with hospital-acquired sepsis (n=284), discrimination and calibration of PIM-2 and PRISM-III remained poor.

Conclusions: PIM-2 and PRISM-III scores are poorly calibrated and underestimate the true risk of mortality in pediatric sepsis patients, limiting their utility to adjust for illness severity in sepsis studies.