(411) Machine Learning to Improve Blood Culture Stewardship in the PICU

Introduction: Blood cultures are necessary to diagnose bloodstream infection (BSI) but have low positivity rates in children (6.5%-15%). This large percentage of negative results presents an opportunity for improved diagnostic stewardship. A predictive model that identifies children at low risk for BSI could reduce unnecessary blood cultures and associated false positives and unnecessary antibiotics. We aimed to develop a machine learning (ML) model to accurately identify children without BSI.

Methods: We conducted a retrospective single-center cohort study to develop and validate a BSI predictive model. We extracted electronic health record (EHR) data for children 3 months to 18 years old who had blood cultures sent in the first 24 hours of PICU admission. We considered all positive blood cultures as BSI. We trained an XGBoost ML model to predict BSI using vital sign, laboratory, and other clinical data. The model was trained on the first 80% of cases (1/1/2011 - 12/21/2018) and tested on the remaining 20% (12/22/2018 - 12/25/2020, temporal validation). The primary model performance metric was negative predictive value (NPV). We applied the models to the test set and computed NPV and the number needed to harm (NNH). Feature importance was determined via Shapley values.

Results: Blood cultures were obtained during 3,121 PICU admissions (2,100 children) and 205 (6.6%) were positive. A BSI probability threshold of 0.001 yielded an NPV of 1.0 in the training set and 0.97 in the test set. Model predictions using that threshold at the time of blood culture collection could have avoided 117 of 595 (20%) negative blood cultures while misclassifying only 4 patients with BSI, yielding a NNH of 29.3. Key model features included mean platelet volume, serum albumin, and temperature rate of change, all of which negatively correlated with BSI risk.

Conclusions: An XGBoost ML model using EHR data reliably predicted low risk of BSI. Model results could have prevented 20% of subsequently negative cultures with a small number of false negative predictions. If implemented as clinical decision support, similar models could help clinicians reduce unnecessary blood cultures among critically ill children.