(857) Predicting Noninvasive Failure in Children With a Risk Failure Model

Introduction: NIV use in respiratory failure has become a standard practice for critically ill children. To date, no one clinical metric has been proven to show the threshold of when a NIV treatment is no longer sufficient. Previous studies used tracheal intubation (TI) as NIV failure definition which lacks clinical validity, given timely transition from NIV to IMV may best serve critically ill children. We hypothesized a machine learning model incorporating multiple clinical data from EHR would be more effective to provide a prediction of NIV failure at a certain timeframe after NIV initiation using clinically meaningful NIV failure definition.

Methods: A retrospective cohort study with 5-year of data from a single PICU. A newly defined composite definition for Complex NIV failure utilizing unfavorable clinical outcomes (ICU mortality, new tracheostomy, IMV for > 7, relevant tracheal intubation associated events during TI, or severe desaturation during TI) was used as the primary outcome. A logistic regression machine learning model (L1 Lasso, L2 Ridge) was used to predict Complex NIV failure. EHR-based hourly vital signs were age normalized using Flemming’s z-score. PaO2 was calculated with Sauther method. Eye/Verbal GCS scores and NIV parameters including FIO2 were also included. Blood gas were classified categorically by acidosis analysis type. The 80% training dataset was used with a ten-fold cross-validation for optimal fitting and model performance was evaluated with the 20% test dataset. The model was compared for four-time frames of therapy (1- 6,12,18, and 24 hrs). Each model was compared for model accuracy with area under the precision recall curve (AUPRC) and area under the receiver operating characteristic (AUROC) with standard deviation.

Results: 3844 NIV cases in final cohort were included with 6% of complex NIV failure rate. The 6-hour treatment timeframe with Logistic Regression Model demonstrated the best performance with AUROC = 0.80 (95% CI, 0.69-0.90) and AUPRC = 0.32 (95% CI, 0.10-0.52), followed by 12-hour timeframe with AUROC=0.72 (95% CI, 0.48-0.96) and AUPRC = 0.29 (95% CI 0.02-0.71).

Conclusions: A Machine learning logistic regression model was successfully created to predict complex NIV failure during NIV treatment. This model required external validation before clinical use.