(1351) Early Use of Critical Care Digital Quality Measures Are Associated With Decreased Ventilator Days

Introduction: We describe the development and deployment of locally interoperable, risk-adjusted digital quality measures (dQMs) for spontaneous awakening trials (SATs), spontaneous breathing trials (SBTs), and low-tidal volume ventilation (LTVV) in critically ill patients as part of a quality improvement (QI) program in a large health system.

Methods: Workflows and data extraction procedures were harmonized to yield dQMs for SATs, SBTs, and LTVV across 38 intensive care units (ICUs) in 21 hospitals. An available, open-source, hospital mortality model, the ‘Global, Open-Source, Severity of Illness Score’ was implemented into an informatics pipeline to create an ‘eGOSSIS’ for risk-adjusted mortality as a balancing measure. An additional gradient boosted ‘UPMC ICU mortality’ model and 4 heterogenous, stacked ensemble predicted duration of mechanical ventilation (DMV) models (1 model predicting up to 14 days of ventilation (14-day DMV model) and 3 multiple classifier models predicting 1, 2, 3, or 4-6 days of mechanical ventilation) were created. Model performance was evaluated with C-statistics, mean standard error (MSE), and concordance correlation coefficients (CCC). Compliance with dQMs was tracked alongside adjusted mortality, as well as actual versus predicted ventilator duration, in a dashboard and presented in feedback sessions with ICU clinical leadership beginning August 2020.

Results: Across all ICUs, compliance with SBTs improved from 81-87%, LTVV 80-90%, and SAT 27%-65%. Both the eGOSSIS and UPMC ICU mortality models had robust performance, with C-statistics of 0.85 (95% confidence interval [CI] 0.84-0.85) and 0.94 (95% CI 0.93-0.94), respectively. The 14-day DMV model MSE was 0.63 and CCC was 0.97. There were 94611 actual vs 94701 predicted ventilator days pre-QI program and 156864 actual vs 159470 predicted post-implementation. The classifier models corroborated fewer actual vs predicted ventilator days post-implementation. Risk-adjusted mortality remained stable.

Conclusions: dQMs coupled with robust risk-adjustment proved useful for efficiently tracking progress related to a ventilator management QI program in a large health system. Future work should focus on developing interoperable dQM bundles and risk models that can span disparate health systems to help improve patient outcomes.