(1298) Validation of Rapid Response System Using Artificial Intelligence in a Japanese University Hospital

Introduction: The efficiency of a rapid response system (RRS) depends on Early Warning Score (EWS) systems. We have used the St. Marianna modified early warning system (SMEWS) for RRS activation. While SMEWS was beneficial, it sometimes missed parameters, potentially underestimation of patient severity. Consequently, we adopted the Visensia Safety Index (VSI)®, an AI-based scoring system. VSI scores range from: ≥3.0 (high risk), 2.1-2.9 (medium risk), 1.5-2.0 (low risk), to 0.0-1.4 (normal risk), based on five parameters. By activating the RRS at VSI ≥3.0, we achieved an activation rate of 68.6 per 1,000 admissions. In the context of a mature RRS at a Japanese university hospital, we validated VSI's performance in predicting unexpected deaths.

Methods: We collected hourly VSI data for the 24 hours before discharge for patients aged 15 and older who were admitted and discharged from St. Marianna University Hospital’s general wards between June 5, 2023, and April 30, 2024. To examine the relationship between unexpected deaths and VSI, we included patients with: (1) SMEWS scores of 0-4 without a score of 3 by a single vital sign, and VSI < 1.5 (normal risk), for 24 hours before discharge, and (2) no missing vital data at discharge. We analyzed the correlation between VSI and mortality at discharge.

Results: We included 14,840 cases, of which 85 were unexpected deaths. The sensitivity of VSI scores ≥1.5 (low risk) for predicting mortality at discharge was 17.9% 18 hours, 36.5% 12 hours, 50.0% 6 hours, 56.6% 4 hours, 60.0% 2 hours, and 64.7% 1 hour before discharge, showing an increasing trend. For VSI scores ≥3.0 (high risk), the sensitivity was 3.6%, 10.6%, 25.6%, 32.5%, 41.2%, and 44.7% during the same time windows. The positive predictive values (PPV) of VSI scores ≥1.5 were 6.7%, 3.9%, 2.2%, 4.9%, 5.1%, and 5.5%, respectively. The PPVs for VSI scores ≥3.0 were 4.2%, 9.7%, 18.9%, 22.0%, 24.3%, and 25.0%, respectively, showing a significant increase (P < 0.0001).

Conclusions: In a mature RRS with an activation rate of 68.6 per 1,000 admissions based on VSI ≥3.0, VSI ≥1.5 showed a sensitivity of 50.0-60.0% 6-2 hours before discharge, and VSI ≥3.0 demonstrated a PPV of 22.0-25.0% 4-1 hours before discharge. VSI proved to have sufficient predictive ability for unexpected deaths.