(1479) Evaluation of State-of-the-Art Object Detection System During Simulated Combat Casualty Care

Introduction: Combat casualty care (C3) is done in challenging environments where clinical documentation (CD) is low, inhibiting effective triage and logistic response. We seek to automate CD through passive use of video and other sensors to infer casualty status, caregiver activities, and resource use. Computer vision (CV) models such as the state-of-the-art “you only look once” (YOLO) neural networks, are excellent at object detection but were developed on civilian and non-clinical datasets with no evidence of accuracy during C3. We tested the latest YOLO models on realistic human and manikin C3 simulations (sims).

Methods: Pilot study exploring accuracy of YOLO CV models at body detection during C3 sims using trained Army combat medics wearing helmet-mounted cameras while providing care to realistic manikins in pre-defined clinical scenarios aligned with combat medic guidelines. Sims recorded using 720p resolution at 30fps, covering 5-6 clinical activities and lasting 10-20 mins. We applied YOLO models and assessed their detection of patient, medic and body posture. We manually adjudicated quality of detection on every 100th frame of the videos (~1 every 3 secs). High quality (HQ) detection occurred when all major body parts and separation of medic and patient were identified without reasonable (e.g., non-detection of a few pixels of a limb) error. False alarms (FA) detected more people than present. Qualitative assessment of inaccurate detections was performed.

Results: 10 sims were performed yielding 59.5 minutes of video. YOLOv8, YOLOv8-pose, YOLOv9, and YOLOv9-seg model detections were each adjudicated on 1,080 frames. All models correctly detected all people in 17% of frames, correctly detected at least one person in 82% of frames, with a FA rate of 45-47%. HQ rates of 34%, 19%, 34% and 35% were found for v8, v8-pose, v9, and v9-seg respectively. Qualitative analysis revealed errors that: combined medic and patient limbs; omitted partial or complete bodies due to camouflage; resulted from image blur and close views.

Conclusions: State-of-the-art CV models lack accuracy for clinical utility in identifying body composition and differentiation between the medic during simulated C3 using helmet-mounted cameras. Customization and tuning for C3 use cases are necessary to support autonomous CD.