(396) One Agent Is Good, Is Five Better? Multi-Agent Versus Single Agent Comparison for Medical Reasoning

Introduction: Recent advancements in artificial intelligence (AI), particularly large language models (LLMs), show promise in supporting clinical decision-making. Inspired by Li et al. (2024), we hypothesized that a multi-agent AI system, mimicking the collaborative nature of intensive care unit (ICU) teams, would outperform a single-agent system in answering critical care medical questions. This has potential to enhance decision support in time-sensitive, high-stakes ICU scenarios.

Methods: We developed two AI systems using Generative Pre-Trained Transformer (GPT)-3.5 Turbo: a multi-agent system with five specialized medical "personalities" (intensivist, cardiologist, endocrinologist, neurologist, general practitioner) and a single-agent generalist system. Both systems were tested on 500 questions from the MedMCQA dataset, focusing on critical care scenarios. The multi-agent system incorporated a discussion round among agents and a meta-agent for final decision-making, mirroring multidisciplinary ICU care. The primary metric was accuracy.

Results: The multi-agent system significantly outperformed the single-agent system, achieving 71% accuracy compared to 45% for the single-agent (p < 0.001). The multi-agent system showed superior performance across all critical care domains, most notable in the complex, multisystem cases typical of ICU patients.

Conclusions: The multi-agent AI system significantly outperformed the single-agent system in answering critical care medical questions, mirroring the benefits of multidisciplinary care in ICUs. This approach could enhance clinical decision support systems in critical care settings, potentially improving patient outcomes by providing rapid, comprehensive insights. Future research should focus on real-time integration of multi-agent AI systems into ICU workflows, evaluating their impact on clinical decision-making, resource utilization, and patient outcomes. As AI continues to evolve, multi-agent systems could become valuable tools in augmenting ICU team capabilities, especially in resource-limited settings or during complex, time-sensitive scenarios.