(387) Raising Awareness of Potential Biases in Medical Machine Learning: Experience From a Datathon

Introduction: While reporting tools and other methods exist to safeguard against bias in machine learning models, the risk of bias remains high in model development, potentially limiting clinical impact. Increased awareness is arguably the first step toward reducing bias. We challenged participants of a multidisciplinary Datathon to explore potential bias in predictive models in the Global Open-Source Severity of Illness Score (GOSSIS).

Methods: The Datathon took place over 2 days in February 2024. Participants included trainees, faculty, and staff from the University of Pittsburgh and UPMC. GOSSIS is validated, globally applicable, intensive-care severity of illness score developed utilizing over 260,000 ICU admissions in the United States and Australia/New Zealand. Participants evaluated bias utilizing three datasets – the Women in Data Science Datathon 2020 dataset, the eICU collaborative research database (CRD) US data subset of GOSSIS dataset, and/or a synthetic derivative of UPMC data. Participants were guided by the Prediction model Risk Of Bias Assessment Tool (PROBAST) questions. Submissions were analyzed to extract tools used and to categorize analysis approaches and conclusions into common themes.

Results: Sixteen participants forming five teams, ranging from 1-5 participants, completed the Datathon. Three teams included both clinicians and informaticians, one team consisted of two clinicians, and the final team had one informatician. Two teams used the Women in Science dataset, two used the synthetic UPMC dataset, and one team used both; none utilized the eICU-CRD dataset. Four teams utilized Python, and one R, for analysis. Age, race/ethnicity, and mortality were the most frequently examined features (4/5 teams each). Three main analysis approaches included GOSSIS score, demographics, and data missingness. No team found clear sources of bias, but several areas of concern arose including, data missingness, population representativeness, and performance differences across hospitals.

Conclusions: Identification and reduction of bias in medical machine learning models remains an important challenge. Our open-ended Datathon demonstrated that multidisciplinary team-based investigation could serve as a model to explore hypotheses and identify these biases.