Back to 2026 Abstracts
From Notes To Registry Fields: Large Language Models Reduce Burden Of Registry Data Abstraction By 70%
Andrew M. Well, MD MPH MSHCT, Daniel Fabbri, PhD, Beth Sneed, Andrew Smith, MD, MSCI, MMHC, Neal Maynord, MD, Garrett Coyan, MD, MS, David Bichell, MD, Ziv Beckerman, MD, Yishay Orr, MD, PhD, Carlos M. Mery, MD MPH.
Vanderbilt University Medical Center, Nashville, TN, USA.
Objective(s): The growth of congenital heart disease registries and required data submissions increases manual abstraction of unstructured chart data . As artificial intelligence — particularly large language models(LLMs) — has advanced, opportunity exists to reduce resource demands. We present pilot results of an LLM-based chart abstractor for the Society of Thoracic Surgeons Congenital Heart Surgery Database(STS-CHSD).
Methods: Six complications from the STS-CHSD were selected.(Figure) Operative notes, progress notes, and discharge summaries from the index hospitalization of 68 unique patients(30 training, 38 test) were digested by the HIPAA-compliant LLM platform. Initial variable definitions followed STS-CHSD criteria and were iteratively refined based on output and prioritizing Sensitivity. LLM results were compared against manual abstractions.
Results: In the training set, the STS-CHSD definitions yielded 89.4% accuracy(Sensitivity 100%, Specificity 87.9%). Definition refinement improved performance to 89.4% accuracy(Sensitivity 100%, Specificity 88.1%). In the test set, the refined definitions achieved 80.7% accuracy(Sensitivity 97.2%, Sensitivity 79.9%). Of 228 data points, 64 were flagged positive, reducing manual review volume by 71.9%, including 12(31.6%) charts that would not have required opening.
Conclusions: Early pilot data with a limited training dataset demonstrate promising results. The near 100% Sensitivity allows for limiting manual chart review to only those datapoints flagged as positive. Further work is needed to further refine variables with larger training datasets, test the reminder of STS-CHSD variables, and develop a workflow integrating the LLM analysis.
Back to 2026 Abstracts
