Background
Echocardiography AI systems such as Us2.ai can flag patients at risk for cardiac amyloidosis (CA) at scale, but retrospective validation requires structured patient-level metadata - confirmed diagnosis, comorbidities, functional class, laboratory values - that remains locked in unstructured doctoral letters. Manual extraction at cohort scale is infeasible.
Methods
We developed CLEP (Clinical Letter Extraction Pipeline), a modular NLP pipeline extracting structured clinical data from German cardiology letters. CLEP applies a rule-based engine across 28 fields spanning CA diagnosis, CA subtype, comorbidities, laboratory markers, NYHA class, Perugini score, and treatment status. Rules are scoped to relevant document sections and include explicit negation handling.
Extraction rules were developed iteratively with LLM assistance (Claude, Anthropic) - a human-in-the-loop workflow accelerating rule authoring while preserving full interpretability. For critical fields, a local LLM (Mistral-Nemo via Ollama) can optionally serve as fallback, receiving only anonymised keyword-filtered context to preserve patient privacy. CLEP was evaluated on 96 manually annotated patients and applied to 6,034 documents from 2,200 patients across three pathways at University
Hospital Heidelberg.
Results
CLEP achieved 93.8% field-level accuracy and 96.8% coverage using rules alone. CA diagnosis reached 94.7% accuracy, CA type 97.5%.
The distribution of CA subtypes (ATTRwt predominating, followed by AL and ATTRv) is consistent with literature and clinical practice across all pathways. Joining CLEP output to Us2.ai scores yielded an AUC of 0.86 for confirmed CA prediction, similar to published Us2.ai validation results (1) and providing independent real-world evidence for validity of both the algorithm and CLEP-extracted labels.
Conclusion
CLEP enables high-accuracy structured extraction from realworld EHR, supporting systematic retrospective validation of echo AI cohorts at scale. Its modular architecture allows rapid adaptation to new clinical domains and document types, making it a broadly applicable framework for LLM-assisted rule development and structured clinical data extraction. We plan to extend this approach to additional cohorts and AI projects beyond cardiac amyloidosis, further establishing CLEP as a reusable infrastructure for real-world AI validation.
1. Diagnosis of Cardiac Amyloidosis on Echocardiography Using Artificial
Intelligence. Ioannou, Adam, Michel G. Khouri, Takeshi Kitai, Sreekanth
Vemulapalli, Chung-Lieh Hung, Sze Chi Lim, Matthew Frost et al. 2026,
Circulation: Cardiovascular Imaging.
