AI-Generated Discharge Summaries in Clinical Practice: A Structured Quality and Safety Assessment

J. Krefting (München)1, F. Förster (Garching)2, S. Bleßmann (Garching)2, D. Zverev (Garching)2, H. Scheidhauer (München)3, T. Trenkwalder (München)3, M. von Scheidt (München)3, H. Schunkert (München)3, T. Keßler (Homburg/Saar)4, A. S. Koepke (Garching)2
1Deutsches Herzzentrum München Klink für Herzkreislauferkrankungen München, Deutschland; 2TUM School of Computation, Information and Technology Munich Center for Machine Learning Garching, Deutschland; 3Deutsches Herzzentrum München Klinik für Herz- und Kreislauferkrankungen München, Deutschland; 4Universitätsklinikum des Saarlandes Innere Medizin III - Kardiologie, Angiologie und internistische Intensivmedizin Homburg/Saar, Deutschland

Background:
Discharge letters are essential for continuity of care but are time-consuming to prepare, particularly in cardiology, where guideline-based follow-up recommendations are crucial for preventing cardiovascular events. Artificial intelligence (AI) may reduce documentation burden by generating structured discharge letters from routine clinical data.

Objective:
To describe the development, clinical use, and evaluation of an agentic AI-supported system for generating cardiology discharge letters within a privacy-preserving clinical infrastructure.

Methods:
An AI-based documentation workflow was developed to generate draft cardiology discharge letters from routinely available clinical information, including diagnoses, procedures, medication data, laboratory results, imaging and diagnostic reports, clinical notes, and discharge recommendations. To ensure data privacy, the large language model (LLM) was operated on GPU infrastructure within the clinical network, avoiding transfer of sensitive patient data to external cloud services. Output was guided by standardized templates, that support consistent drafts and allow new letter formats to be introduced without modifying the underlying model. Drafts were designed for clinician review, editing, and approval before release. Development focused on workflow integration, source traceability, and human-in-the-loop validation. Clinical use was assessed by adoption in routine cardiology workflows, editing patterns, turnaround time, and user feedback. Evaluation compared AI-generated drafts with conventional discharge letters regarding completeness, factual correctness, readability, structure, time savings, errors, hallucinations, and required corrections.

Results:
The AI-supported workflow was successfully implemented and generated structured cardiology discharge letter drafts from heterogeneous clinical source data within a protected clinical IT environment. Physicians rated generated discharge letter drafts across nine dimensions on 5-point Likert scales (n = 96 feedback submissions from 107 sessions). Overall quality received a mean rating of 3.21/5, with writing style (mean 3.25/5), document structure (mean 3.15/5), and medical terminology (mean 3.16/5) rated similarly. Clinical focus was rated lower (mean 2.99/5). When compared directly against manually written letters, generated drafts received a mean rating of 2.51/5, suggesting modest perceived quality relative to the conventional approach.

Drafts required a mean of 6.1 manual corrections (range 0–30, n = 87), indicating substantial inter-document variability. The potential harm of uncorrected errors was rated at a mean of 2.16/5 on a scale from 1 (none) to 5 (catastrophic), with the left-skewed distribution indicating that most errors were judged as low-to-moderate severity. These preliminary findings suggest that local LLM-based documentation support can reduce documentation burden, particularly in simple and routine cases, while preserving clinical control through mandatory human review.

Conclusion:
AI-generated discharge letters may support cardiology documentation by reducing workload and improving standardization. A flexible template-based architecture combined with a locally operated LLM offers a scalable, privacy-preserving approach for sensitive patient data. Mandatory clinician review remains essential for safe implementation.