Challenge
Despite the wealth of information generated in Electronic Health Records (EHR), medical education rarely uses student-authored clinical notes to evaluate learning experiences. Instead, schools rely on retrospective, manual reporting from students and preceptors—a process limited by memory and prone to inconsistency.
New technologies, including large-scale natural language processing and generative AI, offer promising ways to improve how clinical case exposure and patient volumes are tracked in Undergraduate and Graduate Medical Education (UME and GME). These tools can also strengthen the assessment of whether learners encounter the conditions and skills essential for a comprehensive medical education.
Solution
We developed a method to harness Large Language Models (LLMs) to summarize and analyze medical student-authored clinical notes from Epic across inpatient and outpatient settings. This approach incorporates note content alongside encounter metadata such as diagnoses, procedures, and therapeutic interventions.
The result: a novel Clinical Education Dashboard (CED) designed to visualize and quantify trainee experiences, linking them directly to educational goals.
Our first use case focuses on evaluating exposure to “Blue Recs”—key diagnoses or procedures identified by Duke University School of Medicine as high-priority learning opportunities during core clinical clerkships. These hands-on rotations are central to bridging classroom learning with real-world practice.
Unlike billing codes, which often miss clinical nuance or risk being incomplete, student notes reveal deeper insight into what was learned: presenting symptoms, histories, diagnostic reasoning, and management plans. Even so, we needed to identify and read student notes in order to identify which patients they visited and when. By matching student to patient visit, associating the visit with basic metadata, and evaluating the narrative written, the dashboard allows both students and educators to gain a clearer, timelier picture of individual and class-wide clinical experiences.
Impact
The Clinical Education Dashboard offers a first-of-its-kind solution in medical education, with the potential for:
- More accurate and efficient assessment across training programs and specialties
- Proactive individualization of learning opportunities
- Correlation of clinical encounter patterns with educational outcomes
- Increased competitiveness of our graduates for their next positions
- Reduced administrative time and costs due to enhanced student assessment procedures (which, in the future, may also be applied to residents)
- Easy dissemination to other Duke inter-professional health and GME programs
By transforming how clinical learning is measured, the CED empowers educators and students to align training with meaningful, measurable patient care experience.
