A new initiative at Penn Medicine, known as CRISP (Clinical Reasoning Insights for Shaping Performance), aims to use artificial intelligence (AI) to provide detailed feedback to medical students and trainees, with the goal of improving clinical reasoning skills. The project is led by faculty from the Perelman School of Medicine: Jessica Dine, MD, MSHP; Janae Heath, MD, MSCE; Jennifer Kogan, MD; and Ilene Rosen, MD, MSCE. The Department of Informatics, Biostatistics and Epidemiology is also supporting the project.
CRISP will utilize AI-enabled systems that record and analyze clinical encounters through ambient listening technology. These systems are designed to evaluate a clinician’s reasoning process during patient interactions and offer targeted feedback.
“Clinical reasoning is a pivotal piece of excellent patient care,” said Heath, associate program director for the Internal Medicine Residency Program and assistant professor of Pulmonary, Allergy and Critical Care. “And so an improvement in those skills, resulting in higher levels of expertise, is going to directly benefit our patients.”
The initiative was selected as one of 11 projects funded by a four-year $1.1 million grant from the American Medical Association’s Transforming Lifelong Learning Through Precision Education Grant Program. This program focuses on precision education—an approach that tailors training to individual learners rather than using traditional methods that apply uniformly to all students.
“It’s incredibly exciting because not only is this the first true precision education at Penn, but I think it’s a really unique way to do precision education,” said Dine, associate dean of Assessment, Evaluation and Medical Education Research and a professor of Pulmonary, Allergy and Critical Care.
Penn Medicine has been adopting various AI technologies across its system—from tools that synthesize patient histories quickly to AI scribes that help with note-taking during visits.
Teaching clinical reasoning remains challenging because it relies on chance encounters with patients and subjective evaluations by faculty. Traditional observation methods do not allow for real-time assessment during every patient interaction. New technologies now make it possible for institutions like Penn Medicine to capture these skills more objectively.
“Clinical reasoning is a really hard skill to capture well,” Heath said. “But over the last two years, there’s been an explosion of new technologies available, including ones that Penn Medicine has embraced, that have created this new opportunity to capture clinical reasoning in different domains.”
The CRISP project will involve learners in four specialties: Internal Medicine, Emergency Medicine, Surgery and Radiology. The aim is to develop scalable educational tools that provide individualized coaching based on data-driven assessments across these disciplines.
According to project leaders at Penn Medicine—where academic programs are closely linked with clinical operations—the institution provides an ideal setting for exploring innovative approaches due to its collaborative culture among various specialties.
“Penn Medicine is thinking about education across the continuum and through a really innovative lens,” Heath said. “We have team members across the institution… who are pivotal to this project.”
During the first year of funding support from the American Medical Association grant program https://www.ama-assn.org/press-center/press-releases/ama-awards-12m-grants-advance-medical-education-through-precision, the team plans to build prototypes tailored for each specialty’s workflow. For example: In Internal Medicine and Emergency Departments they’ll collect ambient audio from conversations; in Radiology they’ll analyze documentation; while in Surgery they’ll monitor electronic health record interactions.
“This application of generative AI will allow us…to both map…clinical facts from those conversations…to medical knowledge graphs as well as analyze linguistic markers of critical thinking,” said Danielle Mowery, PhD, MS chief research information officer for Penn Medicine and assistant professor of Informatics. “This framework will enable…clinically grounded…feedback…about their diagnostic…and therapeutic reasoning as well as their case review.”
Direct observation remains part of training but data-driven measurements could clarify learner progress more objectively over time—a focus echoed by Rosen: “The idea is to really make a difference in how learners progress through their clinical education…including when they’re finished with their training.”
The analysis includes measuring speech patterns such as word count or semantic detail—factors shown by experienced clinicians—and considers contextual factors like shift length or team dynamics.
There are concerns about privacy if ambient audio collection becomes constant: “It could feel very Big Brother pretty quickly,” said Kogan.
Equity monitoring has been incorporated into CRISP’s design out of concern about potential bias perpetuated by automated assessments.“We’ve built in a lot of equity monitoring,” Heath said.“We’re being really intentional about that piece…because we don’t want…problematic assessments.”
The CRISP project involves input from students alongside educators throughout development—including an upcoming innovation hackathon scheduled for March 13–15 which brings together multiple stakeholders within University of Pennsylvania.https://www.pennmedicine.org/news/news-blog
“The learner voice is really critical,” Heath added.“Residents…and medical students have a role…in development…They’re instrumental…”
Prototypes will be tested beginning in 2027 among several hundred participants before broader rollout.The CRISP team also participates in an AMA consortium where recipients meet regularly online or biannually face-to-face—to share progress/challenges.https://edhub.ama-assn.org/pages/transforming-lifelong-learning-through-precision-education
“By having those conversations,we can accelerate growth more than if we were all working in silos…”Kogan stated.
Faculty hope CRISP becomes a model beyond Penn.“If our goal is…a true learning health system,this is a great prototype…”Dine commented.“Trainees receive feedback…system also learns…and adapts…”
