The radiology department at Einstein Medical Center, Jefferson Health (Philadelphia, PA), earned recognition as a finalist for ECRI's 17th Health Technology Excellence Award for its efforts to increase patient adherence to recommendations for follow-up for imaging exams. The team conducted a study to test the effectiveness of a tracking and reminder system that uses artificial intelligence (AI)—specifically, a natural language processing algorithm—to automate the process.

The Health Technology Excellence Award recognizes outstanding initiatives undertaken by member healthcare institutions to improve patient safety, reduce costs, or otherwise facilitate better strategic management of health technology. For details about the winning submission and other finalists, see The Health Technology Excellence Award: Recognizing Exceptional Health Technology Management.

ECRI congratulates Dr. Jung Yun, Dr. Ryan Lee, and the rest of the team at Einstein Medical Center, Jefferson Health.

​ Einstein Medical Center, Jefferson Health. (Image courtesy of Einstein Medical Center.)​​

To increase patient adherence to recommendations for follow-up for imaging exams by improving the radiology department's ability to identify and contact patients who are due for a follow-up exam.

The team devised and conducted a study to test the effectiveness of an AI-based tracking and reminder system that uses a natural language processing algorithm. The system automated the process of identifying patients who require follow-up imaging based on radiology reports, organizing follow-up recommendations by due date, and sending reminders to patients about due or overdue exams.

​​​​​​​

Radiologists often recommend follow-up imaging when a patient's exam yields abnormal or suspicious incidental findings. At Einstein Medical Center, abnormal findings that require follow-up traditionally were communicated to the ordering provider through a closed-loop message notification system, but to the patient only via a physical letter mailed to the patient's home. The radiology department did not have any means to "close the loop" with patients on recommended follow-up examinations that were dictated in the radiology report. Thus, the department had no way of knowing whether patients were aware of their imaging recommendations and whether they completed their follow-up examinations.

A previous, informal study conducted by the medical center to assess patient adherence—but focusing specifically on radiology reports with pulmonary nodule follow-up recommendations—showed only a 24% adherence rate for imaging follow-up. Such nonadherence is not unique to Einstein Medical Center, as evidenced by a 2022 quality measure initiative proposed by the American College of Radiology (ACR): Novel Quality Measure Set: Closing the Completion Loop on Radiology Follow-up Recommendations for Noncritical Actionable Incidental Findings. As the ACR report notes, care gaps that occur "when radiology follow-up recommendations are poorly communicated or not completed" can adversely affect patient outcomes through "missed or delayed diagnosis."

To improve patient tracking and care coordination as recommended in the ACR quality measure, the Einstein team devised a study to assess the effectiveness of a third-party automated system for tracking outpatient radiology follow-up recommendations and sending reminder notifications to patients.

For this study, the team performed a prospective analysis of all outpatient diagnostic radiology reports, excluding mammographic radiology reports, generated at the institution from July 1, 2021, through August 24, 2022. These reports were processed through the third-party system, which uses a natural language processing algorithm to determine the recommended radiologic examination and the due date.

Patients in the study were randomly assigned to one of two groups (see the figure):

1. An intervention group, which received a series of SMS text notifications (issued by the automated system) before and after the recommended due date. SMS text messaging, using phone numbers retrieved from the patient's electronic medical record, served as the sole method of communication. For this group, text notifications were sent three weeks after the initial examination, then immediately after the recommended due date and every two weeks thereafter if needed, up to a maximum of three total additional reminders.

2. A control group, which did not receive any SMS notifications during the study period. Reminder notifications were sent to the patients in this group once the study period concluded.

Patients were deemed compliant with recommendations if follow-up imaging examinations were completed within the ACR compliance range—a period starting 30 days before the recommended due date and ending 60 days after it. The third-party automated system was used to determine whether follow-up imaging was scheduled or completed.​​

The intervention group received multiple reminder notifications (by SMS text) during the study period, while the control group did not receive any reminders until after the expiration of the ACR compliance range (defined as the period 30 days before and 60 days after the recommended due date).​​

Outcomes were measured by (1) the number of patients who completed their recommended follow-up imaging examination within the ACR compliance range and (2) the adherence rate, or the percentage of completed follow-up recommendations within the ACR compliance range.

The medical center's analysis covered 275 radiology reports. The control group (no notifications) contained 116 outpatient radiology reports, and the intervention group (periodic notifications) contained 159 outpatient radiology reports. Findings are as follows:

• Within the control group, 63 of 116 patients (54%) had completed their recommended follow-up imaging before the expiration date of the ACR compliance range.
• Within the intervention group, 111 of 159 patients (70%) had completed their recommended follow-up imaging before the expiration date of the ACR compliance range.
• The adherence rate of 70% in the intervention group was statistically significantly higher than the adherence rate of 54% in the control group (p = 0.008).

​

Efforts to contact patients can be time-intensive, particularly if the patient is hard to reach, and patients can be lost to follow-up if these efforts are not adequately tracked by primary care physicians, the radiology department, or scheduling staff.

As illustrated by the Einstein team's study, the use of an automated system to identify patients with follow-up imaging recommendations and to send reminder notifications has the potential to improve patient adherence. That, in turn, may lead to improved health outcomes for the patient, as well as increased revenue for the facility.​