Mixed methods analysis of an automated email audit and feedback intervention for fostering (emergency) physician reflection

Abstract

Background

physician refelection requires personalized, timely and growth-oriented feedback. Iterative learning from multiple low-pressure events can be personalized to target areas of weakness and show sequential growth. Since emergency physicians typically work individually to deliver episodic care, opportunities for them to obtain iterative feedback on their clinical performace is often limited. Our study sought to evaluate whether physician reflection is facilitated through the 72hr re-admission alert received by emergency physicians in the Calgary zone.

Implementation

The 72-hr readmission alert is already part of feedback received in the Calgary Zone. Our study was specifically looking at understanding the utility of these alerts to emergency physicians through qualitative interviews. Our team of two interviewers (DA and CP) collected and banked the data through anonymized one-on-one interviews. Themes from these interviews will be used to guide future adjustments made to the alert and dictate it’s future role in emergency physician feedback. Current changes based on preliminary data have included the ability to customize re-admission alert time-frames based on personal preference. We are currently in the process of analyzing the themes that will shape further improvements made to the alert.

Evaluation Methods

This mixed methods realist evaluation consisted of two sequential phases: an initial quantitative phase examining the general features of 72-hr readmission alerts sent over a 1-year period (4024 alerts from May 2017-2018) and a subsequent qualitative phase involving 17 semi-structured interviews to generate “context-mechanism-outcome” (CMO) statements to guide refinement of our program theory.

Results

CMO statements revealed emergency physician stakeholders were concerned that the alert impacted personnel decisions, changed patient return expectations and didn’t involve consulting services. Physicians, who didn’t believe alerts were involved in personnel decisions, were more likely to pursue balanced reflection/acquisition after each alert when receiving illness related returns. Conversely, physicians, who believed alerts were involved in performance assessment/hiring decisions, were more likely to defensively change their practice. Commonly cited areas of improvement were the ability to personally adjust time criteria for alerts and involving consulting services in feedback.

Advice and Lessons Learned

• It is essential to partner with local departments who can use formal (newsletters) and informal (word of mouth) avenues to encourage participation in the study. Participant anonymity must be emphasized when recruiting for qualitative interviews in order to receive the full scope of perspectives.
• Clear and concise scripts highlighting the objective of each question can ensure the quality of responses received and help interviewers probe further into the topic when necessary.
• When performing quality improvement studies on formal feedback mechanisms, faculty leadership buy-in is essential in order to ensure a safe environment for all participants.