2019 HSC Section 2 - Practice Management

ARTICLE IN PRESS

The Procedural Learning and Safety Collaborative and American Board of Surgery are studying the dissemination and implementation of the System for Improving and Mea- suring Procedural Learning (SIMPL) smartphone application as a novel tool to improve assessment and feedback in surgi- cal education. The application has been shown to feasibly integrate into the surgical workflow to increase the fre- quency and timeliness of postoperative feedback. 6 Surgical residents use SIMPL to request feedback on their level of per- formance and independence, which the attending physician then has 72 hours to provide. The SIMPL application uses the Zwisch model, a validated model of assessing autonomy of surgical residents in the operating room, that categorizes the resident’s performance as either Show and Tell, Active Assist, Passive Assist, or Supervision Only. 7 There is also a dictation feature which allows attending physicians to pro- vide more granular feedback on what the resident did well and areas to target for improvement. Currently, residency programs involved in the Procedural Learning and Safety Collaborative have a mean 23.7% completion rate of dictated feedback. When asked about the dictation feature, 7 of 10 programs surveyed described the dictation feature as valu- able 6 ; yet, the quality of the dictated feedback is unknown. Analysis of the delivery of feedback is limited in the field. There are general recommendations regarding the use of open-ended questions and statements that encourage reflections to facilitate a feedback session, as well as broad models of how to structure a feedback session in clinical education. 8-10 However, feedback delivery has not been analyzed as it relates to Speech Act Theory, which posits that every utterance we speak functions as an action. 11 Speech Act Theory is a theory based in linguistics and phi- losophy and has been the basis of discussing the impact of speech in conversations ranging from doctor-patient com- munication to speeches of major political figures during the Gulf Wars. 12,13 Based on Speech Act Theory, the deliv- ery of feedback could influence its quality, effectiveness, and overall benefit to the learner. As SIMPL becomes more frequently used across surgical training programs, the question of the effectiveness of the dictation feature will become an important one. We aim to compare both the quality and delivery of best-case postop- erative feedback given in-person to that given via SIMPL.

TABLE 1. Procedures Included

SIMPL

In-Person

Anorectal

2 5 7 0 4 1 0

7 3 5 2 0 0 1

Laparoscopic colorectal Open colorectal Open hernia Laparoscopic hernia Laparoscopic biliary

Laparoscopic gastroesophageal

procedures. Procedures included in the study ranged in type and complexity. The breakdown of procedures is shown in Table 1 . Feedback collected was provided by 4 general surgery attending physicians (1 woman, 3 men) to 9 surgery residents (1 woman, 8 men) at the University of Wisconsin School of Medicine and Public Health. Attending physicians who already used SIMPL as part of their workflow were chosen for this study as they did not require training on the application and would have higher SIMPL evaluation completion rates. Surgery residents included in the study were those who were already scheduled on the service of included attending physicians. We requested in-person feedback from attending physicians to residents immediately after cases in the operating room. A researcher was present to audio record this feedback. This feedback was transcribed and divided into individual utterances, the smallest group of spoken words that constitutes a full idea. Dictated feed- back from SIMPL was transcribed and divided into utter- ances. We analyzed a total of 37 instances of feedback, 18 of which were given in person while the remaining 19 were delivered via the SIMPL application. The breakdown of the number of feedback encounters between each surgeon and resident, the resident’s year in training, and the medium of feedback delivery is shown in Table 2 . Data Sources We analyzed the feedback in 3 ways. Quality of feedback was evaluated via survey of the feedback recipient as well as by the study team. Delivery of the feedback was analyzed by the research team. To measure the quality of feedback from the recipi- ent’s perspective, we utilized a previously validated semantic-differential survey of recommended and nonre- commended feedback techniques that addresses recipi- ents’ feelings and perceptions of the feedback they received. 8 The survey has 9 bipolar descriptors of rec- ommended and nonrecommended feedback techniques and is scored on a 5-point rating scale based on how

METHODS Setting and Study Population

This study was reviewed by the University of Wisconsin Institutional Review Board and determined to be exempt. As such, informed consent was not required. We collected operative performance feedback given both in-person and via SIMPL on a variety of surgical

Journal of Surgical Education Volume 00 /Number 00 & 2018

195