![Sample our Education journals, sign in here to start your access, latest two full volumes FREE to you for 14 days]({"type":"image" , "src" : "/sda/5932/TOC-Subject-Sample-2021-Education.jpg"})
NEGEA 2015 CONFERENCE ABSTRACT (EDITED)
Measuring an Organization's Culture of Feedback: Can It Be Done?
Steven Rougas and Brian Clyne
Construct: This study sought to develop a construct for measuring formative feedback culture in an academic emergency medicine department. Four archetypes (Market, Adhocracy, Clan, Hierarchy) reflecting an organization's values with respect to focus (internal vs. external) and process (flexibility vs. stability and control) were used to characterize one department's receptiveness to formative feedback. The prevalence of residents' identification with certain archetypes served as an indicator of the department's organizational feedback culture. Background: New regulations have forced academic institutions to implement wide-ranging changes to accommodate competency-based milestones and their assessment. These changes challenge residencies that use formative feedback from faculty as a major source of data for determining training advancement. Though various approaches have been taken to improve formative feedback to residents, there currently exists no tool to objectively measure the organizational culture that surrounds this process. Assessing organizational culture, commonly used in the business sector to represent organizational health, may help residency directors gauge their program's success in fostering formative feedback. The Organizational Culture Assessment Instrument (OCAI) is widely used, extensively validated, applicable to survey research, and theoretically based and may be modifiable to assess formative feedback culture in the emergency department. Approach: Using a modified Delphi technique and several iterations of focus groups amongst educators at one institution, four of the original six OCAI domains (which each contain 4 possible responses) were modified to create a 16-item Formative Feedback Culture Tool (FFCT) that was administered to 26 residents (response rate = 55%) at a single academic emergency medicine department. The mean score of each item on the FFCT (range = 0–100) was analyzed. Convergent and divergent properties of the four archetypes were assessed using a multitrait–multimethod matrix of Pearson's coefficients. Expecting that items in one archetype would diverge from the others, whereas items within an archetype should have strong convergent properties, convergent validity was assessed by comparing items across domains that all related to the same archetype. Similarly, divergent validity was assessed by comparing the correlation of items within an archetype to the correlations of those items within a hetero-domain block (i.e., to other items within the same domain). Results: Three of the four domains of the FFCT (Overall Departmental Characteristics 35.4 ± 15.4, Departmental Foundation of Feedback 46.1 ± 16.7, and Departmental Emphasis of Feedback 30.3 ± 17.7) had the highest mean in the Market archetype (results/achievement oriented), whereas the final domain (Departmental Definition of Successful Feedback 34.8 ± 22.1) had the highest mean in the Clan archetype (personal growth/team achievement). Item responses in the Clan and Hierarchy archetypes had the strongest convergent and divergent validity, respectively. Item responses in the Adhocracy archetype had the weakest convergent and divergent validity. Conclusions: Although the sample size was small, this initial study demonstrates that a modified organizational culture assessment tool can feasibly be utilized to identify the primary formative feedback archetype of a cohort of residents. This may have future implications for measuring changes in culture after the implementation of strategic programs to address formative feedback. Future studies should examine the generalizability of the FFCT to other institutions, as well as address the weak validity evidence of the Adhocracy archetype in the FFCT.
Keywords:
Additional information
Notes on contributors
Anna T. Cianciolo
Anna T. Cianciolo, Ph.D., is Assistant Professor, Department of Medical Education, at Southern Illinois University School of Medicine and Editor of Teaching and Learning in Medicine. Her interest is focused on exploring the influence of context (interpersonal and environmental) on the nature of learning and instruction. Areas of emphasis include small-group collaborative instruction, diagnostic strategy, and professional self-development.
Teresa M. Chan
Teresa M. Chan, M.D., is Assistant Professor, Division of Emergency Medicine, within the Department of Medicine at McMaster University. She also is Assessment Subcommittee Chairperson for the Emergency Medicine Residency Program, Continuing Professional Development of the Division of Emergency Medicine. Nationally, she is one of the inaugural Social Media editors for the Canadian Journal of Emergency Medicine. Her interests include competency-based workplace-based assessment, understanding and describing the emergency physician experience, and online medical education.
Jonathan Sherbino
Jonathan Sherbino, M.D., Med, FRCPC, FAcadMEd, is an associate professor in the Division of Emergency Medicine, McMaster University. He is a senior Clinician Educator with the Royal College, coediting the CanMEDS 2015 framework and cochairing the International Conference on Residency Education, among other projects. He is an adjunct scientist with Program for Education Research and Development, McMaster University, and his education research focuses on clinical decision making.
Lalena M. Yarris
Lalena M. Yarris, M.D., MCR, is Associate Professor, Department of Emergency Medicine, at Oregon Health & Science University. She is Co-Director of the Education Section, Director of the Emergency Medicine Residency Program, and Co-Director of an Emergency Medicine Education Fellowship. She is Deputy Editor for the Journal of Graduate Medical Education and Associate Editor for Academic Emergency Medicine. Her interests include promoting education research in academic emergency medicine, feedback in medical education, teaching and assessing clinical reasoning skills, and teaching and evaluating professionalism and interpersonal and communication skills.