Evaluating the long-term impact of faculty development programs on MCQ item analysis

Abstract

Purpose: Evaluating the long-term impact of faculty development programs (FDPs) can help monitor the effectiveness of the program and identify areas for development. This study examined long-term differences in confidence, knowledge, behaviors, and policies of faculty members who attended FDPs on multiple choice question (MCQ) item analysis and faculty members who did not attend the FDPs.

Methods: A cross-sectional study design was used, by administering a 24-item survey to a representative sample (simple random selection) of 61 faculty members at King Abdulaziz University Faculty of Medicine.

Results: Among respondents, 34% did not attend FDPs; 53% attended 1–3 FDPs; and 13% attended more than 3 FDPs on MCQ item analysis. Results showed that faculty knowledge on elements of MCQ item analysis was significantly greater (p = .01) for members who attended the FDPs. Faculty who attended FDPs on MCQ item analysis were twice more likely to conduct item analysis in general (p = .020) and four times more likely to conduct item analysis for more than 70% of module examinations (p = .005).

Conclusion: FDPs focused on MCQ item analysis can yield systematic changes on faculty confidence, knowledge, and behaviors. Moreover, FDPs also need support from the department and need sustained strategic support to ensure continued effectiveness.

Acknowledgements

The authors thank members of the Medical Education Department at KAU, Nasra Ayoub, and Reda Jamjoom for supplying the relevant information related to MCQ item analysis training sessions provided by the department.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article..

Funding

The publication of this supplement has been made possible with the generous financial support of the Dr. Hamza Alkholi Chair for Developing Medical Education in KSA.

Notes on contributors

Aliaa Amr Alamoudi, MD, PhD, is an Assistant Professor in the Department of Clinical Biochemistry at King Abulaziz University (KAU). Dr. Alamoudi conceptualized and designed the study, acquired and interpreted the data, drafted the initial manuscript, revised the manuscript, and approved the final manuscript.

Basem Salama El-Deek, MD, PhD, is Professor in Community Medicine and in Medical Education departments in the Faculty of Medicine at KAU. Dr. El-Deek contributed to the conceptualization and revision of the manuscript.

Yoon Soo Park, PhD, is Assistant Professor in the Department of Medical Education at the University of Illinois College of Medicine at Chicago. Dr. Park contributed to the conceptualization and revision of the manuscript.

Lana Adey Al Shawwa, PhD, is Associate Professor in the Department of Medical Education in the Faculty of Medicine at KAU. Dr. Al Shawwa contributed to the conceptualization and revision of the manuscript.

Ara Tekian, PhD, MHPE, is Professor in the Department of Medical Education at the University of Illinois College of Medicine at Chicago. Dr. Tekian contributed to the conceptualization and revision of the manuscript.