Decreasing the Ceiling Effect in Assessing Meeting Quality at an Academic Professional Meeting

Abstract

Background: The psychometric properties of evaluations at academic meetings have not been well studied. Purpose: To explore the ceiling effect in the evaluation of quality of a professional meeting and whether a change in the scale labels would decrease the ceiling effect. Methods: Cross-sectional study at two national meetings (2009-2010), attendees completed the evaluation on paper forms or online (5-point Likert scale). Results: Of 1,064 evaluations, the mean session ratings was higher among respondents to the paper version in 2009 (4.2; 95% confidence interval [CI], 4.1 to 4.3) as compared to online responders in 2009 (3.0; 95% CI, 2.9 to 3.1) or online responders in 2010 (3.0; 95% CI, 2.9 to 3.1)(p < 0.001). Conclusion: A ceiling effect was present in the evaluation of an academic meeting. A change in the evaluation scale labels decreased the ceiling effect and increased evaluation variability.