Assessing the assessment: a comparison of the IPASS and ICCAS to measure change in interprofessional competencies following a large-scale interprofessional forum

ABSTRACT

Measuring the effectiveness of interprofessional education (IPE) experiences is essential but challenging. Surveying learners before (pretest) and after (posttest) an IPE experience may result in minimal change due to response shift bias. Retrospective pretest/posttest assessment may ameliorate response shift bias but may also result in inflated change scores due to social desirability bias. We studied a cohort of 675 students from 12 health professions who completed the Interprofessional Attitudes Scale (IPAS) within the 4 weeks before (pretest) and the 3 weeks after (posttest) an IPE forum and completed the Interprofessional Collaborative Competency Attainment Survey (ICCAS) within the 3 weeks after the forum as a retrospective pretest/posttest. We found higher scores on the pretest IPAS than the retrospective pretest ICCAS and greater change in scores on the ICCAS than the IPAS, indicating potential response shift and social desirability biases. Furthermore, we found few significant correlations between change scores in subscales of the two tests, but a high number of strong and significant correlation among the ICCAS subscale change scores. Our results indicate the timing of pretest administration may impact change scores or that subscales of IPAS may be more unique than those of ICCAS. These findings suggest that educators should consider the potential impact of response shift and social desirability biases when interpreting results of the IPAS and ICCAS in response to an IPE learning experience.

KEYWORDS:

• ICCAS
• IPAS
• interprofessional education
• assessment
• response shift bias
• social desirability bias

Acknowledgments

Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under award number UL1TR001412. The content is solely the responsibility of the authors and does not necessarily represent the official views of NIH.

Declaration of interest statement

The authors declare that there are no conflicts of interest.

Additional information

Funding

This work was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health; [UL1TR001412].

Notes on contributors

Janice Tona

Janice Tona, PhD, OTR is clinical associate professor and director of occupational therapy in the rehabilitation science department of the school of public health and health professions, University at Buffalo. As an occupational therapist and educational psychologist, Dr. Tona serves on the University at Buffalo IPE leadership team, and has research interests in curriculum design and interprofessional education experiences.

Jessica Kruger

Jessica Kruger, PhD CHES is clinical assistant professor in the department of community health and health behavior in the school of public health and health professions, University at  Buffalo. As a health educator, Dr. Kruger serves on the University at Buffalo IPE leadership team, and has research interests in public health pedagogy and health behavior decision-making.

Lisa Jane Jacobsen

Lisa Jane Jacobsen, MD, MPH, MSHPEd is Associate Dean for Curriculum and Associate Professor of Obstetrics & Gynecology at the Jacobs School of Medicine and Biomedical Sciences University at Buffalo. In her position as a Curriculum Dean at the Jacobs School, she works with a talented group of faculty leaders from all of UB’s Health Professions Schools to design interprofessional experiences for medical students.  She is committed to educating our physicians of the future about the necessity of interprofessional collaboration in order to delivery safe, efficient, patient centered healthcare.

Ryan Krzyzanowicz

Ryan Krzyzanowicz, DAT, ATC is a clinical associate professor and director of the master of science in athletic training program in the department of exercise and nutrition sciences in the school of public health and health professions at the University at Buffalo. Dr. Krzyzanowicz serves on the University at Buffalo IPE leadership team and his research interests are interprofessional collaborative practice and manual therapies to improve patient outcomes.