The 9-criteria evaluation framework for perceptions survey: the case of VA’s Learners’ Perceptions Survey

ABSTRACT

For its clinical, epidemiologic, educational, and health services research, evaluation, administrative, regulatory, and accreditation purposes, the perceptions survey is a data collection tool that asks observers to describe perceptions of their experiences with a defined phenomenon of interest. In practice, these surveys are often subject to criticism for not having been thoroughly evaluated before its first application using a consistent and comprehensive set of criteria for validity and reliability. This paper introduces a 9-criteria framework to assess perceptions surveys that integrates criteria from multiple evaluation sources. The 9-criteria framework was applied to data from the Department of Veterans Affairs’ Learners’ Perceptions Survey (LPS) that had been administered to national and local samples, and from findings obtained through a literature review involving LPS survey data. We show that the LPS is a robust tool that may serve as a model for design and validation of other perceptions surveys. Findings underscore the importance of using all nine criteria to validate perceptions survey data.

KEYWORDS:

• Education
• psychometrics
• perceptions survey
• evaluation

Acknowledgements

The authors wish to express their appreciation to Tracy McKay, M.S.H.I., B.S., Elena V. Perez, B.S.H.A., and Erwin D. Velasco, B.S.H.C.M. with the Center for Advanced Statistics in Education at the Loma Linda VA Medical Center, Loma Linda, California; Tagbo Arene, M.D., M.P.H., formerly PGY4 resident with the Department of Psychiatry at Loma Linda University School of Medicine and currently Staff Physician, Department of Behavioral Health, County of San Bernardino, California; Department of Veterans Affairs Office of Academic Affiliations Data Management and Support Center in St. Louis MO (Christopher T. Clarke, Ph.D., David Bernett, BA, Terry V. Kruzan, George E. McKay (retired), and Laura Stefanowycz).

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This study was supported in part by VA Health Services Research and Development [grant numbers SDR#13-426, IIR#14-071, and IIR#15-084] and the Office of Academic Affiliations, Department of Veterans Affairs. All statements and descriptions expressed herein do not necessarily reflect the opinions or positions of the Department of Veterans Affairs or its affiliated institutions.

Notes on contributors

T. Michael Kashner

T. Michael Kashner Ph.D., J.D., is Health Science Specialist with the Office of Academic Affiliations, Department of Veterans Affairs, Washington, DC, Research Professor of Medicine, Loma Linda University Medical School, Loma Linda, CA. Email: [email protected]

Christopher T. Clarke

Christopher T. Clarke Ph.D., F.A.C.H.E., is Chief Finance and Informatics Officer with the Office of Academic Affiliations, Department of Veterans Affairs, Washington, DC. Email: [email protected]

David C. Aron

David C. Aron M.D., M.S., Director of Clinical Program Research and Education, VA Senior Scholar, Louis Stokes Cleveland DVA Medical Center, Cleveland, OH, and Professor of Medicine & Epidemiology & Biostatistics, School of Medicine, and Adjunct Professor of Organizational Behavior at the Weatherhead School of Management, Case Western Reserve University, Cleveland, OH. Email: [email protected]

John M. Byrne

John M. Byrne D.O., is Associate Chief of Staff for Education at the Jerry L. Pettis Memorial VA Medical Center, Loma Linda, CA, and Associate Professor of Medicine, Loma Linda University Medical School, Loma Linda, CA. Email: [email protected]

Grant W. Cannon

Grant W. Cannon M.D., is Associate Chief of Staff for Academic Affiliations and Regional Director of the Specialty Care Center of Innovation at the George E. Wahlen VA Medical Center, Salt Lake City, Utah, and Thomas E. and Rebecca D. Jeremy Presidential and Endowed Chair for Arthritis Research, School of Medicine, University of Utah, Salt Lake City, UT. Email: [email protected]

David A. Deemer

David A. Deemer B.S., is a medical student and MacPherson Society Research Program Fellow, Loma Linda University School of Medicine, Loma Linda, CA. Email: [email protected]

Stuart C. Gilman

Stuart C. Gilman M.D., M.P.H., is Director of Advanced Fellowships with the Office of Academic Affiliations, Department of Veterans Affairs, Washington, D.C., and Clinical Professor of Health Science, University of California at Irvine School of Medicine, Irvine, CA. Email: [email protected]

Catherine P. Kaminetzky

Catherine P. Kaminetzky M.D., M.P.H., is Chief of Staff, VA Puget Sound Health Care System, Seattle, WA, and Associate Professor of Medicine, University of Washington School of Medicine, Seattle, WA. Email: [email protected]

Lawrence K. Loo

Lawrence K. Loo MD, MACP, is Staff Physician, VA Loma Linda Healthcare System, and Associate Dean for Education Quality and Outcomes, and Professor of Medicine, Department of Medicine and Department of Medical Education, Loma Linda University School of Medicine, Loma Linda, CA. Email: [email protected], [email protected]

Sophia Li

Sophia S. Li M.D., is a Staff Physician with the Jerry L. Pettis Memorial VA Medical Center, Loma Linda, CA, and Assistant Professor of Medicine, Loma Linda University Medical School, Loma Linda, CA. Email: [email protected]

Annie B. Wicker

Annie B. Wicker B.S., is a Health Science Specialist, Office of Academic Affiliations, Department of Veterans Affairs, Washington, DC. Email: [email protected]

Sheri A. Keitz

Sheri A. Keitz M.D., Ph.D., is Chief, General Internal Medicine and Professor and Vice Chair for Clinical Affairs, Department of Medicine, UMass-Memorial Medical Center at the University of Massachusetts, Worcester, MA. Email: [email protected]