How Do Accredited Organizations Evaluate the Quality and Effectiveness of Their Human Research Protection Programs?

Abstract

Background

Meaningfully evaluating the quality of institutional review boards (IRBs) and human research protection programs (HRPPs) is a long-recognized challenge. To be accredited by the Association for the Accreditation of Human Research Protection Programs (AAHRPP), organizations must demonstrate that they measure and improve HRPP “quality, effectiveness, and efficiency” (QEE). We sought to learn how AAHRPP-accredited organizations interpret and satisfy this standard, in order to assess strengths, weaknesses, and gaps in current approaches and to inform recommendations for improvement.

Methods

We conducted 3 small-group interviews with a total of 19 participant representatives of accredited organizations at the 2019 AAHRPP annual meeting. Participants were eligible if they had familiarity with their organization’s approach to satisfying the relevant QEE standard.

Results

Participants reported lacking clear definitions for HRPP quality or effectiveness but described various approaches to assessing QEE, typically focused on turnaround time, compliance, and researcher satisfaction. Evaluation of IRB members was described as relatively superficial and information regarding research subject experience was not reported as central to QEE assessment, although participants described several efforts to improve consideration of patient, subject, and community perspectives in IRB review. Participants also described efforts to educate and build relationships with key stakeholders as important features of a high-quality HRPP. While generally satisfied with their approaches, participants expressed concern about resource and time constraints that pushed them to be reactive and automatic about QEE, rather than proactive and critical.

Conclusions

The relevant AAHRPP accreditation standard may obscure critical gaps in defining and measuring QEE elements. We recommend that AAHRPP: (1) offer a definition of QEE or require accredited organizations to provide their own, to help clarify the rationale and goals behind assessment and improvement efforts, and (2) require accredited organizations to establish QEE objectives and measures focused on participant outcomes and deliberative quality during protocol review.

Keywords:

• IRB
• human research protection program
• accreditation
• quality
• research ethics

Acknowledgments

We thank AAHRPP leaders Elyse Summers, Michelle Feige, and Lori Kravchick for allowing us to conduct small group interviews at the 2019 AAHRPP annual conference and facilitating invitations to registered participants. We also acknowledge the contributions of these individuals who agreed to participate in our interviews.

Disclosure statement

Holly Fernandez Lynch and Holly A. Taylor are co-chairs of the Consortium to Advance Effective Research Ethics Oversight (AEREO). Members of AAHRPP’s leadership team are also members of AEREO, although their membership post-dates the interviews described in this article. There is no financial relationship between AEREO and AAHRPP. Holly Fernandez Lynch receives funding from the Greenwall Foundation as a Faculty Scholar. She is an unpaid board member of Public Responsibility in Medicine & Research (PRIM&R). She is a paid ethics consultant to the Robert Wood Johnson Foundation. In 2019, she received payments from a law firm for consulting regarding a whistleblower complaint alleging misconduct in research oversight at a public university. Holly Taylor’s effort is funded by the Intramural Program, Clinical Center, National Institutes of Health (NIH). The views expressed by Dr. Taylor do not represent the position or policy of the NIH, the Department of Health and Human Services, or the U.S. government.