A Quality Checklist for Responsible Conduct of Research (RCR) Education: A proposal to complement the Predictive Modeling Tool

Figures & data

Table

Ref no	Review used for the Quality Checklist
[1]	Alison L. Antes, Stephen T. Murphy, Ethan P. Waples, Michael D. Mumford, Ryan P. Brown, Shane Connelly & Lynn D. Devenport (2009). ‘A Meta-Analysis of Ethics Instruction Effectiveness in the Sciences,’ Ethics & Behavior, 19:5, 379–402, DOI: 10.1080/10508420903035380
[2]	Waples, E.P., Antes, A.L., Murphy, S.T. et al. (2009). ‘A Meta-Analytic Investigation of Business Ethics Instruction,’ J Bus Ethics 87: 133. https://doi-org.proxy.library.uu.nl/10.1007/s10551-008-9875-0
[3]	Antes AL, Wang X, Mumford MD, Brown RP, Connelly S, Devenport LD. (2010). ‘Evaluating the effects that existing instruction on responsible conduct of research has on ethical decision making,’ Acad Med Mar;85(3):519–26. doi: 10.1097/ACM.0b013e3181cd1cc5
[4]	Logan M. Steele, Tyler J. Mulhearn, Kelsey E. Medeiros, Logan L. Watts, Shane Connelly & Michael D. Mumford (2016). ‘How Do We Know What Works? A Review and Critique of Current Practices in Ethics Training Evaluation,’ Accountability in Research, 23:6, 319–350, DOI: 10.1080/08989621.2016.1186547
[5]	Medeiros, K.E., Watts, L.L., Mulhearn, T.J. et al. (2017). ‘What is Working, What is Not, and What We Need to Know: a Meta-Analytic Review of Business Ethics Instruction,’ J Acad Ethics 15: 245. https://doi-org.proxy.library.uu.nl/10.1007/s10805-017-9281-2
[6]	Tyler J. Mulhearn, Logan L. Watts, Brett S. Torrence, E. Michelle Todd, Megan R. Turner, Shane Connelly & Michael D. Mumford (2017). ‘Cross-Field Comparison of Ethics Education: Golden Rules and Particulars,’ Accountability in Research, 24:4, 211–224, DOI: 10.1080/08989621.2016.12749744
[7]	Brett S. Torrence, Logan L. Watts, Tyler J. Mulhearn, Megan R. Turner, E. Michelle Todd, Michael D. Mumford & Shane Connelly (2017). ‘Curricular Approaches in Research Ethics Education: Reflecting on More and Less Effective Practices in Instructional Content,’ Accountability in Research, 24:5, 269–296, DOI: 10.1080/08989621.2016.1276452
[8]	Logan L. Watts, Kelsey E. Medeiros, Tyler J. Mulhearn, Logan M. Steele, Shane Connelly & Michael D. Mumford (2017). ‘Are Ethics Training Programs Improving? A Meta-Analytic Review of Past and Present Ethics Instruction in the Sciences,’ Ethics & Behavior, 27:5, 351–384, DOI: 10.1080/10508422.2016.1182025
[9]	Logan L. Watts, Tyler J. Mulhearn, Kelsey E. Medeiros, Logan M. Steele, Shane Connelly & Michael D. Mumford (2017). ‘Modeling the Instructional Effectiveness of Responsible Conduct of Research Education: A Meta-Analytic Path-Analysis,’ Ethics & Behavior, 27:8, 632–650, DOI: 10.1080/10508422.2016.1247354
[10]	E. Michelle Todd, Brett S. Torrence, Logan L. Watts, Tyler J. Mulhearn, Shane Connelly & Michael D. Mumford (2017). ‘Effective Practices in the Delivery of Research Ethics Education: A Qualitative Review of Instructional Methods,’ Accountability in Research, 24:5, 297–321, DOI: 10.1080/08989621.2017.1301210
[11]	Megan R. Turner, Logan L. Watts, Logan M. Steele, Tyler J. Mulhearn, Brett S. Torrence, E. Michelle Todd, Michael D. Mumford & Shane Connelly (2018). ‘How Did You Like This Course? The Advantages and Limitations of Reaction Criteria in Ethics Education,’ Ethics & Behavior, 28:6, 483–496, DOI: 10.1080/10508422.2017.1308193

Figure 1. Intro.

Figure 2. Quality effectiveness and beyond.

Table

Text box 1: additional quality criteria One size does not fit all. While several reviews mention at least some themes that seem valid across contexts, most reviews also stress limitations with regard to the generalizability of their conclusions ([3], [4], [6]-[10]) Goals/Objectives come first; assessment should follow. Without clear delineation of the desired outcomes, measurement and interpretations of the outcomes regarding course effectiveness are not possible ([3]). Also, the criterion type that is used to evaluate education must match the intended learning outcomes, Otherwise, important results from education might go unnoticed ([1]-[3], [5], [8], [11]) Key is what we think participants should learn. Not that we might achieve more by targeting others. There might be trade-offs between what would be the most effective thing to do, and what are considered to be important pedagogical aims for specific audiences. For example, while older participants may generally benefit more than younger participants, it might be particularly important to include younger participants ([8]) RCR education should take the specific needs of participants into account. This follows from the efficacy of RCR education depending, at least in part, on participant characteristics ([5]) More and less effective do not equal “good” and “bad”. “Negative” course characteristics, i.e., characteristics that have a negative correlation with the overall efficacy of a course, may be essential components of RCR education. Likewise, practical constraints (e.g., time, money) may “necessitate” using approaches known to be less effective ([9]) Specify the concept of effectiveness. The effectiveness of RCR education can be defined as “the extent to which observable improvements are made by trainees with regard to ethics-related knowledge, skills, or attitudes” as a basis for RCR. ([11]) Many meta reviews, however, use an aggregate or composite measure of effectiveness. When analyzed separately, different assessment criteria may show different patterns of effects (e.g., [6]-[11])

Figure 3. Main overview.

Figure 4. Impact on learning aims.

Figure 5. Main lessons.