Implementation of a responsible conduct of research education program at Duke University School of Medicine

ABSTRACT

Academic medical centers rarely require all of their research faculty and staff to participate in educational programs on the responsible conduct of research (RCR). There is also little published evidence of RCR programs addressing high-profile, internal cases of misconduct as a way of promoting deliberation and learning. In the wake of major research misconduct, Duke University School of Medicine (DUSoM) expanded its RCR education activities to include all DUSoM faculty and staff engaged in research. The program included formal deliberation of the Translational Omics misconduct case, which occurred at Duke. Over 5,000 DUSoM faculty and staff participated in the first phase of this new program, with a 100% completion rate. The article reports on the program’s development, challenges and successes, and future directions. This experience at Duke University illustrates that, although challenging and resource intensive, engagement with RCR activities can be integrated into programs for all research faculty and staff. Formal, participatory deliberation of recent cases of internal misconduct can add a novel dimension of reflection and openness to RCR educational activities.

KEYWORDS:

• Responsible conduct of research
• research integrity
• RCR education
• ethics education

Introduction

Despite remarkable accomplishments, medical science is not as consistently safe, robust, reproducible, and ethical as it should be (Freckelton 2016; Wells and Farthing 2008). There is growing evidence of gaps in research integrity and compliance, due in part to detrimental research practices (DRPs) and research misconduct (fabrication, falsification, or plagiarism) (Banks et al. 2016; John, Loewenstein, and Prelec 2012; Alberts et al. 2015; Fanelli 2009; Titus, Wells, and Rhoades 2008; NAS 2017). Many factors may contribute to ethical transgressions in research. These include individual factors such as an exaggerated sense of entitlement or poor decision-making; institutional factors such as gaps in research oversight and responsiveness to wrongdoing; shortcomings in publication and peer review practices; and, professional stressors such as the pressure to publish, competition for research funding, and efforts to commercialize research (Davis, Riske-Morris, and Diaz 2007; DeMets et al. 2017; Edwards and Roy 2017; Boughton et al. 2018; Finkel 2019).

In medicine, research wrongdoing is particularly worrisome because it can make medical treatment and clinical research unreliable and unsafe, and involve heavy financial losses in research funding and associated legal costs (Davidson 2017; George 2016). According to one study, research fraud and retracted publications have exposed thousands of patients to treatments of questionable efficacy (Steen 2011). Significant public funds can be lost when medical research is halted or abandoned as a result of DRPs or misconduct. Investigation of a single case of research misconduct has been estimated to cost over $500,000, a figure that does not take into account possible additional legal costs and damages (Michalek et al. 2010). In addition, the average cost of the research lost to the community when a publication is retracted is $400,000 (Stern et al. 2014). Research wrongdoing can also undermine or even end the careers of innocent, often junior, scientists. It can hurt the reputations of institutions, and the public’s trust and willingness to accept the recommendations of the scientific enterprise (for example, Eassom 2016).

Different strategies have been developed to address gaps and shortcomings in common research practices and research integrity. These include education in the responsible conduct of research (RCR) (for example, Bulger and Heitman 2007; Zigmond and Fischer 2014), as well as initiatives to improve research oversight, data management and transparency, and the functioning of research environments (for example, the 2018 Nature series on “How to grow a healthy lab”). There are also efforts to promote more robust investigation of suspected misconduct (Gunsalus, Marcus, and Oransky 2018), as well as improved policies and protections for whistleblowers who report suspected research wrongdoing (Bouter and Hendrix 2017). The need for consistent and tougher penalties for misconduct is also being debated (Galbraith 2017). There is also some discussion regarding the reconceptualization of “misconduct” to include harassment, professional sabotage, inappropriate use of statistics, or failure to disclose significant conflicts of interest (Resnik 2019; Marin-Spiotta 2018).

Although RCR education is increasingly utilized in efforts to improve the ethical and responsible conduct of research (Antes and DuBois 2014), academic RCR education programs rarely reach all faculty and staff in a given organization. According to a recent survey of the 144 top-funded U.S. research institutions with RCR education programs, less than 10% report that all externally funded faculty and staff are required to complete RCR education (Resnik and Dinse 2012). Federal research agencies require RCR education of some funding recipients, such as graduate students, postdoctoral fellows, and junior faculty, but these requirements do not extend to all investigators (Resnik and Dinse 2012; NIH 2009; NSF 2009). This lack of uniformity exists despite evidence that researchers at all academic ranks engage in research wrongdoing, including mid- and senior-level researchers (Fang, Bennett, and Casadevall 2013).

Given this context, there may be a need for more comprehensive and inclusive (i.e., large-scale) RCR education programs. At the same time, large-scale RCR education programs may be incrementally more costly and complex to implement when compared to smaller and more targeted RCR education programs. This paper reports on a new, large-scale RCR education program at Duke University, started in response to concerns over research wrongdoing and a perceived need for improvements in research integrity. The program has a number of distinctive elements: 1) it is mandatory for all research faculty and staff, 2) it provides several different educational options in an effort to increase program efficiency, participation, and acceptance, and, 3) it includes formal deliberation of a recent high-profile case of research misconduct at the institution.

This report describes how this program was implemented, what challenges and interim successes were identified, and how the program is currently evolving. Literature on RCR program development is limited (for example, Mumford, Steele, and Watts 2015; Olson 2014; Appendix C in NAS 2017), and this report may provide some guidance for other institutions and institutional leaders considering large-scale RCR education programs. The report also addresses the possibility of including deliberation of recent misconduct at an institution (i.e., internal misconduct) into core RCR education programs, with the goal of enhancing participation and learning, as well as institutional reflection and openness with respect to past research wrongdoing.

Materials and methods

Background

Duke University is a private academic institution located in Durham, North Carolina, with global ties. At its Durham location, the Duke University School of Medicine (DUSoM) includes the research efforts of basic and clinical faculty members and staff in 39 departments, centers, institutes, and initiatives. Their combined efforts make Duke one of the largest biomedical research enterprises in the country, with nearly $757 million in sponsored research expenditures annually (DUSoM 2018).

Duke University has developed a tradition of providing RCR education to all postdoctoral fellows, and students in research-oriented programs. For example, students in the masters-level Clinical Research Training Program complete a semester-long course in RCR, focused on the ethics of research design, confidentiality, relationships with industry, publication and authorship, DRPs and misconduct, conflict of interest, intellectual property and technology transfer, international health research, and social and ethical implications of genetic technologies and research.

All incoming professional masters and Ph.D. students begin their research careers with in-person RCR orientation sessions where a range of RCR issues are addressed. These sessions are less about the teaching of “right answers” than critical awareness of ethical issues encountered in research. RCR orientation sessions are a requirement for graduation and include several hours of in-person presentations and discussions. For Ph.D. students, initial RCR orientation is followed by topic-specific RCR education sessions that they complete intermittently throughout their doctoral training.

Similarly, all Duke postdoctoral researchers are required to attend an 8-hour Postdoctoral RCR Orientation program (organized once per year) or an RCR Short Course (a series of five interactive sessions of 1.75 hours each) within their first year. Postdocs must also participate in ongoing, topic-specific RCR forums organized by the Graduate School; they are also encouraged to engage with their mentors in informal discussions about topics in RCR.

Similar to other academic research institutions complying with federal funding guidelines (Resnik and Dinse 2012; NIH 2009; NSF 2009), Duke University had not previously required formal RCR education for all research staff and regular-rank faculty. However, in the wake of the Duke Translational Omics misconduct case and its impact (see below), DUSoM leadership made the decision to extend mandatory RCR education to all DUSoM faculty and staff engaged in research. More recently, mandatory RCR education has been extended to all University research faculty and staff. The new DUSoM RCR education program is therefore the initial step in the larger effort to bring mandatory and ongoing RCR education to all research faculty and staff at Duke University, with the goal of improving the quality, practice, and culture of research throughout the institution.

At the center of this coordinated effort is the newly created Duke Office of Scientific Integrity (DOSI). DOSI is comprised of several programs designed to help Duke researchers navigate the increasingly complex environment of research ethics, regulatory compliance, multi-institutional and collaborative research programs, and rapidly evolving technologies. The office manages reporting of financial conflicts of interest, misconduct review, and quality monitoring programs, as well as the program in Advancing Scientific Integrity, Services and Training (DOSI-ASIST). DOSI-ASIST implements tools and educational programming to improve data management practices and promote a culture of integrity, and is the administrative home of the new RCR education program at Duke. Co-location of these educational, compliance, and accountability functions in DOSI raises the visibility and prominence of research integrity at Duke, and provides its researchers with a central resource for supporting research integrity and RCR.

Development of the RCR education program

The new RCR education program in the DUSoM is coordinated by the DOSI-ASIST program in collaboration with the Trent Center for Bioethics, Humanities & History of Medicine. In September 2017, DUSoM leadership outlined the primary goal for a new RCR initiative: to develop and implement a comprehensive and mandatory RCR education program for all SoM faculty and staff engaged in research, in which participants would complete the initial phase of education by the end of June 2018. The broader, long-term goal of the new RCR education program is to bring together faculty and staff in ongoing discussion, dialogue, and learning with respect to research-related regulations, policies, challenges, and best practices, and to advocate awareness and appropriate use of resources and formal support for improving the quality and integrity of research at Duke. A core group of faculty and staff, including representatives from the basic and clinical sciences, research administration/compliance, research integrity, and the Trent Center for Bioethics, Humanities & History of Medicine were assembled to form the DOSI-ASIST Task Force, which was tasked with developing the RCR education program and content. Inclusion of the Duke Translational Omics case in the program grew out of this development process.

Integrating an internal case of misconduct

Duke has been impacted by two high-profile research misconduct cases: the Translational Omics case and a more recent case that resulted in a $112.5 million false claims settlement. Both cases involved research misconduct that occurred in DUSoM research environments, beginning in the mid-2000s (Appendix D in NAS 2017; Science News Staff 2019). Owing to legal proceedings, the more recent case is not yet included as a case study in the RCR education program at Duke (although conversations about including it are taking place).

The Translational Omics case centers on the actions of Anil Potti, a physician-researcher found to have included fabricated and falsified research data in grants and publications. The case has generated a significant body of official reports and literature (ORI Case Summary; NAS 2017; IOM 2012; Freckelton 2016). At planning meetings in 2017, RCR program leaders, who included teaching faculty, advocated for the inclusion of the Translational Omics case as a case study in the RCR education program to increase the legitimacy and acceptability of the program, and to build trust in the program and institution overall.

Program compliance

A key objective of the new program was to ensure that all eligible (defined below) DUSoM research faculty and staff participated in the program in its first, eight-month, phase of operation. To encourage participation in the program, prospective participants could choose to complete either an online RCR education course, an in-person workshop, or a self-assessment quiz (for details on these options, see “Program components,” below). In addition, if they could provide adequate documentation that they had completed an RCR education program at Duke within the previous two years, participants could opt out of the first phase of the program (but not subsequent phases, see below).

To ensure compliance within individual DUSoM units, unit leaders (e.g., department chairs and center/institute directors) were asked to document that their eligible research faculty and staff participated in the education program. Completion of RCR education in this first phase was tied to individual units’ annual performance metrics. Departmental leaders and administrators communicated the importance of participating in this new program via multiple announcements, meetings, emails, and discussions with staff and faculty. Although not a mandated message, some departments indicated to faculty and staff that failure to complete the required RCR education could result in a suspension of research privileges.

Program components

In November 2017, when the DUSoM faculty and staff RCR education program was launched, it was comprised of three educational options:

1) Self-assessment quiz

A “self-assessment quiz” was developed with the recognition that some proportion of DUSoM researchers may be sufficiently knowledgeable about RCR to warrant an exemption from either the online RCR course or RCR workshops (but not future, ongoing RCR education requirements, see “Continuing education,” below). A researcher may have acquired this knowledge through, for example, prior experiences, mentorship, or through formal RCR education teachings. Because an exemption was considered a significant step, the quiz was intentionally designed to test more than just the most basic knowledge of RCR. Therefore, the test included questions aimed at probing understanding, and not just knowledge (i.e., recall) of specific facts and details.

The quiz was structured, to test understanding of core RCR principles and RCR-related standards (NIH 2009). A bioethicist (Simon) wrote the quiz with support from DOSI. The quiz was tested for usability and relevance over three months with input from a variety of faculty and staff. One limitation of the quiz was that it tested knowledge and understanding of misconduct in a global way and not with specific review of, or reference to, the Translational Omics case, a complex case about which general understanding could not be fairly assumed. The quiz did ask questions more generally about the nature of research misconduct, DRPs, and resources available at Duke for bringing forth issues of concern. Based on feedback, separate tests were developed for faculty and staff. Forty-two questions were generated for faculty and 30 questions for staff. To be exempted from the other education options, individuals needed to achieve 90% or higher on their first test attempt.

The quiz was administered electronically (via the Duke Health Learning Management System (LMS), Saba). The records of course completion for the self-assessment quiz administered through the LMS were stored separately from the quiz analytics reports (containing individual quiz scores and individual learner responses to each question). Although the quiz completion data remained intact (data cited in Figure 1), an unexplained error caused a sizable loss of the quiz analytics data within the LMS between February and March of 2018 (4–5 weeks). Additionally, although the quiz was designed to allow only one attempt to achieve a 90% score or higher, an unexplained technical loophole within the LMS allowed a small number of participants to take the test twice (~60 known cases out of over 2700 quizzes listed in the partial analytics report). Consequently, we can report the number of required persons who completed the RCR education program through the self-assessment quiz (Figure 1). However, we can only estimate the quiz pass/fail rate based on the partial analytics data available (see “Program participation” below).

Figure 1. Participation metrics for phase I faculty and staff RCR education program. A) Workflow showing identification, curation, and training as part of Duke faculty and staff RCR education program. * reflects one course completion/required person. If persons completed more than one training, the subsequent trainings are not reported here. Additionally, the total participation in the in-person workshops (433), is not reported here as there were 82 participants who attended, but were not required to complete RCR training at the time of the trainings. A 30-day grace period was allowed for final completions to be reported. B) Faculty/staff composition of 5125 persons identified as required to complete RCR. C) Overall course completion by faculty/staff designation (%).

2) Online course

The DUSoM contracted with the Collaborative Institutional Training Initiative (CITI) to provide research faculty and staff with an online RCR course (CITI). This course is divided into topic-specific modules, each containing comprehensive text, short case studies, and a multiple-choice quiz to test the reader’s understanding of the material. Two DUSoM-specific modifications were made to the CITI RCR course: the addition of a summary of the Translational Omics case and the separation of modules into Required and Elective categories (Table 1). Separation of the modules in this fashion was aimed at reducing the time needed to complete the online course and improving alignment of the CITI RCR course content with the diversity of disciplines, roles, and responsibilities represented in the DUSoM faculty and staff population.

Table 1. Duke customized CITI RCR course.

REQUIRED MODULES (All required)	ELECTIVE MODULES (Two required, participant’s choice)
Introduction to RCR	Using Animals in Research
Authorship	Export Controls and National Security
Collaborative Research	Research, Ethics and Society
Conflicts of Interest	Environmental and Social Dimensions
Data Management	Research Involving Human Subjects
Financial Responsibility	Mentoring
Peer Review
Plagiarism
Research Misconduct
Duke Research Misconduct Case Study (Translational Omics Case)

3) In-person RCR workshops

Eligible individuals could also opt for an interactive, in-person RCR workshop in order to complete the training requirement for the first, eight-month, phase of the new program. In contrast to the CITI RCR course and self-assessment quiz, the RCR workshops were designed to bring together faculty and staff for interactive dialogue and learning about RCR issues and topics. A series of five 2.5 hour workshops were held between February and June 2018, with a capacity of 100 persons per workshop. Five workshops were conducted given available resources and time constraints. To facilitate interaction, registration was capped at 100 participants.

The first half of each workshop involved short presentations on RCR topics, with an emphasis on the prevalence and impact of research misconduct and DRPs, and strategies for minimizing or addressing the potential for wrongdoing in research. Presentations also focused, where appropriate, on ethical principles and concepts for advancing research integrity and reproducibility of results, as well as avoidance of bias and conflicts of interest in research.

In the second half of each workshop, participants worked in small (3–8 person) groups to familiarize themselves with details of the Translational Omics case, and discuss the case. Discussions lasted 70 minutes on average, during which time the small groups were asked to read a written summary of the case, discuss the facts of the case, and then summarize their discussions for the benefit of the full workshop audience. This process was led by the workshop moderator (Simon) and 3–4 facilitators.

Participant feedback on the content, structure, and perceived usefulness of the workshops was collected using an anonymous, brief post-workshop survey that was administered in paper form or electronically.

Identification of eligible participants

The DUSoM defined the group of potential RCR education program participants as all faculty and staff contributing to the design, conduct or reporting of research, following the definition for researchers provided by the NIH’s Policies and Procedures for Promoting Scientific Integrity (NIH 2012). However, there was no single data source available for identifying every Duke faculty and staff member engaged in research. Consequently, an algorithm utilizing multiple databases was developed to help identify faculty and staff members engaged in research. The algorithm excluded students, post-doctoral fellows, and administrative personnel in non-research positions. Duke graduate students and postdocs have had a longstanding requirement to participate in separate RCR education programs designed specifically for their career stages (see above), and were therefore excluded from consideration.

The algorithm was used to select persons present in any one of five databases associated with research from primary organizational units within the DUSoM (Table 2). The resulting list contained some false positives (i.e., individuals not actually engaged in research) and some false negatives (i.e., individuals absent from the list who were engaged in research). Therefore, at least one point-of-contact was elected from each department to review and further refine the initial algorithm-generated list. The individuals identified in the curated list were required to complete education as part of the new DUSoM faculty and staff RCR education program.

Table 2. Algorithm to identify DUSoM persons engaged in research.

Databases used for identification
Listed as key personnel on an active or pending project in the Duke sponsored project system (SPS; the primary data source for reporting on sponsored project proposals and awards, both internally and externally to federal and non-profit agencies)
Listed as personnel on an active or pending Institutional Review Board protocol (IRB)
Listed as personnel on an active or pending Institutional Animal Care and Use Committee protocol (IACUC)
University salary supported by research funds
Designated as a room occupant in laboratory or office space with a research function

A total of 6,148 faculty and staff in the DUSoM were initially identified as eligible for RCR education using the algorithm described above (Table 2 and Figure 1(a)). Once the list of eligible individuals was refined and individuals were added or removed when appropriate (e.g., a person was designated in research space, but was not actually engaged in research), 5,125 individuals (35% of whom were DUSoM faculty and 65% of whom were DUSoM staff) were identified as engaged in research and required to participate in the first eight-month phase of the RCR education program (Figure 1(b)).

Program evaluation

In-Person workshops were evaluated using a brief survey sent out electronically via email to all attendees within a few hours after each workshop (or the day after the workshop in the case of one late afternoon workshop), with no deadline for a return date specified. The average survey response rate from attendees across all workshops was 40%. Because of time and feasibility constraints, individuals who completed the self-assessment quiz or online course did not receive analogous evaluation surveys. Some unsolicited feedback on the quiz and online course was collected via email and in-person conversations with DOSI-ASIST; these are also summarized in the Results.

Results

Program participation

All 5,125 research faculty and staff (100%) identified as eligible for the first phase of the new RCR education program completed the program requirements within its allotted eight-month time frame. The most widely utilized option was the RCR self-assessment quiz, followed by the online course, and the in-person RCR workshop (Figure 1(a)). A small number of individuals were given credit for having already completed a Duke program on an RCR topic within the last two years (Figure 1(a)). A higher percentage of faculty than staff opted for and passed the self-assessment quiz (Figure 1(c)). A higher percentage of staff completed the online course and attended the in-person education (Figure 1(c)). Allowing for the data-loss issues uniquely affecting the calculation of the pass/fail rate for the self-assessment quiz (see Materials and Methods section above), it is estimated that the fail-rate for the faculty version of the quiz was approximately 14% (148/1075) and for the staff version, approximately 29% (586/2015).

Workshop discussions of the Translational Omics case

The Translational Omics case, which occurred at Duke, drew strong interest and discussion in the workshops. The case study, as provided to participants, engendered discussion on lapses in research oversight, conflicts of interest, co-author responsibilities, poor data documentation practices, and errors in judgement, including by DUSoM and institutional leaders, which may have prolonged the case. There was discussion of academic incentives that may have contributed to the wrongdoing, such as the drive to publish prolifically and attract commercial investment in research.

A recurring theme in these discussions was the question of why and how the case was allowed to go on for so long without decisive action on the part of the institution. Participants expressed frustration at leaders who were seen as having failed to act on clear signals that the integrity of the research in question had been compromised. Participants commented on the benefit of hindsight in addressing the Translational Omics case now, and were concerned that similar cases of misconduct could occur in the future, given that warning signals in such cases may not be readily apparent.

Significant discussion focused on system-level factors such as faculty tenure and promotion criteria, the pressure to obtain grants and publish, and the climate of hyper competition in academic research, which participants cited as potential contributing factors for DRPs and misconduct. Participants also expressed frustration at the unrealistic expectation that researchers can maintain an unwavering commitment to ethical coda when the pressures of academic research and job retention may demand moral compromises. For example, one research staff member commented in the context of a discussion on the ethics of research, “Integrity is good and fine, but it doesn’t pay the bills.”

Participants also intensely discussed and commented on the section of the Translational Omics case that detailed the role of a young medical student, also a Howard Hughes scholar at the time, who attempted to alert certain institutional officials to irregularities that he had observed while working for Anil Potti. Many participants drew attention to the heroic efforts of this young medical student. Some wondered why the University had not honored his efforts since that time. The story of this young medical student led to discussion about the importance and pitfalls of reporting misconduct. Participants generally recognized the need to report misconduct in a timely and decisive fashion, but they also questioned the trustworthiness of the institution to take action in response to such reports and to protect reporters of wrongdoing. Participants questioned whether other persons connected to Anil Potti also shared some responsibility for the delayed recognition that data were falsified, such as the lead investigator, the developer of the computational methodologies that were utilized, or other close collaborators and co-authors involved in the research.

The questions developed for guiding discussions of the Translational Omics case also asked workshop participants to consider strategies for preventing or minimizing the potential for wrongdoing in the research environment. Participants shared strategies being used in their own research environments, for example, independent verification of results before publication; blinding individuals generating and/or analyzing the data to certain experimental details; examination of the source data when possible; using auditable data management systems that record all changes made; replication studies; and verification of applicants’ research credentials before hiring.

Workshop evaluations

Evaluation surveys were distributed to the 433 attendees of the in-person RCR workshops, of whom approximately 40% (175 attendees) completed the survey (Figure 2). Survey responses were generally positive, with participants indicating that the workshops were useful for improving their understanding of research integrity and RCR (Figure 2(a-c)). They particularly welcomed the opportunity to discuss the Translational Omics case. Comments in the surveys included:

Participant A: “I really enjoyed the open dialogue, especially since the case study happened at our own institution. It was refreshing to be able to discuss openly … .”

Participant B: “What really drove the conversation for me were the individuals [at the workshop] who were around when it [the case] occurred. It completely changed the viewpoint of the scripted text for me.”

Participant C: “I love that this ‘public discussion’ of RCR is happening. I hope it continues. I would like, going forward, to see concrete strategies come out of these efforts to address the ‘culture’ issues that get in the way of ethical research practice … .”

Figure 2. In-person RCR workshop exit survey participation and responses. A) Workshop survey participants (191) by faculty/staff/other designation (%). B) Exit survey responses on the utility of the workshop. C) Exit survey responses to the proposed goals of the workshop.

Informal feedback

Faculty and staff have on occasion informally communicated their opinions on the new RCR education program (or its individual components such as the workshops, online course, or quiz) to program leaders and organizers. The program as a whole has been viewed as important, necessary, interesting, and/or well organized. Some faculty and staff expressed reservations about the CITI RCR course and their appreciation for the option of alternative educational pathways by means of the workshops and quiz. Unsolicited feedback on the self-assessment quiz included the opinion that a single opportunity to achieve a passing score on the quiz was not fair (compared to unlimited attempts in the CITI RCR course). There was also the view that the multiple choice questions contained inherent bias, which limited the value of the test; and, that the test questions were relevant to some, but irrelevant to other, individual researchers’ responsibilities and interests.

Unsolicited feedback on the online CITI RCR course included several divergent views. For some, the content was generally informative, whereas for others the content was not as relevant to their responsibilities and research interests. Some indicated that the course was too long, taking several hours to complete.

Discussion

In the initial phase of the new DUSoM RCR education program, 100% of eligible research faculty and staff completed their education requirement (online, in-person, or test-based). The objective outcomes of this effort are not yet known. RCR education is widely considered a means of addressing the potential for wrongdoing and noncompliance among research professionals facing the increasingly complex demands of contemporary scientific practice (Antes and DuBois 2014; Fanelli 2009). On the one hand, studies associate a range of positive outcomes with RCR education, including increased awareness, knowledge, and understanding of ethical issues and how to approach them (Torrence et al. 2017). RCR programs are known to produce gains in participants’ ethical decision-making performance and viable mental models for understanding ethical issues, as well as positive changes in organizational outcomes (Torrence et al. 2017; NAS 2017). On the other hand, whether RCR education ultimately minimizes the potential for research wrongdoing is not known.

Numerous challenges also face RCR education programs, including resource and delivery challenges, and goals that may be perceived by stakeholders as too ambitious, unfeasible, and disconnected from the realities facing practitioners of research (Kalichman 2007). Themes expressed in the in-person RCR workshops allude to these challenges, including the perception that ethical principles and values may be out of touch with the systemic pressures of the research enterprise and the everyday concerns and priorities of research stakeholders.

Against this backdrop, Duke University committed to the design and implementation of an RCR education program that, in its first phase, aimed to reach all DUSoM research faculty and staff, provide multiple education options, and integrate formal deliberation of a recent case of internal misconduct. In this narrow context, the program has been successful, yet it has also testified to a range of additional and future challenges, which are discussed in some detail below.

Program feasibility and composition

NIH guidelines emphasize that online education alone is not adequate for purposes of promoting RCR (NIH 2009). Face-to-face learning is preferred. Yet, to be feasible, delivery of face-to-face learning opportunities across a large organization may need to be combined with online and other learning options. Therefore, the DUSoM RCR education program included face-to-face workshops, online education, and a self-assessment quiz, which helped make the program’s initial implementation feasible and potentially increased the program’s acceptability to participants. The DOSI-ASIST Task Force reasoned that faculty and staff would be more likely to participate in a program that incorporated educational options other than online courses, given that many faculty and staff already have numerous ongoing compliance requirements that they need to complete online.

Institutions developing RCR education programs for large cohorts of research faculty and staff may want to consider similar hybrid or “menu-based” approaches. In doing so, they should consider the question of quality across different education options. For example, online courses can provide broader content than in-person workshops, but lack the depth of deliberation that in-person workshops can provide. However, with online courses, participants can also skip the content and complete the requirement by passing the quizzes, which are often administered to allow unlimited “retakes” until a passing score is achieved. These distinctive strengths and weaknesses create the potential for some unevenness at the programmatic level, where a proportion of the participating population may have a broad and general understanding of RCR, while another proportion has deeper understanding across fewer topics.

One response to this potential for unevenness is to mandate that participants complete both an online course and in-person education. This is perhaps a more feasible solution in the context of ongoing education (see “Continuing education,” below), as compared to implementing an initial RCR education program, which was limited by time constraints and designed to ensure exposure to basic ethical considerations.

Identification and tracking logistics

Mandatory RCR education programs aiming to reach large numbers of research faculty and staff in a relatively short amount of time require a significant investment of resources, and questions of feasibility and logistics may quickly arise. For example, in the DUSoM program, the identification, notification, and tracking of prospective participants required a significant investment of time and innovation. Identification of participants required development of an algorithm, utilizing multiple databases to assemble the prospective required population. DOSI-ASIST worked in concert with local departmental leaders and points-of-contact to support curation of the eligible participant list and to communicate information and requirements about the new program, which was, and continues to be a critical aspect of the program’s sustainability.

DUSoM also recently deployed an RCR Education Tracker to automate the identification algorithm. The Education Tracker application facilitates ongoing curation of the participant list, issues reminders, and imports RCR course completion data directly from the Duke LMS. The LMS allows departmental RCR liaisons and faculty and staff participants to independently oversee participants’ progress in RCR education at the unit level and for individual unit members.

Unexpected technical issues can affect long-term data integrity of quiz analytics from specific RCR quizzes or tests. At Duke, we observed that although the Duke LMS infrastructure worked very well to internally house the RCR self-assessment quiz and manage completion data from multiple course options, there were unexpected technical issues with the specific quiz implementation settings. Identified issues with the self-assessment quiz were rectified as quickly as possible, but resulted in the loss of cumulative analytics data and allowed for a small proportion of candidates (known for 60 out of 2700 individuals) to re-take the quiz (some passed twice, some failed twice, and some passed once and failed once). Institutions considering similar large-scale RCR self-assessment quizzes will want to carefully test and continually monitor their tracking and data recording systems over time to identify errors as quickly as possible and periodically archive records when possible.

Integrating internal cases of misconduct

Case studies of misconduct and DRPs are widely and commonly used to ground RCR education in real-world examples and contexts (Antes and DuBois 2014; Kalichman 2013). However, we are not aware of any published accounts of RCR programs incorporating internal misconduct cases, that is, cases of research wrongdoing that occurred at the institution where the educational programs are being implemented. At Duke, the decision to include the Translational Omics case in the new RCR education program was driven primarily by a need for program legitimacy and acceptance, which required a demonstration of openness with respect to past misconduct.

However, one can also imagine research communities coming together with the specific goal of learning from and reconciling the effects of research wrongdoing on morale, trust, and accountability. Similar processes of deliberation have driven efforts to address collectively traumatizing events in indigenous communities (for example, Erfan 2017). It is possible that workshop deliberations of the Translational Omics case inadvertently addressed this need for reconciliation and healing, however, this was not an overt goal of the program.

The social science concept of “reflexivity” is useful for purposes of grasping the potential dynamics and uses of internal misconduct deliberation in RCR education. Social scientists use the concept of reflexivity to examine how their subjective selves, prejudices, and biases impact their scientific methods, assumptions, data, and analyses (Davies 1999; Probst 2015). “Reflexivity,” write Myerhoff and Ruby, “loosens us from habit and custom and turns [us] back to contemplate ourselves” (Ruby 2016). It is possible for academic medical centers to facilitate the climate, opportunities, and operational processes needed to foster a similar ethic of reflexive deliberation toward internal acts of misconduct. Reflexive deliberation of internal wrongdoing can be seen as a way for organizations to break the cycles of silence that typically surround internal misconduct cases. Reflexivity may be a way for leaders to demonstrate accountability and sincerity with respect to their claims of fostering new “cultures” of research integrity.

Reflexive deliberation using case studies of internal misconduct has the potential to deepen researchers’ knowledge and awareness of the numerous complex issues and considerations surrounding research and related wrongdoing. Considered closely and in detail, misconduct cases are rarely only about misconduct. The Translational Omics case, for example, illustrates considerations relevant to many of the RCR foci identified by the NIH, including issues in data management, potential conflicts of interest, human subjects protection, and mentoring (NIH 2009).

At another level, reflexive deliberation may be useful for (re)building organizations, academic communities, and trust following major cases of ethical and system failure. Notably, such efforts almost always seek to address problems in trust, accountability, and organizational culture. New RCR education programs in particular may benefit from the interest and sense of investment generated by reflexive deliberation of cases that have profoundly impacted a local community. Research staff and faculty may develop a distinct sense of connection to an RCR education program that includes open and substantial reflection on an internal case of research wrongdoing.

Institutions interested in reflexive deliberation as part of their RCR education programs may also need to consider the authenticity of the deliberative process. Given the high stakes often surrounding high-profile wrongdoing, it is possible that leadership interests and biases may affect and compromise the deliberative process. Core aspects of the process – how cases are written up, who is invited to deliberate them, and how they are deliberated – can and should be managed so that they reflect the interests and concerns of all stakeholders, and not just the agenda of those initiating the process. Steps may need to be taken to support individuals who are inadvertently distressed or placed at risk, for example, by the disclosure of new information during a deliberative process.

Reflexive deliberation of internal misconduct appears to be a very under-utilized option in RCR education programs. More information is needed to help guide institutions in developing effective, meaningful, and organizationally appropriate ways of making such deliberations part of the RCR education landscape.

Program evaluation

Evaluation of RCR education programs for effectiveness is important, but also conceptually and logistically challenging. The question of what, exactly, should count as effectiveness and how program outcomes should be measured is still an area of much discussion and debate (Mumford, Steele, and Watts 2015; Olson 2014). The need to implement a program expeditiously in the face of numerous feasibility and logistical considerations can temporarily override the need for comprehensive program evaluation. In the DUSoM program, formal evaluation in the first phase was limited to a survey administered to workshop attendees. Plans to evaluate the ongoing RCR education program at Duke more rigorously are currently under consideration. They include offering more systematic surveys to all participants and the creation of a board of external consultants who could regularly review and provide feedback on future RCR education activities. Institutions may want to discuss early on in the program development process the challenge of program evaluation, identifying goals for the evaluative process, and securing resources to conduct systematic evaluations.

Continuing education

Beyond its initial phase, the DUSoM’s RCR education program is evolving in two ways: 1) required RCR education is ongoing for all DUSoM research faculty and staff and, 2) the education program is being scaled across the entire institution (including the School of Nursing, and the non-medical schools/campus) to include all faculty and staff engaged in research at Duke University. In the future, research faculty and staff at Duke will need to complete one online RCR course every three years, and one in-person or “collaborative” education session every three years (Table 3).

Table 3. Evolution of Duke faculty and staff RCR education program.

	Short-Term RCR Program	Ongoing RCR Program
Program duration?	Nov 2017 – June 2018	October 2018 – Present
Who is required to participate?	Required for only DUSoM faculty and staff engaged in research	Required for all Duke faculty and staff engaged in research
What is the requirement?	Complete any one RCR program option and requirement is complete	Complete one self-directed online RCR courseevery 3 years-AND- Complete one in-person RCR collaborative learning experience every 3 years
What are the RCR program offerings?	1) Self-Assessment, or 2) CITI RCR course, or 3) ASIST/Trent In-Person RCR Workshop, or 4) Attest to prior RCR completion within past two years	Self-Directed Online: 1) CITI RCR course (faculty and staff tracks) 2) + additional options coming soon In-Person, Collaborative: 1) ASIST/Trent In-Person RCR Workshop 2) Unit-led Discussion 3) Duke Interactive Workshop on RCR Topic
Tracking program completion	Testing new-algorithm, semi-static list of persons	RCR Education Tracker app, real-time updates to list of persons and completion progress
Admin unit responsible for organizing and tracking completion	ASIST and Departmental leadership and administrators	ASIST and Departmental leadership and administrators

The online RCR course is expected to deliver a comprehensive overview of RCR topics and issues to new Duke employees, as well as to those who previously did not complete the course. In-person education, by contrast, will be designed to expand upon and deepen the knowledge gained through online education with deliberation of topics relevant to Duke researchers. In addition, DOSI-ASIST is organizing monthly research town hall meetings to address topical RCR issues such as plagiarism, authorship, data management resources, international collaborations, and entrepreneurship and conflicts of interest. The recently settled misconduct case is also being considered for reflexive deliberation, building on the lessons learnt through inclusion of the Translational Omics case in the new RCR education program.

RCR program leaders at Duke also envision a more decentralized model of RCR education, so that more research units take control and organize their own RCR programs in the form of lecture series, round-table discussions, town halls, grand rounds, workshops, and similar activities. This model reflects a call by some experts (for example, Kalichman 2014) to de-emphasize reliance on short-term, “top-down” RCR education in favor of integrating RCR education into actual research environments (i.e., departments, centers, institutes, collaborative groups, and individual laboratories). Such efforts are likely to better reflect the interests and priorities of researchers and to improve the translation of RCR into practice. Decentralized RCR education may also help reduce reliance on central support offices, such as DOSI-ASIST, so that such central offices are not the sole providers of faculty and staff RCR education options.

Conclusion

The Duke faculty and staff RCR education program is an ongoing and evolving response to the institution’s recent experience with research wrongdoing, the need for expanded and more inclusive and ongoing RCR education, and a range of feasibility, logistic, and quality considerations. To implement similar large-scale RCR education programs successfully, institutions may need to consider a range of factors, including:

• availability of resources to implement a mandatory program

• research interests and priorities of their intended audience

• potential need for multiple RCR education options

• challenges of identifying the relevant population

• tracking program completions across a large research enterprise

• need for program evaluation tools

• how the program will evolve and be sustained in the future

Balancing these many considerations is a significant challenge in implementing RCR education programs, and, clearly, more research and evidence is needed to help guide institutions through these challenges. RCR education programs would also benefit from further discussion and research on the use of reflexive deliberation of recent internal cases of misconduct and DRPs. Reflexive deliberation potentially adds relevance and timeliness to an RCR education program, deepens RCR learning, and promotes trust and accountability, yet, research is needed to substantiate these claims.

Culture change is a slow process that requires long-term planning and continual effort and focus. RCR education programs will not immediately remedy gaps in research integrity, nor are they likely to eliminate research misconduct. Yet, scientific integrity may be improved with RCR education programs that strive to reach all research faculty, staff, and students that include open, reflexive deliberation of internal cases of research wrongdoing.

Acknowledgments

We recognize the contributions of many persons who helped shape, lead, develop, and implement the Duke Faculty and Staff RCR Education Program. We thank Dean Mary Klotman, Dr. Raphael Valdivia, Dr. Colin Duckett, and Dr. Ross McKinney for their leadership and participation in the in-person RCR workshops; Joe Rusnak and the DOSI-ASIST Task Force; the Duke Learning Management System team, specifically Heather Mabry and Brian Aucoin for assistance with tracking education completions; Leroy Lee for assistance with Duke personnel databases; Molly Starback, Dr. Christopher Nicchitta, and Dr. Hugh Crumley, for their advice and support on the emerging RCR education program; the Duke Office of Research Informatics, specifically April Feickert, Lori Evans, Chet Corey, and Paula Morrison for developing the RCR Education Tracker; and the DUSoM unit-level RCR liaisons and leaders for their continued efforts to promote RCR outreach within their departments, centers, and institutes and more broadly across Duke University.

Disclosure statement

No potential conflict of interest was reported by the authors.