Assessment criteria for research misconduct: Taiwanese researchers’ perceptions

ABSTRACT

Assessing the severity of an instance of research misconduct is undoubtedly challenging, especially when the result of the assessment may be key to suggesting subsequent sanctions. However, only a few references are currently available in the Taiwanese academic context. In a previous study, the present authors developed The Assessment Criteria for Research Misconduct (The Criteria) based on existing international policies and guidelines and reviewed by local research scholars for content validity. The Criteria, with a total of 28 items, were organized into three sections: general criteria for determining case severity, aggravating criteria, and mitigating criteria. In the current study, the authors further conducted a survey and collected data on 277 Taiwanese researchers’ perceived importance of each criterion included in The Criteria. The results showed that participants generally agreed with the importance of all criteria. However, the group that lacked case-handling experience attributed significantly greater levels of importance to the criterion of original will (proactive, passive, or coercive) toward participation in misconduct than did the experienced group. In addition, the participants exhibited greater variation in the perceived importance of the mitigating criteria. Finally, the possible utility of The Criteria in real contexts and training materials is suggested in the study.

KEYWORDS:

• Empirical studies of research ethics
• legal and regulatory issues in research
• misconduct in research
• research integrity
• responsible conduct of research
• science policy

Introduction

Researchers across nations and disciplines should be committed to ensuring research integrity. Once an instance of research misconduct is established, determining its severity for subsequent administrative sanctions is a necessary yet difficult task, especially considering the complexity of the case and the different levels of knowledge and experience among review panel members. Taiwan, like many world economies, pays great attention to research and development (R&D), considering R&D an essential factor in survival and a display of national strength. Therefore, the Taiwanese government, like many R&D-intensive countries, takes research seriously and continuously attempts to promote research integrity by all means (Resnik, Rasmussen, and Kissling 2015). In Taiwan, there are two government agencies in charge of developing policies related to research ethics and integrity: the Ministry of Education (MOE), which governs the functions of higher education, and the Ministry of Science and Technology (MOST), which supports research activities. These policies and associated regulations, guidelines and mandates (i.e., MOST 2019) provide a general framework that requires research institutions to oversee intramural research activities thoroughly and responsibly. Accordingly, all research institutions, including universities and colleges, have their respective policies and regulations for the management of research activities, education and training concerning research integrity, and procedures for handling research misconduct. Taiwanese governmental policies are continually revised to align with global standards, and institutions must update their own policies and rules accordingly.

Among research integrity-related policies, regulations concerning misconduct have received the most attention. Documents from the MOE and the MOST (MOE 2016, 2021; MOST 2020) all include a definition of misconduct and procedures for investigating and adjudicating it. These two government agencies not only require institutions to have clear handling procedures but also oversee institutional investigations and have the authority to make final decisions on alleged cases. However, there is an element missing from this framework for action: no suggested criteria are available for assessing an instance of research misconduct. In practice, the judgment of case severity is usually made by a review panel in which members come from different academic disciplines and have unique personal views and past experiences. As a result, the final judgments depend on the panel members and occasionally are inconsistent across cases. In other words, even though Taiwanese government agencies and institutions have strived to ensure fairness and justice during the investigation process, the objectivity and transparency needed for the assessment of severity leave room for improvement, especially when an assessment leads to suggested sanctions.

In Taiwan, the need for assessment criteria has intensified since the occurrence of high-profile cases in recent years, such as the SAGE peer-review scandal (SAGE Publications 2014), falsified data in the journal papers of a top university president (Retraction Watch 2017), and plagiarism in a mayoral candidate’s master’s thesis (Chung 2020). Academics and the public, both international and domestic, followed these cases closely and saw how the allegations were investigated, what assessments were made, and what sanctions were imposed. This attention posed great pressure on relevant government agencies, universities and their respective review panels. The final judgments of these cases, of course, led to wide discussions concerning both the positive (pro) and negative (con) positions. Given the situation that few existing criteria for assessing research misconduct are operationally credible in Taiwan, everyone, whether an academic or not, seemed to judge case severity from their own point of view. The worst scenario of a judgment would be if, due to a lack of consensus and trust, respondents, review panelists, the research community, institutions, government agencies, and the public as a whole felt dissatisfied with the investigation process and results (see also Nouchi et al. 2020). Therefore, we responded to the need to develop a set of assessment criteria that are more comprehensive, consistent, and applicable to the Taiwanese academic context.

In a previous study (Hsueh, Pan, and Chou 2019), we compiled a draft of a self-developed assessment criteria by collecting international and local references. There are a few references available for facilitating the decision-making process by suggesting criteria for consideration; such criteria are offered by the National Science Foundation (2002), the Netherlands Code of Conduct for Research Integrity (2018), and the Guide to Managing and Investigating Potential Breaches to the Australian Code for the Responsible Conduct of Research (hereafter referred to as the Australian Guide) by the National Health and Medical Research Council (2018). The National Science Foundation (45 CFR § 689.3 2002) suggests five criteria to consider: (1) how serious the misconduct was; (2) the degree to which the misconduct was knowing, intentional, or reckless; (3) whether it was an isolated event or part of a pattern; (4) whether it had a significant impact on research records, research subjects, other researchers, institutions or public welfare; and (5) other relevant circumstances. Moreover, the Netherlands Code of Conduct for Research Integrity (2018) lists 12 assessment criteria when considering established misconduct. Some criteria are simple to consider, such as (b) intentional act, gross negligence, carelessness, or ignorance; and (g) scientific/scholarly publication or popularizing article/teaching materials or an advisory report. Some of these criteria pertain to the person who committed the misconduct, including (f) potential benefits; (i) position and experience; and (j) any prior violations. Other criteria relate to the impacts of the misconduct, including (c) the possible consequences for the validity of the research and for prevailing scientific knowledge; (d) potential effects on trust in research and among researchers; and (e) potential impact on individuals, society and the environment. Still other criteria take disciplinary differences into account, including (a) the extent of noncompliance and (h) opinions within the disciplines. In general, these 12 criteria are more concrete for review panels. The frequency of violations and the impact of misconduct on others are suggested as relevant factors by both NSF regulations and the Netherlands Code. “Repeated breaches” is included as a factor to consider when determining the seriousness of a breach in the Australian Guide in addition to other factors, such as the extent of the departure from accepted practice, the extent to which the breach affects the trustworthiness of research, the researcher’s level of experience, and institutional failures (National Health and Medical Research Council 2018). The European Network of Research Integrity Offices (2019) also highlights certain aspects of systemic problems that need to be handled to prevent similar occurrences in the future, such as specific pressure, poor stewardship, lack of RI training, and insufficient guidelines (p. 25).

The Assessment Criteria for Research Misconduct (hereinafter referred to as The Criteria) developed in our previous study (Hsueh, Pan, and Chou 2019) address most of the criteria listed in the NSF and the Netherlands Code, but our criteria omit the basic criterion for judging an established instance of misconduct, that is, whether the misconduct is knowing, intentional, or reckless. The reason for this omission is that The Criteria are intended to be applied to established research misconduct. In other words, the misconduct intended to be assessed by The Criteria has already been judged as subjectively intentional (direct or indirect), reckless, or negligent. Other criteria in the NSF and the Netherlands Code, as well as the Australian Guide, were either rewritten or reorganized specifically for the Taiwanese academic context. Certain new criteria were also included to address, for example, the type of research misconduct found in the instance because the Taiwanese MOST (2020) lists a broader range of misconduct types, such as self-plagiarism and duplicate publishing without citation, which are either debatable or considered less serious than typical falsification, fabrication and plagiarism (FFP).

More importantly, we added the dimensions of aggravation and mitigation into The Criteria to align with the general principles of proportionality and to ensure the fairness of administrative sanctions. This addition responds to the suggestion from The European Code of Conduct for Research Integrity (2017) that any action taken should be proportionate to the severity of the misconduct violation. For example, the dimension of aggravation necessitated new criteria, including whether the misconduct has a serial pattern, whether there is any interference with the investigation process, and whether there is any involvement of criminal law. Likewise, the dimension of mitigation adds criteria such as whether a correction or retraction has been issued, whether a proactive confession was made, and whether the misconduct has an interdisciplinary nature, as suggested by a document from the MOE (2021). Keränen (2006) also suggests considering excusing (mitigating) conditions for less serious research misconduct, such as isolated incidents, but not for more serious (significant and grave) research misconduct, such as that with foreseeable, multiple negative consequences or malevolence. A recent study conducted by Abdi, Nemery, and Dierickx (2021) also suggests that case-specific circumstances for aggravation or mitigation be included in the evaluation of research misconduct, including the experience of the accused, the nature and extent of the misconduct, the importance of the scientific product, the context of the relationship between the accused (e.g., PhD student and supervisor), repetition of the violation, the potential harm caused, and the specific role or input of the coauthor. We found that most of these circumstances had already been included when drafting our criteria.

The initial draft of The Criteria included 28 items and had already been reviewed by a total of six local experts in research integrity fields before the distribution of the research survey. Four of these experts were law professors who had been highly engaged in research integrity training during recent years. The other two professors were scholars dedicated to morality and ethics education. All invited experts had considerable experience managing research misconduct allegations. Additionally, the initial version of The Criteria was reviewed and commented on by two anonymous referees for its first publication in Chinese (i.e., Hsueh, Pan, and Chou 2019); that is, the authors had already revised this version of The Criteria by adopting the professors’ and referees’ review comments.

In the current study, we present the final draft of The Criteria and demonstrate how Taiwanese researchers perceived the importance of these criteria by conducting a survey. This study presents the survey results and shows our efforts in raising the issue and building a consensus concerning The Criteria among all researchers in Taiwan. If a considerable level of agreement could be obtained among researchers concerning the importance of each criterion, the authors could then compile important criteria in the final version of The Criteria and put The Criteria into practice.

Specifically, three research questions are proposed in this study:

1. How do Taiwanese researchers perceive the importance of the general criteria for assessing the severity of research misconduct cases?

2. How do Taiwanese researchers perceive the importance of the criteria for assessing the possibility of aggravation?

3. How do Taiwanese researchers perceive the importance of the criteria for assessing the possibility of mitigation?

Materials and methods

Instrument: The criteria survey

A survey method was employed in this study. A self-developed, anonymous instrument, The Criteria Survey, consisted of three sections that included a total of 28 closed items to be scored. The survey items are displayed in the tables in the Results section. Section 1 presented the importance of the general criteria, which contained 16 items focusing on reviewing the nature of a confirmed instance of research misconduct. These items were covered by three frameworks when considering the seriousness of misconduct – intentions, consequences, and role-related duties – as suggested by Keränen (2006). Section 2 presented the importance of aggravating criteria, which contained four items intended to assess the possibility of aggravation. Section 3 presented the importance of mitigating criteria, which listed an additional eight items intended to examine the possibility of mitigation. In addition, example considerations or questions for each criterion were provided in the original survey for participants to better make their choices on the agreement scale, as listed in the Appendix to this study.

Each item of The Criteria Survey was presented in a 6-point Likert-type scale format. Research participants were encouraged to score each item from one to six: 1–strongly unimportant, 2–unimportant, 3–somewhat unimportant, 4–somewhat important, 5–important, and 6–strongly important. The Criteria Survey was originally developed and administered in traditional Chinese and then translated into English for the current article.

Research participants

Since we want to raise awareness concerning misconduct assessment issues in local research communities, our target population included full-time researchers of all levels of academics across different disciplines in Taiwan. Their academic titles included assistant, associate, and full professors and research fellows at Taiwan universities and research institutions. No limitations were set concerning prior experience in handling research misconduct allegations, but participants were required to indicate their level of experience. In other words, the participants were engaged in research, but they might or might not have had experience serving on a case review panel when they participated in the study.

Sampling procedures

Approximately 700 hard copies of The Criteria Survey were distributed through purposive sampling. Of those copies, 305 were mailed to the research integrity officers (RIOs) of institutions subsidized by the MOST. These officers are usually vice presidents for research or directors of R&D and are responsible for research integrity policy or case handling within their respective universities or institutions. The rest of the surveys were mailed to academics at all levels by the present authors’ professional networks through snowball sampling. During the answering process, each potential participant received a package containing research materials, including (1) IRB-approved research information and recruitment sheets, (2) a hard copy of The Criteria Survey and additional questions concerning demographics (without name identification) and previous experience with case review, and (3) a prepaid postage envelope. Once participants consented to join the study and completed the survey, they could submit the survey either by returning the questionnaire in a self-sealed envelope to the authors directly or by mailing the questionnaire back to the authors using the prepaid envelopes. Participants answered The Criteria Survey individually and anonymously without any reward or external pressure, such as a time limit.

A total of 292 participants completed the survey, and the response rate was 41.71%. However, responses from 15 individuals were excluded from the dataset due to low reliability (incomplete survey responses, missing values in a whole section, and disqualification of respondent). Hence, the valid sample size was 277, and the effective rate was 94.86%.

Statistical analysis

The descriptive statistics (mean and standard deviation) demonstrate how participants rated the importance of each criterion. Although a comparison between experienced participants and unexperienced participants using a t test was not the focus of our research, the answers of those groups and all answers as a whole are presented for greater clarity. Moreover, the kurtosis (K) value for each criterion was calculated. A K-value greater than +3.0 would indicate a high degree of consensus and a sharper curve than a normal distribution, while a K-value equal to 0 would indicate responses following a normal distribution. On the other hand, a K-value less than −1.0 would indicate a wide variety of opinions, meaning that the curve would be flatter than a normal distribution (Wager et al. 2009).

Results

General demographic characteristics

Table 1 presents the 277 participants’ demographic characteristics. In brief, more than 70% of participants were from comprehensive universities (public: 136, 49.10%; private: 65, 23.47%). Nearly 60% (164, 59.21%) of participants were full professors or full research fellows. Moreover, three-quarters of participating professors and researchers (212, 76.53%) had worked in academia for more than ten years (≥ 21 years: 115, 41.52%; 11–20 years: 97, 35.02%); only 22 (7.94%) participants had worked in academia for less than five years and were considered early-career researchers.

Table 1. Participants’ demographic characteristics (N = 277).

Demographic Items		N	%
Type of research institution	Public comprehensive university	136	49.10
	Public university of science and technology	34	12.27
	Private comprehensive university	65	23.47
	Private university of science and technology	35	12.64
	Public research organization	7	2.53
Academic rank	Full professor/research fellow	164	59.21
	Associate professor/research fellow	66	23.83
	Assistant professors/research fellow	45	16.25
	Missing	2	0.72
Years of serving in academia	≥ 21 years	115	41.52
	11–20 years	97	35.02
	6–10 years	37	13.36
	≤ 5 years	22	7.94
	Missing	6	2.17
Self-defined research field	Humanities and social and behavioral sciences (HSBS)	170	61.37
	Science, engineering, medicine, and agriculture (SEMA)	98	35.38
	Interdisciplinary research across HSBS and SEMA	8	2.89
	Other	1	0.36

(1) A public/private comprehensive university is an institution for higher learning and research within the traditional education system. The university departments are from various academic disciplines. The majority of students are from academic high schools. A public/private university of science and technology is a higher education institution within the vocational education system. Most university departments focus on the training of higher-level professional and practical work skills. The majority of students are from vocational high schools.

(2) The survey allowed respondents to skip demographic questions they preferred not to answer, so some data were missing.

Additionally, participants were given broad multiple-choice options to define their research disciplines and were further categorized into three groups accordingly. A total of 170 (61.37%) participants worked in humanities and social and behavioral sciences (HSBS)-related disciplines, while 98 (35.38%) participants worked in science, engineering, medicine, and agriculture (SEMA)-related subjects. Eight (2.89%) participants defined themselves as interdisciplinary researchers working across HSBS and SEMA areas.

Table 2 demonstrates participants’ previous experience in handling and investigating alleged violations of research integrity.

Table 2. Participants’ experience in handling and investigating suspected research misconduct.

Questions	Item	n	%
Within the past five years, have you ever handled or investigated any allegations of suspected research misconduct? (n = 277)	No	173	62.45
	Yes	104	37.55
If your answer is “no,” (n = 173)
Would you be willing to be invited to do so in the future?	Yes	72	41.62
	No	49	28.32
	Depends	23	13.29
	Missing	29	16.76
If your answer is “yes,” (n = 104)
Would you be willing to be invited again to do so in the future?	Yes	62	59.62
	No	19	18.27
	Depends	20	19.23
	Missing	3	2.88
How many allegations have you handled or investigated, regardless of whether any research misconduct was found or not?	1–3 cases	66	63.46
	4–6 cases	9	8.65
	7–9 cases	2	1.92
	≥ 10 cases	7	6.73
	Missing	20	19.23
Have you ever received training specifically for handling and investigating allegations before doing so, in either a formal or informal way?	Yes	18	17.31
	No	81	77.88
	Missing	5	4.81
Again, if your answer is “yes,” (n = 104)	Mean	SD	K
How confident are you that you have sufficient capability to propose subsequent sanctions taken against case respondents?	3.98	0.74	0.14
How confident are you that you are able to fairly and objectively propose subsequent sanctions?	4.21	0.70	1.33

Participants were allowed to skip experience questions, so some data were missing.

Mean: Rating from 1–no confidence, 2–slight confidence, 3–neutral, 4–moderate confidence, 5–high confidence.

K: kurtosis values.

Specifically, although 173 (62.45%) of 277 participants indicated no experience in handling or investigating research misconduct allegations within the past five years, 72 participants (41.62%) indicated a willingness to serve as a case reviewer or investigator in the future.

On the other hand, 104 (37.55%) of 277 research participants indicated that they had handled or investigated research misconduct allegations within the past five years. Among these individuals, 66 (63.46%) participants had dealt with only a small number of suspected cases (i.e., fewer than three cases). In comparison, seven participants (6.73%) indicated that they had handled or investigated more than ten instances. Moreover, the 104 participants in this category generally had a moderate level of confidence in their capability to propose subsequent actions and sanctions taken against the respondents of allegations (Mean = 3.98, SD = 0.74, K = 0.14; note that 3–neutral, 4–moderate confidence). However, these participants indicated greater confidence in proposing sanctions fairly and objectively (Mean = 4.21, SD = 0.70, K = 1.33). Additionally, 62 (59.62%) of 104 participants indicated a positive attitude toward repeating this service. Only 18 participants (17.31%) indicated that they had ever received training for managing allegations, while 81 participants (77.88%) had no such experience.

Perceived importance of the general criteria for assessing case severity

Table 3 lists the criteria as sorted from the largest to the smallest mean values rated by all participants (not in the original order of items in our survey).

Table 3. The mean, SD, and K values of the importance of each criterion in Section 1.

Section 1: The Importance of the General Criteria for Case Severity	All (n = 277)			Experienced (n = 104)			Unexperienced (n = 173)			t
	Mean	SD	K	Mean	SD	K	Mean	SD	K
1.The actual behaviors constituting the research misconduct found in the instance.	5.58	0.67	6.14*	5.62	0.60	2.87	5.56	0.71	6.81*	0.66
2. The type of research misconduct found in the instance.	5.53	0.76	7.62*	5.55	0.78	5.78*	5.52	0.76	9.06*	0.35
3. The extent of the research misconduct’s potential effects, especially negative effects, on the effective communication of scientific knowledge and the security of public interest.	5.26	0.97	3.77*	5.25	0.98	4.24*	5.26	0.97	3.63*	−0.84
4. The respondent’s status and role in the instance.	5.21	0.86	3.30*	5.15	0.96	3.89*	5.25	0.80	2.05	−0.88
5. The extent to which the respondent or interested parties will enjoy any interest and/or potential benefit from the outcomes of the research misconduct.	5.17	1.00	3.63*	5.16	0.88	2.80	5.18	1.07	3.74*	−0.24
6. The respondent’s motivation and purpose behind the research misconduct.	5.12	1.08	2.71	5.11	1.12	2.68	5.13	1.06	2.83	−0.15
7. The respondent’s original will toward participation in the instance.	5.12	0.84	4.75*	4.97	0.91	3.74*	5.20	0.79	5.79*	−2.23*
8. The degree of deviation between the generally accepted research practices of the respondent’s affiliated research discipline and the respondent’s actual behaviors constituting the research misconduct.	5.11	1.01	2.90	5.10	0.89	0.83	5.13	1.08	3.35*	−0.25
9. The extent of the research misconduct’s potential effects, especially negative effects, on the research of the respondent.	5.09	1.07	2.43	5.13	1.01	2.45	5.06	1.11	2.40	0.58
10. The respondent’s attitude (e.g., degree of cooperation) in the course of the investigation process.	5.00	1.03	1.38	4.94	0.96	0.16	5.04	1.07	1.97	−0.77
11. The extent of the research misconduct’s potential effects, especially negative effects, on the respondent’s affiliated institution(s) and affiliated research discipline(s), as well as related research by other researchers.	4.95	1.09	1.09	4.98	1.08	1.64	4.94	1.11	0.85	0.33
12. Whether the research involving research misconduct has been released or published academically. If yes, what were the forms of publication or release.	4.84	1.03	0.66	4.93	0.84	−0.28	4.78	1.13	0.43	1.29
13. Whether the respondent violated any research policy or standards of and/or contractual agreement with research funding agencies.	4.81	1.04	0.95	4.84	1.01	1.12	4.79	1.06	0.91	0.35
14. Whether the respondent’s research misconduct is under investigation by a third party.	4.47	1.31	0.17	4.51	1.29	0.04	4.45	1.33	0.24	0.36
15. Whether the respondent has received research integrity training.	4.06	1.29	–0.26	3.88	1.30	−0.37	4.17	1.28	−0.16	−1.78
16. The respondent’s level of academic qualifications.	4.04	1.48	–0.83	4.05	1.51	−0.98	4.04	1.47	−0.72	0.04

Mean: rating from 1–strongly unimportant, 2–unimportant, 3–somewhat unimportant, 4–somewhat important, 5–important, and 6–strongly important.

*K (kurtosis) value higher than 3.00.

*t value meaning p< .05.

In general, participants tended to agree with the importance of all 16 general criteria in Section 1, as the mean values of their rankings of all items were 4.04 or higher. The only exception was Item 15 (receipt or nonreceipt of training), which was given a mean value of 3.88 by the experienced group. It is also noteworthy that 10 out of 16 mean values were rated over 5.00 by all participants. Although the rankings of each criterion’s mean value were slightly different between the experienced group and the unexperienced group, agreement levels on each criterion were similar.

Taking all participants as a whole, the five most recognized criteria for the general assessment of the severity of misconduct were Items 1 (actual behaviors), 2 (the type of misconduct), and 3 (the extent of the potential negative effects on society and individuals), followed by Items 4 (the respondent’s status and role) and 5 (the extent of potential interests and benefits from misconduct).

As shown in Table 3, the only significant difference between the experienced group and the unexperienced group related to Item 7 (original will with respect to participation, t = −2.23, p < .05), meaning that the unexperienced group tended to rate the respondent’s own will as a more important consideration than did the experienced group.

The K values of the aforementioned top five items were >3.00, ranging from 3.30–7.62, indicating a high degree of consensus concerning the degree of importance of each criterion among the participants. None of the K values in Table 3 were less than – 1.00 in Section 1.

In contrast, it is noteworthy that the importance of Item 16 (respondent’s level of academic qualifications) was the item rated lowest by all participants and by the unexperienced group, followed by Item 15 (receipt or nonreceipt of training). The K values of the two items were less than 0.00 for all participants, the experienced group and the unexperienced group, meaning that the participants exhibited greater variation in their opinions concerning the degree of importance of these two criteria than in their responses to other items.

Perceived importance of the criteria for possible aggravation

As shown in Table 4, participants were prone to agree with the importance of all four criteria in Section 2. The mean values of all items were 5.16 or higher (between 5–important and 6–strongly important). In other words, the importance of each criterion in the aggravation group was agreed upon by most participants.

Table 4. The mean, SD, and K values of the importance of each criterion in Section 2.

Section 2: The Importance of Criteria for Possible Aggravation	All (n = 277)			Experienced (n = 104)			Unexperienced (n = 173)			t
	Mean	SD	K	Mean	SD	K	Mean	SD	K
17. The respondent has violated research integrity more than once, especially if it is apparent that the misconduct is not a single event but has constituted a “pattern” of noncompliance. The frequency and duration of the current and past noncompliance shall be examined.	5.49	0.74	7.04*	5.52	0.74	5.83*	5.48	0.74	7.96*	0.43
18. The respondent intentionally interferes with the process of handling and investigating the allegation.	5.45	0.78	3.94*	5.45	0.77	4.00*	5.46	0.79	4.04*	−0.05
19. The research misconduct involves the violation(s) of criminal law.	5.29	1.08	3.82*	5.30	1.00	2.11	5.28	1.13	4.43*	0.10
20. The research misconduct has caused serious and difficult-to-recover damage to the respondent’s affiliated institution(s).	5.19	0.94	3.70*	5.25	0.88	2.11	5.16	0.98	4.22*	0.75

Mean: rating from 1–strongly unimportant, 2–unimportant, 3–somewhat unimportant, 4–somewhat important, 5–important, and 6–strongly important.

*K (kurtosis) value higher than 3.00.

Specifically, there was consistency in the rankings of means among all participants, the experienced group, and the unexperienced group. The four criteria for weighing the likelihood of aggravation were Items 17 (the frequency and duration of current and past noncompliance), 18 (respondent’s possible interference with the process of handling and investigating the allegation), 19 (violations involving criminal law), and 20 (causing serious and difficult-to-recover damage to the respondent’s affiliated institutions). There was no significant difference in any mean values between the experienced group and the unexperienced group. Table 4 also shows that most of the K values in Section 2 were >3.00, ranging from 3.70–7.96. The only two exceptions were the K values of Items 19 (k = 2.11) and 20 (k = 2.11) by the experienced group. In general, the results suggest that participants had reached a medium to high level of consensus concerning the degree of importance of all aggravating criteria included in this section.

Perceived importance of the criteria for possible mitigation

Participants generally agreed with the importance of the eight criteria in Section 3. The mean values of their rankings of all these items were 4.55 or higher (see Table 5; note that 4–somewhat important, 5–important, and 6–strongly important). Although the rankings of means for each criterion were slightly different between the experienced group and the unexperienced group (e.g., Items 25 and 26), their agreement levels on each criterion were similar.

Table 5. The mean, SD, and K values of the importance of each criterion in Section 3.

Section 3: The Importance of Criteria for Possible Mitigation	All (n = 277)			Experienced (n = 104)			Unexperienced (n = 173)			t
	Mean	SD	K	Mean	SD	K	Mean	SD	K
21. Prior to the finding (or report) of research misconduct, the respondent has taken the initiative to correct or withdraw the research product(s) involving the wrongdoing, including products such as journal articles and conference papers.	5.13	0.92	2.19	5.15	0.86	1.28	5.12	0.95	2.50	0.29
22. Prior to the finding (or report) of research misconduct, the respondent has taken the initiative to confess the wrongdoing to related parties, such as affiliated institution(s), supervising departments(s), and research funding agencies, as well as law enforcement agencies (as appropriate).	5.12	0.87	2.26	5.11	0.81	1.88	5.13	0.91	2.42	−0.20
23. In the course of the investigation process, the respondent has proactively confessed the wrongdoing and intends to repent.	5.09	0.88	1.37	5.07	0.87	0.91	5.11	0.89	1.70	−0.39
24. The research misconduct does not seriously deviate from the commonly accepted research practices of the respondent’s affiliated research discipline.	4.86	1.10	1.54	4.97	0.97	0.44	4.79	1.17	1.58	1.32
25. The research product(s) involving research misconduct are considered an interdisciplinary effort. However, the respondent is likely not professionally active in the particular research field involved in the instance. Additionally, the respondent is not the primary author of the research product(s).	4.71	1.08	1.38	4.66	1.11	2.27	4.74	1.06	0.76	−0.57
26. The research misconduct occurred prior to the time when relevant research policies and standards in Taiwan were formulated.	4.65	1.15	0.94	4.68	1.03	0.35	4.62	1.22	0.97	0.41
27. The respondent shall not be exempted from sanctions even if the individual claims a lack of awareness of research policies and standards, unless this lack of awareness was inevitable for the individual for legitimate reason(s). Mitigation of the sanctions might be prescribed in accordance with the circumstances.	4.61	1.11	1.57	4.57	1.02	1.15	4.63	1.16	1.76	−0.46
28. The research misconduct consists of obvious circumstances for lenient treatment.	4.55	1.08	0.60	4.55	0.98	0.44	4.55	1.13	0.61	−0.05

Mean: rating from 1–strongly unimportant, 2–unimportant, 3–somewhat unimportant, 4–somewhat important, 5–important, and 6–strongly important.

*K (kurtosis) value higher than 3.00.

Among the eight criteria in Section 3, participants indicated that the most important criteria (M > 5.00) for considering mitigation were Items 21 (respondents’ active initiative in correcting or withdrawing the research product(s) involving misconduct), 22 (respondents’ confession of their wrongdoing to related parties), and 23 (respondents’ proactive confession of the wrongdoing and intention to repent).

However, the K-values of all items presented in Section 3 were <3.00 in all three groups, meaning that the participants did not reach a high consensus concerning the degree of importance of the mitigating criteria. In other words, although participants agreed with the importance of these criteria in general, they held relatively varied perceptions regarding the level of importance.

Discussion

The present study first presented The Assessment Criteria for Research Misconduct (The Criteria) and administered a survey (The Criteria Survey) to investigate Taiwanese researchers’ perceived importance of each criterion. The results of the survey indicated that more than half of participants were full professors or researchers, and over three-quarters of participants had served in academia for more than 10 years. The reason for this fact is probably that the surveys were sent to the research integrity officers of all Taiwanese universities or institutions, and the authors employed the personal networks of senior researchers. Although this sample may not represent the expected proportions of the target audience of this study, the authors appreciate these senior researchers’ contributions to The Criteria for their more intimate familiarity with research systems and practices in Taiwan. It is also worth noting that over one-third of participants had experience handling misconduct cases. However, the authors were somewhat surprised that the importance ratings given by the experienced group were quite consistent with those given by the unexperienced group. That is, given such large sample sizes (i.e., n > 30) for the experienced group (n = 104) and the unexperienced group (n = 173), there was only one significant difference in comparisons among the items by t test. The similarities and differences between these groups are detailed as follows.

For the first research question concerning general criteria, the results showed that participants generally agreed with the importance of all 16 assessment criteria included; the means of their answers ranged from 4.04 (between somewhat important and important) to 5.58 (between important and strongly important). The criteria that were agreed to be most important included respondents’ actual behaviors and the type of misconduct, status and roles in the instances, original will concerning involvement in misconduct, the extent of potential negative effects on society and individuals, and the extent of potential interest and benefits from misconduct. This result means that identifying the type(s) of misconduct is considered very important because the Taiwanese MOST and MOE focus on a wider range of misconduct types than traditional FFP. Moreover, the results indicate that the judgment of severity pertains not only to the misconduct (behavior) itself or the people involved (respondents) but also to the potential effects (impacts) caused by this misconduct. Relatively speaking, the respondent’s level of academic qualifications (e.g., academic rank and past research experience), past research integrity training, and the question of whether such misconduct is under investigation by others might not be taken into serious consideration. It is interesting to find that the criterion regarding training in research integrity is ranked important (mean = 4.06) but in a lower order position (15/16 in this category). This is probably because almost all Taiwanese junior researchers (graduate students, researchers first applying for MOST grants and their team members) have been mandated to receive at least 6 hours of RCR education since 2014. Most senior researchers are required or strongly encouraged to attend related training sessions by their respective institutions. In other words, no researcher in Taiwan would be considered totally ignorant or unknowledgeable about research integrity; rather, the question is how much training they have received.

Although minor differences existed in the rankings of means between the experienced group and the unexperienced group, the two groups’ agreement levels concerning the importance of each general criterion were similar. The interesting, major difference pertained to criterion 7: original will with respect to participation in misconduct. The inexperienced group tended to consider this item to be more important for incorporation in The Criteria than did the experienced group. It is noteworthy that the example question (see Appendix) for this criterion was: how was the respondent involved in the instance of wrongdoing – proactively, passively, or coercively? Therefore, the unexperienced group, which was probably composed of younger researchers, may have paid more attention to – or had more sympathy for – passive or coercive participants (i.e., students or younger researchers) in the misconduct case and thus considered this criterion to be more important to take into account.

In terms of the consensus concerning the importance of these general criteria, a high degree of consensus was reached for criteria 1, 2, 3, 4 and 7, and a lower degree of consensus was reached for criteria 15 and 16 among participants in the current study. Therefore, by strictly evaluating both the mean and the associated k-value of each criterion by the all-participant group, and considering that all Taiwanese researchers are already required to take relevant RCR training, the authors suggest deleting the last three criteria (14, under other investigation, 15, training, and 16, level of academic qualifications) in Section 1 to make The Criteria shorter if necessary.

To answer the second question concerning possible aggravation, participants indicated that the frequency and duration of respondents’ noncompliance in past research were important. This finding is consistent with the criteria suggested by the NSF policies (2002) and the Netherlands Code (2018). Moreover, whether the following situations were relevant should be considered: interfering with the investigation process, violating criminal laws, and seriously harming affiliated institutions in any aspect. All four items had high degrees of consensus (k-values equal to or higher than 3.70). The results also show high levels of agreement within the experienced group and the unexperienced group and no significant difference between these two groups. In sum, our participants unanimously considered the four aggravating criteria to be important in The Criteria.

Third and finally, participants had general agreement but more variable perceptions regarding the degree of importance of mitigating criteria compared with their views on other criteria. Sample criteria include the actions of correction or withdrawal prior to the allegation, confession of the wrongdoing prior to or during the allegation, and the degree of deviation from commonly accepted research practices. The results imply that some participants might have a positive but relatively conservative attitude toward mitigation opportunities given to respondents in research misconduct instances. It is also probable that some participants believed there is little room left for mercy when judging the degree of severity for subsequent sanctions for an established instance of research misconduct.

Implications

First and foremost, during the process of collecting research data, certain participants mentioned orally that handling and judging cases of research misconduct are more complicated than they expected. They also voiced a demand for operationally credible resources for facilitating the process. Hence, based on the current results, the authors believe that The Criteria have drawn attention from researchers, achieved a consensus, and are ready for use in actual cases in Taiwan. For example, research institutions and research funding agencies (e.g., the MOST) can simply adopt The Criteria as a resource for assessing research misconduct. Such agencies can also adopt The Criteria as a reference when developing their own guidelines to better meet their needs.

Moreover, since our results indicated that most participants who were experienced in handling such cases did not receive any formal or informal training before they did so, the authors believe that The Criteria may be used in practice and are suitable to function as training material for enhancing the capabilities of academics. For example, The Criteria can be used to perform simulation exercises for case studies. By providing authentic case scenarios (e.g., adapted from high-profile cases as mentioned above), the logic and comprehensiveness of The Criteria can be converted into tacit knowledge for participants in training and can become a mental decision-making strategy or “the ruler in the heart,” following the Chinese proverb for making judgments more substantial and efficient, even if participants do not have The Criteria on hand.

Conclusion, limitations and future studies

To enhance the quality of the assessment of research misconduct and to better formulate subsequent sanction decisions, the authors developed a new resource, The Assessment Criteria for Research Misconduct (The Criteria, Hsueh, Pan, Chou 2019). In the current study, the perceived importance of The Criteria among nearly 300 Taiwanese researchers was investigated, and the results showed that these researchers presented a positive attitude toward the new resource. Hence, The Criteria seem to be ready for practical use.

The study has at least two limitations. First, this study compared the experienced group and the unexperienced group without considering other background factors, such as type of research institution, academic rank, years of serving in academia, or research field. Future studies could focus on the effects of these background factors on the consensus of The Criteria.

Second, The Criteria Survey collected only quantitative data among self-selected samples. Although the results have shown a high degree of consensus concerning the importance of each criterion among all surveyed participants, future studies can employ qualitative research methods to continuously revise The Criteria and enrich the example considerations and questions. The authors especially suggest the use of focus group interviews in which young researchers, senior researchers, institutional RIOs, and representatives from government agencies can have meaningful dialogs on the assessment of research and research misconduct.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by National Yang Ming Chiao Tung University [Higher Education Sprout Project]; Ministry of Education, Taiwan [Center for Taiwan Academic Research Ethics Education]; Ministry of Science and Technology, Taiwan [MOST109-2745-V-009-001-MY2,MOST110-2511-H-A49-008-MY4].