JPA Promotes Open Science

The scientific endeavor is one in which the capacity to replicate findings is a foundational tenant. It is unfortunate that there have been a number of highly publicized cases over the past decade of scientists who have not practiced science with the required levels of intellectual integrity. To help address this problem, the Center for Open Science (COS) was founded, “to increase openness, integrity, and reproducibility of research” (retrieved from https://cos.io/about/mission/ on March 7, 2017). As part of their efforts, a series of standards were created in 2014 under the direction and guidance of recognized discipline leaders, journal editors, and funding agency representatives. These guidelines were also incentivized, so that authors of peer-reviewed research could have their efforts publicly recognized for having complied with the newly created standards.

The standards can be implemented at one of three levels of complexity and are provided in Table 1. As shown, engagement in the standards is provided across Levels 1, 2, and 3. For instance, at Level 1, authors can provide information about where their database is stored and how other scientists can access the data set for independent verification or replication. It also gives journals a number of degrees of freedom about the extent to which they will ask authors to comply with the provision of data analysis plans and whether the study was preregistered on another site (often required with grant-funded studies). By contrast, Level 3 requires the most intense level of compliance for journal authors, and journals using the Level 3 standards require authors to comply with these guidelines. Journal participation in these standards is completely voluntary, although for those journals that select participation—and for those authors who comply with the journals’ standards—a badge can be awarded, which usually is printed on the first page of the article to demonstrate compliance and commitment to open science.

Table 1. Levels of transparency as provided by the Center for Open Science.

	Level 0	Level 1	Level 2	Level 3
Citation Standards	Journal encourages citation of data, code, and materials, or says nothing.	Journal describes citation of data in guidelines to authors with clear rules and examples.	Article provides appropriate citation for data and materials used consistent with journal's author guidelines.	Article is not published until providing appropriate citation for data and materials following journal's author guidelines.
Data Transparency	Journal encourages data sharing, or says nothing.	Article states whether data are available, and, if so, where to access them.	Data must be posted to a trusted repository. Exceptions must be identified at article submission.	Data must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication.
Analytic Methods (Code)Transparency	Journal encourages code sharing, or says nothing.	Article states whether code is available, and, if so, where to access them.	Code must be posted to a trusted repository. Exceptions must be identified at article submission.	Code must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication.
Research Materials Transparency	Journal encourages materials sharing, or says nothing.	Article states whether materials are available, and, if so, where to access them.	Materials must be posted to a trusted repository. Exceptions must be identified at article submission.	Materials must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication.
Design and Analysis Transparency	Journal encourages design and analysis transparency, or says nothing.	Journal articulates design transparency standards.	Journal requires adherence to design transparency standards for review and publication.	Journal requires and enforces adherence to design transparency standards for review and publication.
Preregistration of studies	Journal says nothing.	Article states whether preregistration of study exists, and, if so, where to access it.	Article states whether preregistration of study exists, and, if so, allows journal access during peer review for verification.	Journal requires preregistration of studies and provides link and badge in article to meeting requirements.
Preregistration of analysis plans	Journal says nothing.	Article states whether preregistration with analysis plan exists, and, if so, where to access it.	Article states whether preregistration with analysis plan exists, and, if so, allows journal access during peer review for verification.	Journal requires preregistration of studies with analysis plans and provides link and badge in article to meeting requirements.
Replication	Journal discourages submission of replication studies, or says nothing.	Journal encourages submission of replication studies.	Journal encourages submission of replication studies and conducts results blind review.	Journal uses Registered Reports as a submission option for replication studies with peer review prior to observing the study outcomes.

Note. © Center for Open Science. Reproduced by permission of Center for Open Science. Permission to reuse must be obtained from the rightsholder.

To participate in this process, one has to apply to become a signatory, and at the current time, more than 3,000 journals have become signatories; however, other groups can become a signatory as well. At the September 2016 meeting of the Board of Trustees of the Society for Personality Assessment, Dr. Christopher Hopwood presented the mission of the COS to the Board, and suggested that the society consider becoming a Level 1 signatory. The Board unanimously agreed. I voted in favor of this initiative, and indicated that I would look into adapting the same standards for JPA. Consequently, I spoke with staff at the COS, as well as another journal editor who had adopted these standards more than 1 year earlier. I also enlisted the input of Dr. Hopwood and Dr. Richard Handel to evaluate how this transition would occur and how to implement the process. I also solicited the input of all associate editors, who were favorable about this change.

With this editorial, I am announcing the implementation of this process effective May 1, 2017. It should first and foremost be noted that authors submitting papers to JPA are not required to participate in the transparency guidelines, although JPA is certainly encouraging it. The guidelines and process for submissions are presented at www.tandfonline.com/loi/hjpa20. However, the instructions are also listed in Table 2. Please note that the process of awarding the badges will occur after a paper has been accepted for publication. At that time, authors will have the opportunity to address any remaining issues that will be required to meet the guidelines. The badges are fully described at the COS Web site; however, here is a brief summary of what the criteria are for them to be awarded:

•	The Open Data Badge is awarded when researchers make their data set publicly available for others to reproduce the reported results.
•	The Open Materials Badge is awarded when researchers make their materials publicly available to others to directly replicate their study.
•	The Preregistered Badge is awarded when researchers make their preregistered design and analysis plan publicly available so that it can be compared with the published results.

Table 2. Steps toward receiving Transparency Badge.

1. When an author's paper is accepted for publication, s/he will be asked in the acceptance letter whether or not they wish to receive a badge acknowledging their participation in Level 1 transparency as defined by the Center for Open Science.

2. If author agrees, s/he needs to provide evidence that the data and data analytic plan have been uploaded on an approved site. Examples of these sites include

• http://socialscienceregistry.org/

• http://openscienceframework.org/

• http://egap.org/design-registration/

• http://ridie.3ieimpact.org/

3. Once there is certification of the uploaded materials, the author informs the JPA Editor.

4. Authors confirm that in their accepted manuscript they have documented in a footnote where the uploaded materials can be found.

5. The JPA Editorial Office and publishers will verify the authors’ reports and will award the badge, which will be on the first page of the article.

Although this change will be novel, it certainly is congruent with the almost 80-year history of JPA, which has sought to publish the highest quality research conducted with the highest levels of integrity. I trust the guidelines presented here and on the Web site will be clear to all authors interested in participation. I look forward to promoting this initiative and hope that JPA authors will become leaders in the field by supporting formalized efforts to promote scientific transparency and integrity. For those who have any further questions, please feel free to send an e-mail to the journal office at [email protected] or the COS at [email protected].