Gambling researchers’ use and views of open science principles and practices: a brief report

ABSTRACT

Scientists across disciplines have begun to implement ‘open science’ principles and practices, which are designed to enhance the quality, transparency, and replicability of scientific research. Yet, studies examining the use of open science practices in social science fields such as psychology and economics show that awareness and use of such practices often is low. In gambling studies research, no studies to date have empirically investigated knowledge of and use of open science practices. In the present study, we collected survey data about awareness and use of open science practices from 86 gambling studies research stakeholders who had attended a major international gambling studies conference in May 2019. We found that – as hypothesized – a minority of gambling research stakeholders reported: 1) either some or extensive experience using open science research practices in general, and 2) either some or regular experience using specific open science practices, including study pre-registration, open materials/code, open data, and pre-print archiving. Most respondents indicated that replication was important for all studies in gambling research, and that genetic, neuroscience, and lab-based game characteristic studies were areas most in need of replication. Our results have important implications for open science education initiatives and for contemporary research methodology in gambling studies.

KEYWORDS:

• Gambling
• research design
• reliability
• best practice
• survey

Introduction

Efforts to promote open science have advocated for greater endorsement of open science principles related to research development, reporting, and access among gambling researchers (e.g. Blaszczynski & Gainsbury, 2019; Louderback et al., 2021). Such principles suggest that (1) research development should be transparent; (2) research reporting should be complete, and not dependent upon outcomes; and (3) research access should be open. Practices that support these principles include, but are not limited to, use of research pre-registration and registered reports (i.e. public documentation of, or peer review of, research methods and analytic plans prior to commencing a study) for transparency of development, clear separation of prespecified and ad hoc analyses for completeness of reporting, and freely available materials, studies, and data (i.e. unrestricted availability of research components and products) for research access (see Nosek et al., 2015). Recent efforts to make open science more widespread are in response to observations of questionable research practices (e.g. p-hacking, or use of analytic approaches to produce a preferred p-value, and HARKing, or hypothesizing after results are known; Bishop, 2019; Kerr, 1998; Wicherts et al., 2016) and poor research replicability in the published behavioral research literature (see Open Science Collaboration, 2015; also see Klein et al., 2018). Notably, recent research suggests that rigorous adoption of open science principles and practices is associated with high replicability in novel behavioral research (Protzko et al., 2020), suggesting that increased adoption of such approaches might hold the potential to favorably impact the gambling studies research literature.

Research about open science beliefs and practices

Open science only recently has begun to gain widespread interest in the scientific community (Banks et al., 2019). Studies documenting researchers’ beliefs and practices related to open science tend to report limited experience, but growing interest in practices such as data sharing (e.g. Abele-Brehm et al., 2019; Houtkoop et al., 2018), open peer review (e.g. Ross-Hellauer et al., 2017), and open access publication and paper repositories (Creaser et al., 2010; Rodriguez, 2014; Rowley et al., 2017; Xia, 2010), among others. Additional studies have examined facets of open science in practice, for example, showing that adherence to pre-registrations is typically not perfect (Claesen et al., 2019) and that liberal researcher degrees of freedom (i.e. methodological flexibility inherent in research design or analytic plans; Wicherts et al., 2016) remain common in pre-registered studies (Veldkamp et al., 2017). However, to date, there have been no studies of gambling researchers’ understanding or use of open science principles and practices.

Open science beliefs and practices among gambling researchers

The gambling studies field is not entirely absent of open science practices. For example, during 2009, the Division on Addiction at Cambridge Health Alliance, with funds from the online gambling operator bwin.party, created an open data archive, The Transparency Project (Shaffer et al., 2009). Upon its creation, the Division on Addiction used The Transparency Project to share its industry-funded player data research datasets and improve the transparency of its works. This early instance of open science facilitated scientific progress – allowing independent researchers to publish empirical research on gambling that they otherwise could not (e.g. Brosowski et al., 2012; Coussement & De Bock, 2013; Percy et al., 2016). However, this effort did not stimulate widespread discussion of open science principles and practices among gambling researchers. In fact, despite a growing recent awareness of and literature pertaining to issues including research replication (Klein et al., 2018), scientific transparency (McNutt, 2016), and the need to embrace open science (Frankenhuis & Nettle, 2018; Munafò et al., 2017; Nosek et al., 2015; Open Science Collaboration, 2015), additional published discussion of the need for scientific self-reflection and adoption of open science tactics among gambling researchers did not occur until about a decade later (Blaszczynski & Gainsbury, 2019; Heirene, 2020; Heirene & Gainsbury, 2020; LaPlante, 2019, 2020; LaPlante & Gray, 2019; Louderback et al., 2021; Wohl et al., 2019).

Within this context, understanding how well gambling researchers understand open science practices and their value to the research process can provide insight into gaps between actual and ideal research practices in this field. To gain a preliminary understanding of how well open science is integrated into gambling research, we collected primary data from a convenience sample of gambling research stakeholders who presented or coauthored presentations at a major international conference on gambling research during 2019 with a survey that measured experience with open science and related practices.

The present study

The present study was primarily descriptive and includes some exploratory comparisons; however, because academic discussion of open science practices is limited among gambling research stakeholders, we hypothesized that:

H1: A minority of respondents will endorse that they have some or extensive experience with open science principles, generally.

H2: A minority of respondents will endorse that they have some or regular experience with specific types of open science practices.

H3: A minority of respondents will endorse that the concept of replicability is relevant to all gambling studies.

Methods

We pre-registered our study protocol on the Open Science Framework (https://osf.io/xq2b6). The Cambridge Health Alliance Institutional Review Board reviewed and approved this study (exemption granted, 45 CFR 46.104(d)(Category 2(i)).

Participants

Our initial list of participants included all possible gambling conference registrants who presented or coauthored presentations at the 17th International Gambling and Risk Taking Conference that took place in Las Vegas, Nevada, USA during May 2019 (N = 331). This conference markets itself as the largest in the field of gambling studies and includes both U.S.-based and international scholars, researchers and other gambling stakeholders (University of Nevada, Las Vegas, 2019). The presenter population from which we sampled was diverse in academic interests; the conference hosted presentations across multiple disciplines related to gambling, such as history, business, social sciences, and mathematics (Digital Scholarship at UNLV [DSUNLV], 2019). Because this study primarily was exploratory, we did not complete a power analysis, but instead sought to enroll all possible of these registrants. Upon obtaining the list of potential survey recipients and removing invalid e-mail addresses as well as e-mail addresses for authors of the present study, we distributed the survey invitation, consent form, and survey using Qualtrics to 315 potential respondents beginning 30 April 2020. We also used the Qualtrics survey system to send three reminder e-mails, one per week after the initial e-mail invitation. We stopped collecting data on 29 May 2020, one week after sending out the final reminder e-mail. We did not reimburse respondents for their participation. Of the 315 total individuals sampled, 86 people responded to the survey, representing a response rate of 27.3% (86/315 = 0.273).

Measures

Our survey was an adaptation of the Beaudry et al. (2019) Swinburne Open Science Survey (see survey questionnaire on pp. 7–12 in our pre-registration: https://osf.io/xq2b6) and included the following domains: (1) General experience with open science practices; (2) Experience with pre-registration; (3) Concerns with pre-registration; (4) Experience with open materials/code; (5) Concerns with open materials/code; (6) Experience with open data; (7) Concerns with open data; (8) Experience with pre-print archiving; (9) Concerns with pre-print archiving; (10) Feelings about replicability; (11) Areas in gambling studies in need of replication; (12) Job type; (12a) Academic job experience; and, (13) Country of residence.

We also collected data on participants’ thoughts and opinions related to open science and gambling studies with two open response questions:

1. Do you have any other thoughts or opinions about open science principles or practices that you would like to share?

2. Do you have any other thoughts about the current state of research in the field of gambling studies that you would like to share?

Analytic strategy

We completed descriptive analyses of all survey items. To examine our hypotheses, we recorded whether a minority or a majority reported some or extensive/regular experience with general and specific open science practices. Likewise, for the item that addressed the concept of replicability, we reported whether a minority or a majority reported that the concept of replicability is relevant for all gambling studies.

We used Fisher’s exact tests to examine relationships between open science practices and concerns, and the following categories: (1) job type (i.e. primarily academic or primarily non-academic); (2) academic job experience (i.e. developing, early, mid, later, and late career); (3) region of residence (using the United Nations Geoscheme; https://www.emiw.org/fileadmin/emiw/UserActivityDocs/Geograph.Representation/Geographic-Representation-Appendix_1.pdf, i.e. Africa, Americas, Asia, Europe, and Oceania); and (4) research productivity quartile (using the Scimago Institutions Ratings of scientific research productivity by country as of 9/10/2020; quartile 1 represents the most productive countries; https://www.scimagojr.com/countryrank.php?order=h&ord=desc).¹ We used the standard two-tailed p < 0.05 criterion for determining if each test was statistically significant.

To analyze the two open response questions, we created a word frequency cloud for each question using the wordcloud() package in R (version 3.6.2). We reported the top 30 most used words that have substantive meaning in each word cloud (i.e. excluding articles including: a, I, and, the, you/your, etc.).

Results

Results related to our assessment of hypotheses are reported here, and a full listing of results is available on our Open Science Framework project page in our online Supplemental Findings document (https://osf.io/qrjnd/).

Experience with open science principles and practices

Table 1 shows that a minority of respondents reported some or extensive experience using open science practices in their own research, confirming Hypothesis 1. Likewise, our examination of specific open science practices indicated that for all practices considered, a minority of respondents reported some or regular experience using a particular practice in their own research. Therefore, Hypothesis 2 also is confirmed.

Table 1. Percentage of respondents who indicated some or extensive/regular experience with aspects of open science

Variable (n)	Valid %
I have some or extensive/regular experience with …
Open Science Practices (86)	44.18
Study pre-registration (86)	31.40
Open materials/code (83)	32.53
Open data (83)	48.19
Pre-print archiving (82)	15.86

Concerns with open science principles and practices

In preplanned exploratory analyses, we examined stakeholders’ concerns about specific open science practices. Table 2 shows that minorities of respondents endorsed each concern across all practices. Pluralities suggested the following primary concerns: (1) pre-registration: I need to look at my data before I can decide how to best analyze it; (2) open materials/code: There could be issues related to intellectual property; (3) open data: There could be issues related to privacy; and, (4) pre-print archiving: Non-peer reviewed findings might add noise to the literature. Proportions of respondents who indicated they had no concerns ranged from 22.89% for open data to 31.33% for open materials/code.

Table 2. Top three concerns by open science practice

Variable (n)	Valid %
Concerns with study pre-registration (86)
Need to look at data before deciding how to best analyze it	33.72
Stifles research creativity or flexibility	25.58
Others might take ideas	23.26
Concerns with open materials/code (83)
Issues related to intellectual property	37.35
Might lose control over materials/code	34.94
Might not receive appropriate credit	24.10
Concerns with open data (83)
Issues related to privacy	44.58
Others might use my data for another study	33.73
Might lose control over how data are used	32.53
Concerns with pre-print archiving (82)
Might add noise to the literature	37.80
Journals might not publish findings if there is a pre-print	36.59
Others might copy my ideas & Might reveal differences in pre-print and publication	17.07

This table shows the top three concerns reported for each of the four specific open science practices (i.e. pre-registration, open materials/code, open data, and pre-print archiving) in descending order.

Importance of replicability

We observed that 51.22% of respondents who provided their opinion (n = 82) suggested that replicability is relevant for all gambling studies. Hypothesis 3 is not confirmed. From greatest to least, proportions of respondents (n = 69) ranked these areas first as most in need of replication: (1) genetics (18.84%), (2) lab-based game characteristic studies (15.94%), (3) neuroscience (14.49%), (4) prevalence surveys, public policy impacts, & responsible gambling program evaluations (8.70% each), (5) problem gambling measurement tools (7.25%), (6) socio-economic impact studies (5.80%), (7) responsible gambling tools & other studies (4.35% each), and (8) business research (2.90%).²

Bivariate analyses

In preplanned exploratory analyses, we examined relationships between open science practices and concerns and key respondent categorization groupings. First, we examined job type (n = 82). Most respondents reported having an academic-related occupation (74.39%). We observed no differences in five open science experiences by job type. We also observed that for 1 of 34 open science concerns (i.e. concern that open materials/code could violate epistemology; 1 df; p < 0.05; φ = 0.24) there was a statistically discernible effect. Specifically, 5 out of the 21 (23.8%) respondents from non-academic jobs thought this was a concern as compared to only 4 out of 61 (6.6%) academic job respondents.³ There also was no difference in opinions about the relevance of replicability to gambling studies by job type.

Second, we examined academic job experience. Among those who indicated their academic experience (n = 61), a plurality of respondents (39.34%) indicated that they were 11–30 years post terminal degree. We observed that for 1 of 5 open science experience analyses (i.e. experience with pre-registration; 12 df; p < 0.05; V = 0.34), there was a statistically discernible effect. Specifically, only one respondent who was in graduate school for a Doctoral degree (11.1% within that category) and one respondent who was 1–10 years post-terminal degree (5.6%) reported not being aware of study pre-registration, as compared to 12 people who were 11–30 years post-terminal degree (50.0%) and 3 people who were 31–40 years post-terminal degree (42.9%). Likewise, for 2 of 34 open science concerns (i.e. concern that pre-registration stifles research creativity or flexibility; 4 df; p < 0.05; V = 0.45; and, concern that pre-prints might allow others to copy one’s ideas; 4 df; p < 0.05; V = 0.51), there was a statistically discernible effect. Specifically, 45.8% of respondents (n = 11) who were 11–30 years post-terminal degree thought pre-registration could stifle creativity vs. only 11.1% (n = 1) for respondents who were in graduate school for a Doctoral degree and 5.6% (n = 1) for respondents who were 1–10 years post-terminal degree. Moreover, 41.7% of respondents (n = 10) who were 11–30 years post-terminal degree thought that pre-prints might allow others to copy one’s ideas, as compared to no respondents who were in graduate school and no respondents who were 1–10 years post-terminal degree who voiced this concern. There also was no difference in opinions about the relevance of replicability to gambling studies by academic job experience.

Results from the analyses of views of and concerns with open science practices and replicability by region and research productivity, as well as the two word clouds, are reported in the online Supplemental Findings document (https://osf.io/qrjnd/). Importantly, we note that the region and research productivity analyses had very small cell counts for running inferential tests and are reported for sake of completeness, but should be interpreted with caution. The word clouds showed that terms related to transparency occurred frequently (‘open’, ‘sharing’, ‘available’, ‘guidelines’), as did terms related to potential limitations (‘barriers’, ‘problem’), and different stakeholder groups (‘researchers’, ‘journals’, ‘public’, ‘companies’, ‘casinos’).

Discussion

In this study, we surveyed 86 stakeholders from a major gambling studies conference to better understand the extent to which respondents were aware of open science practices, potential concerns related to open science, and views regarding research replicability. We found that although many respondents were aware of open science in a general sense and some open science practices specifically, only a minority of respondents had used open science practices in their own research. Most gambling researchers viewed replication as important for all studies, suggesting that there is considerable interest in replication for the existing academic literature. Exploratory analyses examining open science experience and open science concerns identified few differences by job type and academic job experience.

Overall, our findings suggest a fairly broad need for open science education among gambling researchers. Specific areas of need include addressing concerns that open science might prevent research flexibility, lead to a loss of credit for important research and research materials, and the possibility of degrading the research literature by circumventing peer-review. Training and practical exposure to open science practices should make clear that tools are available to address many of these concerns already. For example, publishing timestamped ‘transparent change’ documents alongside research pre-registrations allows researchers to maintain analytic flexibility and innovation in real time (see an example of a transparent change document here: https://osf.io/25xr9/). Likewise, new citation practices for open data can provide new avenues for publicly crediting important research data and associated materials. Finally, preprint servers actually might improve the peer-reviewed literature by providing a clear and open feedback process from a diversity of authors that the existing process currently does not tap.

Implications for gambling studies research

Despite our finding that the majority of respondents thought that replication is important for all gambling studies, there has not yet been a comprehensive examination of the replicability of research findings in this field. This is potentially problematic, because replication rates in gambling studies might mirror those from disciplines such as psychology, which uses similar research methodologies. Such replication rates tend to be alarmingly low. For example, Klein et al. (2018) examined 28 classic and contemporary social psychology effects and found that only 54% of the effects replicated. Recently, a z-curve analysis (Brunner & Schimmack, 2020) indicated evidence of publication bias and an Expected Replication Rate ranging from 0.61 to 0.79 in the gambling product safety literature (McAuliffe et al., 2020).⁴ We note that this range is in line with observations from a number of other social sciences. More replication work – including direct replication of published gambling studies – is needed to understand the validity of the published literature.

We also show that the use of open science practices is limited to a minority of respondents in our sample. This rate of participation is similar to other academic disciplines (e.g. in psychology, see Giofrè et al., 2017; in education, see Sampson et al., 2013). Part of this lack of awareness and use of open science practices might be due to limited education in graduate school and among early career professionals on this topic, but much might also relate to the absence of related continuing education opportunities for mid-career researchers. It remains to be determined whether the issues identified as limiting other areas of behavioral science (e.g. poor replication rates), including psychology, economics, marketing and more, also affect gambling studies. However, given the overlap of research methods and theoretical underpinnings, we suggest that open science education, across all topics, should be more widespread among aspiring and established scholars alike (see Banks et al., 2019; Schönbrodt, 2019). Additionally, the unexpected finding that a majority of respondents view replication as important for all types of gambling studies suggests that gambling studies researchers might support a large-scale replication initiative – especially for topics such as genetics, lab-based game characteristic studies, and neuroscience. Such an undertaking would not be easy, but as LaPlante (2019) and Wohl et al. (2019) argue, it is essential that gambling studies evaluate the replicability of its literature because a considerable body of policies rest upon its empirical research findings.

Study limitations

Of course, our study is not without limitations. First, although our sample was based on a population of gambling stakeholders from a major conference, it was still a convenience sample and might not represent the views of all gambling stakeholders. Second, the response rate was middling, so there might be selection bias in the sample. In particular, potential respondents who responded to the survey might be different than those who did not respond to the survey. Third, our population included attendees from only one conference on gambling studies in the United States, so it might not be representative of other populations of gambling researchers such as those who typically attend conferences in Europe, Asia, or other areas. Fourth, we completed the online survey during the COVID-19 pandemic, so our response rate and responses might have been influenced by this major global health crisis. Fifth, our preliminary description of open science practices and opinions relate to the questions we employed, so other constructions (e.g. other response options about areas most in need of replication) might yield a different picture. Sixth, our sample size was small, which likely limited our power to detect statistically significant effects (e.g. for bivariate analyses, which mainly suggested no discernable differences among groups).

Concluding thoughts

Open science provides numerous benefits for the scientific process, including enhancing transparency, researcher independence, objectivity, and scientific rigor. Yet, this approach is relatively new, and much remains to be discovered about how to apply open science principles and practices to all types of research. Gambling studies are diverse, containing multiple disciplines. Some aspects of open science might be easier to apply to some disciplines than others. Facilitating additional empirical research about the use of open science among gambling studies researchers will help the field better understand knowledge gaps for education planning and help identify disciplines that might need open science innovations to engage effectively. Conversely, future work might explore how gambling studies might inform the next generation of open science principles and practices. Nonetheless, by embracing open science principles and practices, gambling studies can reexamine its key findings and potentially experience similar benefits as other fields, advancing reliable findings and moving past those that are not. These benefits are particularly important given the clinical and policy implications of many findings in gambling studies.

Open scholarship

This article has earned the Center for Open Science badges for Open Data, Open Materials and Preregistered. The data and materials are openly accessible at https://osf.io/7dgnm/.

Acknowledgements

We would like to acknowledge and graciously thank the anonymous survey respondents who took part in our study. We appreciate your interest in participating in research about open science and transparency in gambling studies.

Funding sources

This work was supported by Entain PLC. Study pre-registration, statistical analyses, data interpretation, and manuscript preparation were conducted without any input from Entain PLC. We did not require approval from Entain PLC to publish this study.

Data availability statement

The data described in this article are openly available in the Open Science Framework at https://osf.io/7dgnm/.

Disclosure statement

When this article was published, the Division on Addiction was receiving funding from EPIC Risk Management; the Foundation for Advancing Alcohol Responsibility; Entain PLC (formerly, GVC Holdings PLC); Massachusetts Department of Public Health, Bureau of Substance Addiction Services via St. Francis House; Massachusetts Department of Public Health, Office of Problem Gambling Services via Health Resources in Action; MGM Resorts International via the University of Nevada, Las Vegas; National Institutes of Health (National Institute of General Medical Sciences and National Institute on Drug Abuse) via The Healing Lodge of the Seven Nations; Substance Abuse and Mental Health Services Administration via the Addiction Treatment Center of New England; and Substance Abuse and Mental Health Services Administration via the Gavin Foundation. During the 5 years prior to the publication of this article, the Division on Addiction has also received funding from ABMRF/The Foundation for Alcohol Research; Caesars Enterprise Services, LLC; Danish Council for Independent Research via Aarhus University Hospital; the David H. Bor Library Fund, Cambridge Health Alliance; DraftKings, Inc.; Fenway Community Health Center, Inc.; Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals, Hong Kong; Massachusetts Department of Public Health, Bureau of Substance Addiction Services; and the Massachusetts Gaming Commission, Commonwealth of Massachusetts.  During the past 5 years, the International Gaming Institute has received research funding from MGM Resorts International, Wynn Resorts Ltd., Las Vegas Sands Corporation, Caesars Entertainment Corporation, Ainsworth Game Technology, U.S.-Japan Business Council, State of Nevada, Knowledge Fund, and State of Nevada Department of Health and Human Services. IGI runs the triennial research-focused International Conference on Gambling and Risk Taking, whose sponsors include industry, academic, and legal/regulatory stakeholders in gambling. A full list of sponsors for the most recent conference can be found at https://www.unlv.edu/igi/conference/17th/sponsors.  During the past 5 years, Debi A. LaPlante has served as a paid grant reviewer for the National Center for Responsible Gaming (NCRG; now International Center for Responsible Gaming [ICRG]), received travel funds, speaker honoraria, and a scientific achievement award from the ICRG, has received speaker honoraria and travel support from the National Collegiate Athletic Association, received honoraria funds for preparation of a book chapter from Universite Laval, received publication royalty fees from the American Psychological Association, and received course royalty fees from the Harvard Medical School Department of Continuing Education. Dr. LaPlante is a non-paid member of the New Hampshire Council for Responsible Gambling.  During the past 5 years, Eric R. Louderback has received research funding from a grant issued by the National Science Foundation (NSF), a government agency based in the United States. Dr. Louderback’s research has been financially supported by a Dean’s Research Fellowship from the University of Miami College of Arts & Sciences, who also provided funds to present at academic conferences. He has received travel support funds from the Hebrew University of Jerusalem to present research findings.  During the past 5 years, Brett Abarbanel has received funding from the Manitoba Gambling Research Program, GP Consulting, U.S.-Japan Business Council, Wynn Las Vegas, Victoria Responsible Gambling Foundation, Connecticut Council on Problem Gambling, Bermuda Casino Gambling Commission, the States of Nevada and California, Canadian Partnership for Responsible Gambling, iDevelopment and Economic Association, GLG Consulting, Majestic Star Casinos, MGM Resorts International, Marina Bay Sands, ProPress Germany, and Caesars Entertainment. Dr. Abarbanel has received reimbursement for travel from Association Cluster Sport International, Kansspelautoriteit, Gamification Group (Finland), British Columbia Lottery Corporation, International Association of Gaming Advisors, GambleAware, Las Vegas Convention and Visitors Authority, Ultimate Media Ventures, Canadian Partnership for Responsible Gambling, IGT Latin America, University of Salford, and the National Collegiate Athletic Association (USA). During this same time period, Dr. Abarbanel was a member of the Singapore National Council on Problem Gambling International Advisory Panel, for which she was reimbursed for her time.

Supplemental data

Supplemental data for this article can be accessed here.

Additional information

Notes on contributors

Debi A. LaPlante

Debi A. LaPlante is Director of the Division on Addiction, Cambridge Health Alliance, a Harvard Medical School teaching hospital and an Assistant Professor of Psychiatry at Harvard Medical School. With her colleagues, she developed the Syndrome Model of Addiction and engaged in a seminal 10-year research program on Internet gambling.

Eric R. Louderback

Eric R. Louderback is a Research & Evaluation Scientist at the Division on Addiction, Cambridge Health Alliance, a Harvard Medical School teaching hospital. He has expertise in diverse topic areas including the analysis of data from actual internet gamblers, algorithms to predict risky gambling behavior, and responsible gambling program evaluation.

Brett Abarbanel

Brett Abarbanel is Director of Research at University of Nevada, Las Vegas, International Gaming Institute, with an affiliate appointment at University of Sydney, Australia. Her research covers gambling technology and policy, responsible gambling, and esports and video games, for which she is a founding director of the Nevada Esports Alliance.

Notes

1. Results related to region of residence and research productivity, as well as the two word clouds (see pp. 27–28 in supplement), are available in the online supplement (https://osf.io/qrjnd/).

2. Although we pre-registered the ranked choice question regarding areas most in need of replication, our reporting of these descriptive statistics was unplanned and exploratory; that is, we had no expectations for the descriptive patterns we might observe.

3. Although we did not pre-register an intention to complete Bonferroni adjustments for our bivariate analyses, we note that employing such an adjustment to each family of tests suggests that we no longer observe any statistically significant effects for the bivariate analyses, except for the analysis of region of residence and experience with study pre-registration, and the analysis of research productivity quartile and the concern that others might ask for assistance with open materials/code. However, Bonferroni is a conservative adjustment that might be too severe for exploratory research (Bender & Lange, 2001).

4. The z-curve is a relatively new approach to assessing the replicability of a given field, or ‘methods for predicting the success rate if sets of significant results were replicated exactly’ (Brunner & Schimmack, 2020, p. 1). In brief, z-curve uses published test statistics from a given field to derive the average power of a set of published studies and estimate statistics such as the Expected Replication Rate and the Expected Discovery Rate. Interested readers should consult Bartoš and Schimmack (2020) for additional details.