The puzzle of sharing scientific data

Figures & data

Figure 1. The research design: a mixed-methods approach to address RQ1and RQ2.

Table 1. Enablers and deterrents for data sharing in science according to prior literature and implications from collective action and epistemic cultures perspectives

Reasons for sharing data from policy and community perspective	Description	Literature	Implications from Epistemic cultures perspective
Reproducibility	Sharing research data and making them easier to peer review increases transparency and opportunities for the reproduction of research findings. It also increases the potential for publishing negative results and enables accurate verifications of research findings.	(Baker 2015; Fecher et al. 2015; Lyon 2016; OECD 2015; Pujol Priego and Wareham 2019; Tenopir et al. 2015)	Data sharing practices may be community-bound because of the epistemic culture of the community. Differences in data sharing practices across scientific communities depend on how community values shape the norms towards integrating the importance of scientific reproducibility, acceleration of scientific progress, scientific quality, efficiency, and innovation into their collective behaviour.
Higher scientific efficiency and progress	The availability of the Gene Expression Omnibus (GEO) database at the U.S. National Center for Biotechnology Information led to more than 1,150 published articles by third-party contributors by the end of 2010.	(Borgman 2015; Lyon 2016; OECD 2015; Pasquetto, Randles, and Borgman 2017; Piwowar, Vision, and Whitlock 2011;Whyte and Pryor 2011)
Higher scientific quality and transparency	Sharing research data is related to the strength of the evidence supporting the results, the quality of the statistical results reporting, and enables transparency and greater scrutiny of research.	(Kupferschmidt 2018; Wicherts et al. 2011)
Reasons for sharing/not sharing data by the individual scientists	Description	Literature	Implications from a Collective Action perspective
Personal credit and rewards	There is a lack of consistency in the way data is cited. The scholarly system is heavily biased towards publications; secondary products such as data or code are rendered far less credit. Relatedly, for scholars driven by credit, sharing data offers little benefit, particularly in light of intentions to try to publish future articles out of the same data, or aggregating it with complementary datasets.	(Borgman 2015; Piwowar, Vision, and Whitlock 2011) (Harley, Acord, and Earl-Novell 2010; Howison et al. 2015; Meijer et al. 2017; Plantin et al. 2018)	Employing a collective action perspective, we would expect that scientists would share the data if they perceive gains from its contribution to the commons by adjusting the values and costs associated with the resource contribution. Data sharing practices would depend on how the community adjusts the costs (e.g. lack of credit, rewards, costs of input metadata) with the potential value of the data contribution.
Misuse, misinterpretation, liability concerns	Uncertainty over who is going to reuse the data and for what purposes and lack of understanding of the data and thus misuse.	(Meijer et al. 2017; Tenopir et al. 2015; Wallis et al. 2013)
Lack of skills	Lack of expertise and knowledge of tools to make their data available.	(Borgman 2015; European Commission 2019; OECD 2015)
Costs to prepare data	The effort and time-consuming activity of providing contextual information and detailed descriptions of the data. Costs could span an estimated two to three weeks from an average of a two-year research grant application.	(Edwards 2010; Holzner, Igo-Kemenes, and Mele 2009; OpenAire 2019)

Table 2. Factors related to data sharing

Variable	Survey question	Relevant factor represented	Type
Data sharing (Dependent variables)
Willingness	I am willing to allow others to access my research data	Data sharing	5-point scale (Strongly disagree to Strongly agree)
Experience	I have previously shared my research data with others	Data sharing	5-point scale (Strongly disagree to Strongly agree)
Enablers: Epistemic culture lense
Reproducibility of research	Reproducibility of research	Reproducibility	Yes/No
Data reuse	Data reuse	Higher scientific efficiency and progress	Yes/No
Research aggregation	Research aggregation	Higher scientific efficiency and progress	Yes/No
Importance of data sharing in field	Sharing research data is important for doing research in my field	Higher scientific quality	5-point scale (Strongly disagree to Strongly agree)
Collaboration	More possibilities for collaboration	Higher scientific quality	Binary
Deterrents: Collective action incentives
Data sharing rewarded in field	Sharing research data is associated with credit or reward in my field	Personal credit/rewards	5-point scale (Strongly disagree to Strongly agree)
Higher paper acceptance	Article more likely to be accepted for publication	Personal credit/rewards	Yes/No
Higher citation	Article more likely to be cited	Personal credit/rewards	Yes/No
Compliance to funding body	Compliance with funding body mandates	Misuse and liability concerns	Yes/No
Compliance with journal/publisher	Compliance with journal or publisher requirements	Misuse and liability concerns	Yes/No
Training	I have received sufficient training in research data sharing	Lack of skills	5-point scale (Strongly disagree to Strongly agree)
Costs	Effort required prior to sharing data	Costs to prepare data	5-point scale (Strongly disagree to Strongly agree)

Table 3. Researcher willingness to share data for various reasons and experience of it

		Willingness	Experience
Epistemic culture	Reproducibility of research	0.287 (0.131)*	0.181 (0.119)
	Data reuse	0.205 (0.118)	0.366 (0.117)**
	Research aggregation	0.111 (0.122)	−0.017 (0.114)
	Importance of data sharing in field	0.565 (0.08)***	0.568 (0.072)***
	Collaboration	0.251 (0.118)*	0.181 (0.113)
Collective action	Data sharing rewarded in field	0.094 (0.053)	0.029 (0.047)
	Higher paper acceptance	−0.199 (0.113)	−0.023 (0.115)
	Higher citation	−0.179 (0.109)	0.007 (0.106)
	Compliance with funding body	0.360 (0.166)*	0.293 (0.172)
	Compliance with journal/publisher	0.034 (0.130)	0.027 (0.130)
	Training	0.091 (0.047)	0.136 (0.046)**
	Costs	−0.048 (0.071)	0.072 (0.070)
	N	491	489
	Test statistic	0.000	0.000
	R^2	0.278	0.143

p-values: * <0.05, ** < 0.01, ***<0.001

Analysis performed in survey data 2018

Figure 2. Data sharing by discipline.

The fields are ordered by their researchers’ average experience sharing data. The fields included are Maths (MATHS), Medicine and Allied Health (MED), Engineering (ENG), Materials Science (MAT SCI), Computer Science (COMP SCI), Social Sciences, Humanities & Economics (SOC HUM ECON), Chemistry (CHEM), Physics & Astronomy (PHY), Life Sciences (LIFE SCI) and Earth & Environmental Science (ENV). The responses of ‘N/A’ and ‘Neither agree nor disagree’ were left out for clarity. Width relates to sample size. + denote significant difference to at least one other field after posthoc Dunn test without adjustment (p-value<0.05). * denote fields that continue to have significant differences to at least one other field after posthoc dunn test with Holm adjustment (p-value <0.05).

Figure 3. Data sharing outcomes across all fields.

Plots show percent responses across researchers from all fields. Asterisks denote p-value after Kruskal–Wallis test (* <0.05, ** < 0.01, ***<0.001) comparing different fields in their responses. The type of data shared is less than 100% due to researchers saying that it is not very important in their field.

Figure 4. Comparison of data sharing among disciplines, highlighting life sciences and physics/astronomy.

For reference, we also show the mean response across all fields. For questions with 5-point Likert scale coded from 0 to 4, we divided the response by 4 for the plot. All the statistical analysis was done with the original (non-transformed) data through Kruskal–Wallis test. Asterisks denote p-value after Kruskal–Wallis test comparing the two fields of Life Science and Astrophysics in their responses. (* <0.05, ** < 0.01, ***<0.001).

Table 4. Data collection sources for both Molecular Biology (MB) and High-Energy Physics (HEP)

	MB – Open Targets	HEP- Reana and related platforms
Primary data sources	13 interviews with scientists and managerial team of OT	5 interviews with scientists and managerial team of Reana and related platforms; and 4 interviews with CERN programmers and data architects.
Observations	Study visit to Genome Campus OT Open Days – workshop, working groups and social event (June 2019)	Study visits to CERN (2018, 2019, 2020). Partner in H2020 funded CS3MESH4EOSC, a constituent project of the European Open Science Cloud https://cordis.europa.eu/project/id/863353, and ATTRACT https://attract-eu.com/. Interviews and discussions with open data related services at CERN (Zenodo, Open Data Portal, CS3-ScienceMesh).
Secondary data sources	41 publications 1 tutorial on OT infrastructure 3 outreach posts; 19 release notes; 6 posts; 7 websites	Experimental data policy and guidelines: CMS, ALICE, ATLAS, LHCb, OPERA data policy; CERN open data terms of use; 22 guidelines in CERN open data portal; CERN Analysis Preservation Portal; Joint declaration and Taskforce documentation on HEP data preservation; Reana workshop presentations June 2018; 12 runnable examples of Reana; 6 publications, 6 release notes, 2 blog posts.

Table 5. Theoretical progression of our analysis

Empirical observations from data sources		Identification of theoretical constructs
MB- Open Targets	HEP- Reana
‘So, we have a platform that is public and open to everybody. Then, for the experimental projects, the partners share the data while they are creating it in Google buckets.’ ‘We have other features that are private, that we do not share with others. Those hidden features allow me, for instance, to work with my compound library on the platform, which I do not share with other OT partners.’	‘Open access to its data by people outside the collaboration can be considered at four levels of increasing complexity.’ CMS experiment preserves ‘the reconstructed data and simulations by keeping available a copy of the data reconstructed with the best available knowledge of the detector performance and conditions for each period of data-taking a virtualised computing environment, compatible with the software version with which the original data can be analysed’ (Dphep Study Group 2009: 7).	Modularity (Mechanism 1)
‘We have an internal internet that we use to say here’s what this data is and you can request it and I will send it to you in a password protected encrypted format. They get it sent and then I send them passwords separately, and then they take it from there. Lots of our partners, they prefer sometimes to use the raw data, so they all go through their own pipeline.’ ‘Once the project themselves have published the data in their own time, once that’s publicly available, then we can link to it on our public platform. In the meantime, though, it’s all very confidential and we don’t share anything outside of the internal platform that we have. It’s up to the project to have that publication before we start sending it out to the world.’	‘New data will enter the portal once the embargo periods for them are over.’ (CERN Open Data Portal) ‘The first data release of 2010 data took place in 2014.’ (R1) ‘The first data release was followed by a full analysis of the procedure, which was endorsed by the Collaboration Board in 2015, and regular data releases, accompanied by appropriate simulated data, each approved by the Collaboration Board, are now taking place.’ (CMS April 2018)	Time delay (Mechanism 2)
‘There is a need to coordinate the integration of data into OT, both from the projects that generate data but also with the data providers such as Chembl and Uniprot and all the data that goes into the platform to keep it up to date. We also work with the developer team that creates some of the features that users will use to visualize the data coming through.’	‘The data preservation process should follow well-defined policies, defined as soon as possible during the lifetime of the collaborations, and possibly embedded in a global HEP data preservation initiative.’	Boundary organisation (Mechanism 3)

Table 6. Similarities and differences observed between Reana (HEP) and open targets (MB)

Similarities (What mechanism)	Differences (How the mechanism is represented)
	HEP – Reana	MB – Open Targets
Modularity (Mechanism 1)	HEP establishes four layers of data: raw data is not released, while more curated versions of data are opened (level 2 in open data portal and reused in Reana; level 1 from publications through HEP library systems).	In MB, raw data from target associations with metadata is released in OT. However, the aggregations with data related to the next steps of the drug discovery process (e.g. proprietary compound libraries) remain closed.
Time Delay (Mechanism 2)	The embargo period of HEP is around 5 to 10 years, depending on the experiments. After the embargo period in HEP, only a % of the data is agreed to be released.	In MB, the time delay between the generation of the data and release in OT is of 18–24 months. In MB, all the data generated is shared in OT infrastructure.
Boundary organisation (Mechanism 3)	The boundary organisation and what makes the interface that mediates the data flows between researchers and establishes the rules, responsibilities, and drivers in data policies varies in the two cases. In HEP, the prominent role is played by the experiment, which decides rights and responsibilities across data. These rules prevail across infrastructures, including Reana. The competition over the data is not between scientists but between experiments.	In MB, the different experimental projects need to comply with the data governance and rules of OT, which establishes the protocols to avoid unintended spillovers and regulates the process to release the data.

Figure 5. Mechanisms that enable HEP and MB researchers to share data. The (+) show significant motivator/deterrent of data sharing from our study.

Table

Survey in this study			UNESCO data			Difference
Fields	Percent among repondents	Grouped	Fields	Percent across all researchers	Grouped
SocSci + Arts Hum + Economics	15.3%	15.3%	Social sciences	14.7%	20.8%	−5.5%
			Humanities	6.1%
Life Sciences	13.8%	38.1%	Natural sciences	18.3%	24.2%	13.9%
Earth & Env. Science	12.9%
Physics & Astronomy	7.0%		Agricultural and veterinary sciences	5.9%
Chemistry	4.4%
Computer Science	7.7%	31.9%	Engineering and tech	41.6%	41.6%	−9.7%
Engineering	15.9%
Material Science	4.1%
Maths	4.3%
Medicine and Allied Health	12.1%	12.1%	Medical and health sciences	13.4%	13.4%	−1.3%
Other	2.6%	2.6%				2.6%
Total	100.0%	100.0%	Total	100.0%	100.0%

Baker, M. 2015. “First Results from Psychology’s Largest Reproducibility Test.” Nature News.

 Google Scholar

Fecher, B., S. Friesike, M. Hebing, and R. S. Phillips. 2015. “What Drives Academic Data Sharing?” PLoS ONE 10 (2): 25. doi:https://doi.org/10.1371/journal.pone.0118053.

 Web of Science ®Google Scholar

Lyon, L. 2016. “Transparency: The Emerging Third Dimension of Open Science and Open Data.” LIBER Quarterly 25 (4): 153–171.

 Google Scholar

OECD. 2015. “Making Open Science a Reality.” No. 25 OECD Science, Technology and Industry Policy Papers, Paris: OECD Publishing. Accessed January 2022 https://wiki.lib.sun.ac.za/images/0/02/Open-science-oecd.pdf

 Google Scholar

Pujol Priego, L., and J. Wareham. 2019. REANA: Reproducible Research Data Analysis Platform: Open Science Monitor Case Study. European Commission, Directorate-General for Research and Innovation, Publications Office, 2019. http://publications.europa.eu/publication/manifestation_identifier/PUB_KI0219176ENN.

 Google Scholar

Tenopir, C., E. D. Dalton, S. Allard, M. Frame, I. Pjesivac, B. Birch, D. Pollock, K. Dorsett, and P. van den Besselaar. 2015. “Changes in Data Sharing and Data Reuse Practices and Perceptions among Scientists Worldwide.” PLoS ONE 10 (8): e0134826. doi:https://doi.org/10.1371/journal.pone.0134826.

 PubMed Web of Science ®Google Scholar

Borgman, C. L. 2015. Big Data, Little Data, No Data: Scholarship in the Networked World. Cambridge United States: MIT Press.

 Google Scholar

Pasquetto, I. V., B. M. Randles, and C. L. Borgman. 2017. “On the Reuse of Scientific Data.” Data Science Journal 16: 8. doi:https://doi.org/10.5334/dsj-2017-008.

 Google Scholar

Piwowar, H. A., T. J. Vision, and M. C. Whitlock. 2011. “Data Archiving Is a Good Investment.” Nature 473 (7347): 285. doi:https://doi.org/10.1038/473285a.

 PubMed Web of Science ®Google Scholar

Whyte, A., and G. Pryor. 2011. “Open Science in Practice: Researcher Perspectives and Participation.” International Journal of Digital Curation 6 (1): 199–213. doi:https://doi.org/10.2218/ijdc.v6i1.182.

 Google Scholar

Kupferschmidt, K. 2018. “Researcher at the Center of an Epic Fraud Remains an Enigma to Those Who Exposed Him.” Science. Accessed January 2022 https://www.sciencemag.org/news/2018/08/researcher-center-epic-fraud-remains-enigma-those-who-exposed-him.

 Google Scholar

Wicherts, J. M., M. Bakker, D. Molenaar, and R. E. Tractenberg. 2011. “Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results.” PLoS ONE 6 (11): e26828. doi:https://doi.org/10.1371/journal.pone.0026828.

 PubMed Web of Science ®Google Scholar

Harley, D., S. K. Acord, and S. Earl-Novell. 2010. “Peer Review in Academic Promotion and Publishing: Its Meaning, Locus, and Future.” Center for Studies in Higher Education. Accessed January 2022 https://eric.ed.gov/?id=ED512030

 Google Scholar

Howison, J., E. Deelman, M. J. McLennan, R. Ferreira da Silva, and J. D. Herbsleb. 2015. “Understanding the Scientific Software Ecosystem and Its Impact: Current and Future Measures.” Research Evaluation 24 (4): 454–470. doi:https://doi.org/10.1093/reseval/rvv014.

 Web of Science ®Google Scholar

Meijer, I., S. Berghmans, H. Cousijn, C. Tatum, G. Deakin, A. Plume, and A. Rushforth, et al. 2017. Open Data: The Researcher Perspective. Netherlands: University of Leiden.

 Google Scholar

Plantin, J.-C., C. Lagoze, P. N. Edwards, and C. Sandvig. 2018. “Infrastructure Studies Meet Platform Studies in the Age of Google and Facebook.” New Media & Society 20 (1): 293–310. doi:https://doi.org/10.1177/1461444816661553.

 Web of Science ®Google Scholar

Wallis, J. C., E. Rolando, C. L. Borgman, and L. A. Nunes Amaral. 2013. “If We Share Data, Will Anyone Use Them? Data Sharing and Reuse in the Long Tail of Science and Technology.” PLoS ONE 8 (7): e67332. doi:https://doi.org/10.1371/journal.pone.0067332.

 PubMed Web of Science ®Google Scholar

European Commission. 2019. “Cost-Benefit Analysis for FAIR Research Data: Cost of Not Having FAIR Research Data.“ Accessed January 2022. https://op.europa.eu:443/en/publication-detail/-/publication/d375368c-1a0a-11e9-8d04-01aa75ed71a1.

 Google Scholar

Edwards, P. N. 2010. A Vast Machine: Computer Models, Climate Data, and the Politics of Global Warming. Cambridge: MIT Press.

 Google Scholar

Holzner, A., P. Igo-Kemenes, and S. Mele. 2009. “First Results from the PARSE.Insight Project: HEP Survey on Data Preservation, Re-Use and (Open) Access.” ArXiv:0906.0485 [Hep-Ex, Physics: Physics]. Accessed January 2022 http://arxiv.org/abs/0906.0485

 Google Scholar

OpenAire. 2019. “RDM Costs.” OpenAIRE, accessed March 10 2020. https://www.openaire.eu/how-to-comply-to-h2020-mandates-rdm-costs

 Google Scholar

Dphep Study Group. 2009. Data Preservation in High Energy Physics . ArXiv:0912.0255 [Hep-Ex, Physics: Physics]. Accessed January 2022 https://arxiv.org/pdf/0912.0255.pdf

 Google Scholar