(Un)Available upon Request: Field Experiment on Researchers' Willingness to Share Supplementary Materials

Abstract

This article reports results of a field experiment in which two hundred e-mails were sent to authors of recent articles in economics that had promised to send the interested reader supplementary material, such as alternative econometric specifications, “upon request.” The e-mails were sent either by a researcher affiliated at Columbia University, New York or the University of Warsaw, Poland; furthermore, the authors' position (assistant professor) was specified in half the e-mails only. Overall, 64% of the approached authors responded to our message, of which two thirds (44% of the entire sample) delivered the requested materials. The frequency and speed of responding and delivering were very weakly affected by the position and affiliation of the sender. Gender or affiliation of the author, number of citations or journal impact factor or the type of object in question seemed to make no difference. However, authors of published articles were much more likely to share than authors of working papers.

Keywords:

• data sharing
• field experiment

ACKNOWLEDGMENTS

The authors are grateful for the efforts taken by the subjects in the course of the study. The first author acknowledges financial support from the Marie-Curie European Reintegration Grant 239172.

Notes

1. Note that we did not claim in our e-mails that we were working on a related project and would actually consider, e.g., re-use of the obtained stimuli in a subsequent study, although we acknowledge this was likely to be our subjects' understanding.

2. For the same reason we did not send reminders and we decided to limit ourselves to the sample size of 200, although it would have been easy to expand it if necessary.

3. Partial compliance was rare. Qualitatively, results do not change when these cases are included as positive observations. Occasionally, even more enthusiastic response than full compliance was observed: one author sent the requested material immediately and then, presumably having forgotten about it, he sent it again after three months. Several authors got interested in our projects and asked us to share our findings; one of us was invited to lunch and subsequently to a presentation at the author's university, suggesting that sending random e-mails may be an effective way of networking.

4. To protect identity of the authors and secrecy of correspondence we cannot offer access to all the e-mails we have received. However, we are willing to prepare a version involving no names, places, obvious references to the paper in question or typical figures of speech possibly enabling recognition of personal style. As this requires additional effort which appears useless from the viewpoint of our study itself, it will be, as the reader may guess, performed upon request.

5. However, testing for all JEL codes, for which we have reasonably large number of observations (when verifying equality of compliance rates we split between two categories that we find to be different—published and unpublished papers) we end up with ten different tests, so a conservative Bonferroni correction would question significance of this result.