Journal watch

Click to increase image sizeClick to decrease image size

Nancy Milligan

Dianthus Medical Limited

[email protected]

Why won't you give me your data?

In 2005, Dutch researcher Jelte Wicherts and his colleagues contacted the corresponding authors of 141 papers published in four high-ranking psychology journals requesting their datasets to assess the impact of outliers on statistical outcomes.¹ Although all the authors had signed statements confirming that they would share their data with others to allow verification, 73% failed to do so. Why?

To answer this and other questions, Wicherts and his colleagues conducted a new study, recently published in PLoS One,² in which they tested whether there is a link between willingness to share data and the strength and accuracy of statistical results. Does unwillingness to share data stem from fear that reanalysis will expose errors and challenge the authors' conclusions?

Wicherts et al. took a sample of 49 papers from their original study and used the information they contained – reported test statistics (t, F, and χ²), degrees of freedom and P values – to test whether (1) accuracy of statistical reporting and (2) sizes of P values varied according to whether or not the corresponding author had supplied the dataset.

Of the 49 reporting errors they found, a whopping 96% involved reported P values that were smaller than the recalculated ones. A significant majority (73%) occurred in papers whose authors had failed to provide data, while none of the corresponding authors of the seven papers in which supposedly significant P values were in fact found to be non-significant had given Wicherts and his colleagues their data.

In a second recent study,³ Wicherts and his colleague Marjan Bakker analysed 281 psychology papers and found that 15% of them incorrectly assigned statistical significance or non-significance to at least one result.

Further analysis in the PLoS One study² showed that P values were, on average, higher in papers whose data had not been shared. But does authors' fear of their work being undermined, of P values losing their significance explain these findings?

By Wicherts and his colleagues' own admission, this is not the only possible explanation. Could it instead be the case that researchers who analyse their data with more rigour also archive them better and thus have an easier job of retrieving them on request?

Irrespective of what lies behind it, something must be done about the seemingly widespread failure to share data. According to Wicherts, what we need is for journals and other bodies to implement mandatory archiving policies. Making it impossible to publish papers without depositing the data in a web archive would surely alleviate the problem.

Stephen Gilliver

Center for Primary Health Care Research

[email protected]

How short can an abstract be?

Biomedical journals specify word limits for abstracts in the articles they publish. The upper limit is usually in the range of 100–250 words. Sometimes it is difficult to keep within these limits. However, it seems that not all authors have this problem. The abstract below appeared on the physics preprint server arXiv and was sent to Medical Writing by Jim Hartley ([email protected]).

Can apparent superluminal neutrino speeds be explained as a quantum weak measurement?

M V Berry¹, N Brunner¹, S Popescu¹ and P Shukla²

¹H H Wills Physics Laboratory, Tyndall Avenue, Bristol BS8 1TL, UK

²Department of Physics, Indian Institute of Technology, Kharagpur, India

Abstract

Probably not.

Keywords: Quantum measurement, interference, neutrino oscillations

Source: http://arxiv.org/abs/1110.2832

Conflicts of interest: what do peer reviewers think?

Whether or not industry sponsorship causes bias in scientific papers has been much debated. On the other hand, until now, no one has looked at whether conflicts of interest influence how peer reviewers view and review manuscripts.

To explore peer reviewers' feelings about financial conflicts of interest, Suzanne Lippert and her colleagues sent a 29-question web-based survey to 410 active reviewers for Annals of Emergency Medicine, one of the many journals that now require authors to make statements regarding their conflicts of interest.

Most of the 218 reviewers who provided complete responses to the survey felt that authors were influenced by their financial ties to industry.¹ However, this did not clearly translate into changes in the way they evaluate manuscripts.

While a majority of reviewers claimed that they would read more carefully papers whose authors had conflicts of interest, and felt that the credibility of such papers would be reduced, considerably fewer would change their recommendations to the editor.

In their responses to one particular question, three-quarters of reviewers expressed doubt as to whether authors of industry-sponsored articles have full access to data. Meanwhile, a small majority (54%) believed that an honorarium of any size biases an author's judgement, which does not exactly lend support to Lippert et al.'s proposal that authors divulge the sizes of the payments they have received from companies.

Interestingly, a smaller proportion of reviewers who themselves had received such payments considered that they cause bias. Do the experiences of these reviewers not square with the suspicions of those who have never consulted for pharmaceutical companies? Are academics who do not believe that honoraria cause bias more likely to accept them? We can but speculate.

Lippert et al. further suggest that authors confirm that they had full access to the study data, while acknowledging that this is already covered by ICMJE guidelines.² Their third key proposal—that peer reviewers themselves disclose industry payments—is, and has long been, a stipulation of the journal whose reviewers they surveyed.^3,4

In other conflict-of-interest news, David Isaacs, Editor-in-Chief of the Journal of Paediatrics and Child Health, has written an editorial warning of the dangers of financial conflicts of interest and refuting the notion that declaring them does anything to prevent bias.⁵

Stephen Gilliver

Center for Primary Health Care Research

[email protected]