A randomized trial alerting authors, with or without coauthors or editors, that research they cited in systematic reviews and guidelines has been retracted

ABSTRACT

Retracted clinical trials may be influential in citing systematic reviews and clinical guidelines. We assessed the influence of 27 retracted trials on systematic reviews and clinical guidelines (citing publications), then alerted authors to these retractions. Citing publications were randomized to up to three e-mails to contact author with/without up to two coauthors, with/without the editor. After one year we assessed corrective action. We included 88 citing publications; 51% (45/88) had findings likely to change if retracted trials were removed, 87% (39/45) likely substantially. 51% (44/86) of contacted citing publications replied. Including three authors rather than the contact author alone was more likely to elicit a reply (P = 0.03). Including the editor did not increase replies (P = 0.66). Whether findings were judged likely to change, and size of the likely change, had no effect on response rate or action taken. One year after e-mails were sent only nine publications had published notifications. E-Mail alerts to authors and editors are inadequate to correct the impact of retracted publications in citing systematic reviews and guidelines. Changes to bibliographic and referencing systems, and submission processes are needed. Citing publications with retracted citations should be marked until authors resolve concerns.

KEYWORDS:

• Publication integrity
• retraction
• systematic review
• clinical guideline
• impact

Introduction

Once concerns are raised with journals, delays in the publication of expressions of concern or retraction notices may be lengthy (Grey, Avenell, and Bolland 2021; Loadsman 2019). Even after the retraction of publications, citations usually continue (Candal-Pedreira et al. 2020; Mott, Fairhurst, and Torgerson 2019), although these later citations may constitute a minority of all citations for the retracted work (Bolland, Grey, and Avenell 2022). The continued citation of retracted clinical trials is particularly likely to put patients at risk (Steen 2011, 2012). Previously, Avenell et al. (2019) concluded that the findings of 13/32 (41%) systematic reviews, clinical guidelines and other reviews would likely change if the retracted trials they cited were removed.

Presently, there are no integrated systems to alert researchers that they have cited work with expressions of concern or retractions. The Zotero, EndNote, LibKey and Papers bibliographic systems can automatically alert researchers to entries in Retraction Watch’s database of expressions of concern or retractions (Retraction Watch's website 2022). Initiatives like this should start to reduce the citation of research in academic literature written after work has been retracted, but may not aid correction of literature which has already been published that cites retracted work.

If a researcher finds that their publication cites retracted work, what action should they take? Detailed guidance is lacking. Committee on Publication Ethics (COPE) guidance only states that “Articles that relied on subsequently retracted articles in reaching their own conclusions, such as systematic reviews or meta-analyses, may themselves need to be corrected or retracted” (Committee on Publication Ethics 2019). Of all publications, systematic reviews and clinical guidelines are likely to have the greatest influence on clinical practice and patient care and policy, thus it is particularly important to develop effective systems to alert citing authors to expressions of concern and retractions to ensure their publications are rapidly updated.

Following suspicions aroused during the conduct of a Cochrane review (Avenell et al. 2009), 33 randomized trials by Yoshihiro Sato and Jun Iwamoto were investigated, and findings published (Bolland et al. 2016). Of these publications, twenty-eight were retracted between 2016–2019 in the field of osteoporosis; research particularly relevant for older people with dementia, stroke, and Parkinson’s disease. In this paper, we examined the influence of 27 of the retracted trials on citing systematic reviews and guidelines, where authors appeared unaware that they had cited retracted work. We assessed whether corrective action was needed, and alerted authors to these retractions. We did not know who to alert by e-mail for these systematic reviews or guidelines, or how many reminders might be needed. Therefore, citing publications were randomized to notification of the contact author or the contact author and coauthors, with or without notification of the journal editor. We assessed the responsiveness of recipients, and the incidence and nature of any corrective action undertaken.

Materials and methods

The study protocol is available upon request to the corresponding author. Ethical approval was not required.

Literature searching

By 1 November 2019, 28/33 affected trial reports from the original investigation (Bolland et al. 2016) had been retracted. One affected trial report was retracted on 24 October 2019 and was not included in this study, as the authors would likely have had insufficient time to receive notification before our study commenced. Web of Science (WoS) was searched on 22 November 2019 for publications citing the 27 included trial reports (see list in Appendix 1), and Scopus on 16 January 2020 for any additional citing publications. Citing publications were only included if they were systematic reviews, including meta-analyses, or clinical guidelines. We also hand searched personal files for citing clinical guidelines relating to osteoporosis. Self-citing systematic reviews with Yoshihiro Sato or Jun Iwamoto as first authors were excluded. Systematic reviews or clinical guidelines which only cited retracted trial reports in their introduction, discussion, or as excluded studies (e.g., Cochrane reviews’ category of excluded studies) were not included, thereby limiting citing publications to those whose findings were most likely to be affected.

From July to September 2020, we searched Web of Science, and the websites of journals and guideline organizations, to identify any published evidence (e.g., correction notice, correspondence in the journal) to indicate that authors of citing publications had identified that they had cited one or more of the 27 retracted trials. Citing publications which were unaware of the retractions were included in our trial.

Assessment of impact

For each citing publication, two authors independently categorized the likely impact of the retracted trial reports: whether the findings were likely to change when the retracted trial(s) was removed (Yes/No/Uncertain), the size of any likely change (substantial, moderate, minor), and whether re-analysis of data by the authors was required. Substantial effects were judged if there was a change in the direction of effect, e.g., from negative to neutral, or neutral to positive. Moderate effects were changes within the same effect size. Minor changes were small and did not change the interpretation of the result.

When there was disagreement, a third author made an independent assessment and the majority view was taken. If the citing publication provided a quality assessment rating for the retracted trial report(s), this was also extracted.

Randomization

In a factorial design, with 1:1 randomization, all citing publications were randomized to either an e-mail to the contact author and the journal editor, or only the contact author, using a random number generator (MS Excel 2010). If no editor contact was available, a publisher contact was used, or an organizational contact for guideline organizations. Citing publications with two or more authors were randomized by the same process to an e-mail to the contact author only, or up to 3 authors (out of contact, first, second and final authors). In all cases, coauthors and editors were copied into e-mails to contact authors.

E-mail notifications

Between 15 September 2020 and 14 October 2020, e-mails were sent to contact authors of each citing publication, and depending on randomization, copying in the editor, or other authors. Each e-mail provided the full reference of the citing publication, a list of the retracted trial reports cited, and a pdf of the citing publication highlighting the retracted trial reports. Contacts were advised that a full list of all retractions of publications by the Sato/Iwamoto group was available from the Retraction Watch’s database (http://retractiondatabase.org/RetractionSearch.aspx?). Recipients were asked to confirm receipt of the e-mail and asked whether they believed any action needed to be taken for the citing publication. E-Mails did not contain our assessments of the impact on the citing publication of removing the retracted trial report(s).

If e-mails “bounced,” alternative e-mail addresses for the same authors were sought, e.g., from institutional websites or newer publications. If no e-mail address could be located, another author was sought as a replacement. E-Mails were sent monthly for a maximum of three months. Once any author or editor replied for a citing publication, no further e-mails were sent for that publication. If authors or editors replied asking what they should do, our reply stated that there was no official guidance, but they might consider publishing a notice with the publication.

Assessment of outcomes

We followed-up for outcomes for one year after initial e-mails were sent. The outcomes were: any reply yes/no, time for a reply from first author, time for any reply, action taken (notice published, including date, whether and to what extent the notice provided the impact of the retracted trial(s) and whether re-analysis was undertaken); notice planned, as notified to us; unclear action planned; no action planned. We searched PubMed, Web of Science or Scopus, journal and organizational websites for any notice published relating to the citing publication.

Two authors independently categorized and analyzed themes in the text of replies to our e-mails, which were checked by a third author.

Comparisons between groups or by assessed impact for categorical variables were done using chi-square tests and for continuous variables using the Wilcoxon test for non-parametric data. All analyses were performed using R (R 3.5.1, 2019, R Foundation for Statistical Computing, Vienna, Austria) and P < 0.05 was considered statistically significant.

Results

Results of the search

Figure 1 details the flow chart for the identification of the 88 citing systematic reviews and clinical guidelines included in the trial, published between 2003 and 2020, citing at least one of the 27 retracted trial reports (see Appendix 2), Of these citing publications, 10/88 (11%) cited a trial report that already had been retracted or had an expression of concern prior to submission.

Figure 1. Flow chart for the results of the search.

The 27 retracted trial reports had multiple integrity issues in common, including failure of randomization, improbable recruitment and productivity, remarkably positive outcomes inconsistent with existing literature, inconsistencies between and within trials, failure of ethical oversight and admitted data fabrication (Bolland et al. 2016; Gross 2016).

Description of citing publications

Of the 88 citing publications, 18 (20%) were clinical guidelines or position statements from professional organizations relating to a wide range of medical conditions: the prevention or management of osteoporotic fractures, prevention of falls, preventing or managing stroke and transient ischemic attack, provision of nutrition support, managing limb muscle dysfunction in patients with chronic obstructive pulmonary disease, and preventing vitamin D deficiency. By August 2021, these 88 citing publications had themselves been cited 213 times by clinical guidelines or position statements.

By August 2021, 72 citing publications listed by WoS themselves had 10,493 citations, and 10 unlisted by WoS had 424 citations on Scopus. Of these 82 citing publications, 67 (82%) had at least one citation in 2021. Six citing publications were not listed by WoS or Scopus, four of which were clinical guidelines. Two citing publications, clinical guidelines dating from 2008 and 2014, relating to stroke and transient ischemic attack, had more than 1500 citations each [Kernan et al. 2014; The European Stroke Organisation (ESO) Executive Committee and the ESO Writing Committee 2008], with numerous citations in 2021.

Quality assessment of retracted trial reports by citing publications

42/88 (48%) citing publications with 69 retracted trial report citations performed a quality assessment of those reports. Of these 69, 41 were assessed using qualitative scoring [18 Jadad scale (Jadad et al. 1996); 9 PEDro scale (PEDro scale 2022); 25 by a risk of bias approach including 20 based on Cochrane risk of bias (Higgins et al. 2011)], and three with a descriptive approach. Of the 41 reports with a quality score, seven were given the highest possible score, 28 above the middle score, two below the middle score, and four were not reported. Of the 25 articles with a risk of bias assessment, 14 were scored as low risk across the majority of categories, six as not low risk across the majority of the categories, one a description only, and four were not reported. Thus, the majority of the retracted trials were assessed as low or moderate risk of bias or moderate to high quality. None of the citing publications documented any concerns regarding the publication integrity of the trial reports.

Analysis of impact of retracted trial reports

Table 1 shows that, overall, 45 (51%) of citing publications had findings that we assessed would likely change if the retracted trial reports were removed; in 39/45 (87%) there was likely to be a substantial change, i.e., a change in the direction of that finding; and 20/45 (44%) were assessed as needing reanalysis of data. Agreement between the two main assessors was good: whether findings were likely to change – agreement 85%, kappa 0.75; the size of effect if findings changed – agreement 96%, kappa 0.84; need for reanalysis – agreement 100%. The assessed impacts on citing publications were similar across the randomized groups (Table 1).

Table 1. Assessment of impact on citing publications.

	Authors			Editors
	Contact author n = 40	Up to 3 authors n = 42	Only 1 author* n = 6	Editor contacted n = 44	Editor not contacted n = 44	All n = 88
Findings likely to change n(%)
Likely	24 (60)	19 (45)	2 (33)	24 (55)	21 (48)	45 (51)
Unlikely	11 (28)	16 (38)	4 (67)	16 (36)	15 (34)	31 (35)
Uncertain	5 (12)	7 (17)	0 (0)	4 (9)	8 (18)	12 (14)
Effect size if findings change n(%)
Substantial	22 (92)	16 (84)	1 (50)	21 (88)	18 (86)	39 (87)
Moderate	1 (4)	3 (16)	1 (50)	2 (8)	3 (14)	5 (11)
Minor	1 (4)	0 (0)	0 (0)	1 (4)	0 (0)	1 (2)
If findings change is reanalysis needed n(%)
Yes	11 (46)	8 (42)	1 (50)	11 (46)	9 (43)	20 (44)
Analysis not done	13 (54)	11 (58)	1 (50)	13 (54)	12 (57)	25 (56)

* Six publications had only one author and were not, therefore, eligible to be randomized to copying in other authors.

Outcome assessment

E-Mails regarding two citing publications were not sent because of events between searching and e-mail dispatch. One publication was retracted because meta-analysis findings relied entirely on retracted trial reports (Zhang et al. 2014), and one because the guideline had been updated removing the retracted trial report (Healthcare Improvement Scotland 2015), with guideline staff aware of the retracted trial report through other correspondence with us. Thus, e-mails were sent for 86 publications from a total of 88 randomized.

E-mail replies

We received a reply for around half (44/86, 51%) of citing publications that were emailed (Table 2). No author or editor indicated that a correction notice had been planned before our e-mail arrived. 26/41 (63%) of e-mails to up to three authors produced a reply compared to 18/45 (40%) e-mails to the contact/only author (P = 0.03). Copying the editor had no effect on the proportion of replies received (P = 0.66). Only 9/18 (50%) of guideline authors or organizations provided any e-mail reply. For all replies, the mean time to the first reply was 15 days and the median (range) 2 days (0–63). There was no difference in time to first reply between e-mails to the contact/only author and up to three authors (P = 0.41), or if the journal editor or guideline organization contact was included in the e-mails (P = 0.29). Whether we received any reply, the time for any reply, and the time for a reply by the first author did not differ by year of the citing publication, (<median of 7 years old versus ≥ 7y), whether the publication cited a retracted trial report at the time of submission or not, or whether the findings were judged likely to change by removal of the retracted trial, the size of the likely change or the need for data reanalysis.

Table 2. Outcomes from e-mails.

	Authors			Editors
	Contact author n = 40	Up to 3 authors n = 42	Only 1 author* n = 6	Editor contacted n = 44	Editor not contacted n = 44	All n = 88
Any reply received n(%)
Yes	16 (40)	26 (62)	2 (33)	23 (52)	21 (48)	44 (50)
No	23 (57)	15 (36)	4 (67)	20 (45)	22 (50)	42 (48)
No e-mail sent	1 (2)	1 (2)	0 (0)	1 (2)	1 (2)	2 (2)
Time to first reply received
Mean (SD) days	17 (22)	14 (20)	16 (22)	11 (15)	20 (24)	15 (20)
Median (range) days	5.5 (0–63)	1 (0–64)	16.5 (1–32)	1 (0–39)	7 (0–64)	1.5 (0–64)
Time to reply by first author
Number of replies received	12	17	2	16	15	31
Mean (SD) days	21 (24)	9 (15)	16 (22)	11 (15)	18 (22)	15 (19)
Median (range) days	19 (0–63)	1 (0–39)	16.5 (1–32)	1.5 (0–39)	7 (0–63)	2 (0–63)
Outcomes n(%)
No reply	22 (56)	15 (37)	3 (50)	19 (44)	21 (49)	40 (47)
No action planned	4 (10)	10 (24)	0 (0)	5 (12)	9 (21)	14 (16)
Unclear action planned	7 (18)	13 (32)	1 (17)	11 (26)	10 (23)	21 (24)
Action planned
Notice planned	0 (0)	2 (5)	0 (0)	2 (5)	0 (0)	2 (2)
Notice published	2 (5)	0 (0)	1 (17)	2 (5)	1 (2)	3 (3)
Letter published	3 (8)	1 (2)	1 (17)	3 (7)	2 (5)	5 (6)
Retraction	1 (3)	0 (0)	0 (0)	1 (2)	0 (0)	1 (1)

* Six publications had only one author and were not, therefore, eligible to be randomized to copying in other authors.

Among the 44 citing publications for which we received a reply, 35 (80%) either planned no action or it was unclear what they intended to do (Table 2). Three said that they planned a notice in their initial e-mails, but no notice was published within one year of being contacted.

Published action taken

Nine publications had taken action by one year after our notification (Table 2, seven replied to our e-mails and two did not): five assessed by us as having findings likely to change, with four of these judged as substantial effects. Three of these nine publications published notices (Gillespie et al. 2012; Herrmann et al. 2007; McCarus 2006). Five published letters (Murad et al. 2011, 2012; Peterson 2014; Qaseem et al. 2008, 2017) and one paper was retracted (Zhong and Chen 2009). In five of the nine cases the notice or letter was not linked from the original citing publication (Herrmann et al. 2007; McCarus 2006; Murad et al. 2011, 2012; Qaseem et al. 2008).

In the four cases we judged findings likely to be changed with substantial effects, one paper was retracted (Zhong and Chen 2009), and another author indicated that previous findings were now “called into question” (Peterson 2014). One Cochrane review wrote that the retracted trial report had “minimal impact” (Gillespie et al. 2012), whilst another indicated that the effect was “uncertain” (Qaseem et al. 2008). In the one case where we had considered that reanalysis was required, this was undertaken and new results presented (Murad et al. 2012). The same author re-analyzed data in a further study (Murad et al. 2011).

In summary, one year after our notification e-mails, for 39 of the 44 publications whose findings we considered would change, no action was taken. Four of the remaining 42 publications emailed had also publicly alerted their readers (Herrmann et al. 2007; McCarus 2006; Murad et al. 2012; Qaseem et al. 2017).

We pooled outcomes into two groups (no action planned or unclear if action was planned, and action planned or taken). There were no differences between outcomes whether we contacted up to three authors or only the contact author, whether we contacted the journal or guideline organization or not, or whether the findings were judged likely to change by the removal of the retracted trial, the size of the likely change or the need for data reanalysis.

Two citing publications (Tricco et al. 2017 in JAMA Internal Medicine; Invernizzi et al. 2009 in Parkinsonism and Related Disorders) were published by journals which had themselves retracted one of the source trial reports, in one case after the citing publication came out, and one before. Both citing publications had been cited multiple times in 2021, and neither has been corrected.

Themes identified in e-mail replies

Contact authors from four citing publications asked us for advice as to what they should do. In another four cases the contact author already knew of retractions relating to their publication, but had not provided any update in public.

For 12 citing publications correspondents thought that there was no or minimal impact, so did not plan to alert readers. For three of these 12, we had judged likely substantial impact from removal of the retracted trial reports, but correspondents provided no justification for their decisions.

In three cases, the contact authors said the retractions occurred after their papers were published, and thus they would not take action, or it was unclear what they planned; in one the retraction was three years after the systematic review was published, in two cases some retractions took place 2–5 months before publications were submitted or published online.

In five cases (four of which had publications cited in 2021) authors stated that their publication was too long ago (2003–2014) to warrant reassessment or action, and for the remaining publication it was unclear what was planned.

In three cases, contacts replied that they did not have the resources to undertake a re-analysis.

Two contact authors planned to update their reviews removing retracted trial reports, but it was unclear when this would happen, and both existing reviews continued to be cited in 2021 without correction.

Although we expected coauthors to contact the lead author who had ultimate responsibility for the publication, for seven citing publications coauthors appeared to have made decisions in the absence of the contact author.

In one case, the contact author indicated no intention of taking responsibility for assessing issues, and in another the journal editor did not think he had a responsibility for the citing publication. In three cases, editors appeared to lead discussions on corrective action.

For one clinical guideline, judged to be likely substantially impacted, with over 1500 citations, including 37 in other guidelines, the contact author asked to be removed from the mailing list and gave no other response.

Discussion

Twenty-seven clinical trial reports published 1997–2012 and retracted 2015–2019 have been disseminated in 88 systematic reviews and clinical guidelines. None of the quality assessments published in the citing publications identified any of the retracted trials as concerning. We judged that the findings of 45 (51%) of these citing publications were likely to change after retracted trial reports were removed. For 39/45 (87%) effect sizes were likely to be substantially different. One year after notifying authors and/or journals that their publications had cited retracted trial reports, only 9 had published any kind of public notice, and only four of the 39 publications judged most likely to be substantially affected had alerted readers. Although copying e-mails of notification to more than one author was more likely to elicit a reply, we found no effect of including other authors on addressing concerns in those authors’ systematic reviews or guidelines. Nor did copying in editors or contacts for guideline organizations provoke significantly more responses or alter the frequency of publication of correction notices. Guideline organizations were no more likely to reply or update their readers than other publications.

Whether guidelines’ and systematic reviews’ findings were judged likely to change by removal of the retracted trial, the size of the likely change, or the need for data reanalysis, had no demonstrable effect on outcomes. However, our trial may have been underpowered to detect differences. Our trial data reflect not only systematic reviews and guidelines in osteoporosis care but also the evidence base for older people with dementia, stroke, and Parkinson’s disease. Authors in other fields might respond differently, but we suspect this is unlikely.

In a recent estimate of the epistemic effect of retractions on meta-analyses, Fanelli, Wong, and Moher (2021) examined the potential impact of retracted publications on 50 meta-analyses published between 2013–2016, mostly in clinical journals. The effect size across all meta-analyses without retracted studies was not statistically different from the effect size including those studies. Fanelli, Wong, and Moher (2021) indicated that the impact of retractions was likely to be variable and context specific. To some extent, meta-analyses and systematic reviews by their methodology should be protected from the influence of retraction of a single study, and all but one of the 50 meta-analyses they examined each included only one retracted report. However, in the case of retracted trial reports from Sato and Iwamoto, comparable clinical trials from other authors contributing evidence were absent or few, with small sample sizes. Sato and Iwamoto often published several trial reports of similar interventions in high risk populations (e.g., patients with low vitamin D status, Parkinson’s disease, or stroke), resulting in frequent instances of more than one retracted trial report in citing publications. Thus, the potential for influence is much greater than found by Fanelli, Wong, and Moher (2021).

In a study of 98 retracted randomized controlled trials, Kataoka et al. (2022) found none of the 77 systematic reviews or clinical guidelines including trials retracted beforehand in their evidence synthesis subsequently made corrections. Only 5% (6/130) of evidence syntheses in systematic reviews and 11% (2/18) of clinical guidelines made corrections when trials were retracted after article publication. Three of the publications making corrections were prompted by our trial. Kataoka et al. (2022) demonstrate that failure to correct systematic reviews and clinical guidelines extends beyond the retracted work of Sato and Iwamoto.

There is already a lengthy delay in alerting readers to publications with integrity issues (Grey, Avenell, and Bolland 2021). Inactivity in correcting systematic reviews and clinical guidelines that cite retracted publications exacerbates potential harm to patients, and may mislead clinicians, policy makers and other researchers. While prompter retractions, or at least notifications of concern, would likely reduce misinformation arising from publications with integrity issues, system-wide changes are needed to truly stem that adverse impact.

Our trial suggests that automated e-mails, generated for example from bibliographic citation databases, that notify retractions to either authors of citing publications or publishers (Hamilton 2019; van der Vet and Nijveen 2016), will not elicit prompt corrections or updates to those publications. Although our e-mails arrived unannounced and did not come from these sources, the details they provided were greater than could be provided by automated systems. Our e-mails may have been treated as junk mail, although this seems unlikely, given that they came from an academic e-mail address. That we received no answer or saw no update to citing publications does not mean that publication of notices will not occur in the future, but if a response does not occur within a year, it seems unlikely to expect that one will eventuate.

What needs to change? Firstly, when researchers write papers it is essential that the bibliographic databases and popular search engines (van der Vet and Nijveen 2016) they consult have up-to-date, accessible, information on retractions, expressions of concern and corrections. Whatever the notice, it must be clearly linked to the affected publication, rather than as a disconnected update, as happened with four of the nine notifications from this trial. Consistent terminology to mark affected publications needs to be visibly and rapidly adopted, and used throughout the publishing process (Schneider et al. 2021). Wider use of Crossref’s CrossMark system by publishers would also allow readers to review retractions and corrections when viewing articles (Cosentino and Veríssimo 2016).

Secondly, at manuscript submission authors need to check that they have not cited retracted work, and confirm this in the submission process (Sox and Rennie 2006). The International Committee of Medical Journal Editors has long had guidance for this (Sox and Rennie 2006; International Committee of Medical Journal Editors 2022); journals should require confirmation at the submission stage. Confirmation that retracted papers have been excluded from systematic reviews should be stated in the publication and could be included in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist for reporting of systematic reviews (PRISMA 2020). Authors should also confirm at submission of systematic reviews and guidelines that they will publish a note updating their publication if cited references in their evidence syntheses are retracted.

Thirdly, publishers should improve quality control of their publications by checking submitted and published articles for citations to retracted work, using software packages coupled with Retraction Watch’s database of retractions (Retraction Watch's website 2022), in much the same way they may use software to help detect plagiarism (Cosentino and Veríssimo 2016). Publishers could support the maintenance and oversight of a comprehensive database of retractions.

Fourthly, as we have shown, automated alerts from citation databases or publishers to authors are unlikely to lead to changes alerting readers to retractions, except in a minority of cases, and then not promptly. Although they are not the entire solution, automated alerts should become part of a system-wide approach to mitigate the impact of retractions, because even if only a minority of authors act on these alerts, that will still provide readers with an assurance that the paper remains reliable after the retracted citations are removed.

Finally, and most important, we would argue that readers must be alerted at least to the presence of retracted citations in the evidence syntheses of systematic reviews and clinical guidelines as soon as possible. Until authors publish a note, attesting to the influence or lack of influence of retracted citations, citing publications should be visibly marked on the front page. These “reliability flags” (Fu and Schneider 2020) ought to be the responsibility of publishers, who should alert citing authors to their presence. Such flags would alert readers before authors provide a notice of impact, and in cases where citing authors do not feel it is their responsibility to address these issues. The Cochrane Library has adopted similar processes: flagging retracted studies in Cochrane reviews with an editorial note until the review is updated, whether or not the systematic review’s findings are changed (Cochrane Editorial and Publishing Policy Resource 2022). Given the importance of rapidly updating readers, one Cochrane Group asks authors to turn round revised reviews within 6 weeks (Alfirevic et al. 2021).

In the current analyses, the majority of the retracted trials in the citing publications were assessed as low or moderate risk of bias, or moderate to high quality. In only one instance did an author’s e-mail reply indicate that there had been some concerns about the retracted trial reports at the time of undertaking their systematic review. Existing quality assessment tools are not configured to identify concerns about publication integrity. Those are better identified using specific tools such as the REAPPRAISED checklist. This is best applied during evaluation of the individual trials at the time of their initial reporting or where there are concerns during meta-analysis (Grey et al. 2020).

Many of these issues and solutions to dealing with cited retractions were raised back in 2006 by Sox and Rennie (2006), but remarkably little progress is evident. With the advent of Retraction Watch’s database and better information systems, it must be possible to provide solutions, if all sectors of academic publishing are prepared to work together. Until that time, patients are at risk from the unrecognized and/or uncorrected effects of retracted publications in influential clinical guidance documents. Unless correcting systems are put in place, policymakers who use affected clinical guidelines could unintentionally put patients at risk of harm.

Contributorship

All authors designed the research. GDG performed trial randomization and allocation. AA and MJB collated data. AA, MJB and AG coded data. MJB performed the analyses. AA drafted the paper. All authors critically reviewed and improved it. AA is the guarantor for the article. All authors had access to all data. AA takes responsibility for the integrity of the data and the accuracy of data analysis.

Transparency statement

AA affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained

Acknowledgments

We thank Paul Manson, Information Specialist at the Health Services Research Unit, University of Aberdeen, for assistance with references obtained from Web of Science.

Disclosure statement

No potential conflicts of interest were reported by the author(s).

Data availability statement

Study data are either available in the public domain or can be obtained from the lead author upon reasonable request.

Additional information

Funding

The study received no specific funding. No funder had a role in study design, the collection, analysis, and interpretation of data, the writing of the article, or the decision to submit it for publication. MB is the recipient of an HRC Clinical Practitioner Fellowship. The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. The authors are independent of the HRC.

Appendix

1. Sato Y, Maruoka H, Oizumi K. Amelioration of hemiplegia-associated osteopenia more than 4 years after stroke by 1 alpha-hydroxyvitamin D3 and calcium supplementation. Stroke 1997;28:736–9.

2. Sato Y, Honda Y, Kuno H, Oizumi K. Menatetrenone ameliorates osteopenia in disuse-affected limbs of vitamin D- and K-deficient stroke patients. Bone 1998;23:291–6.

3. Sato Y, Kuno H, Kaji M, Saruwatari N, Oizumi K. Effect of ipriflavone on bone in elderly hemiplegic stroke patients with hypovitaminosis D. Am J Phys Med Rehabil 1999;78:457–63.

4. Sato Y, Manabe S, Kuno H, Oizumi K. Amelioration of osteopenia and hypovitaminosis D by 1alpha- hydroxyvitamin D3 in elderly patients with Parkinson’s disease. J Neurol Neurosurg Psychiatry 1999;66:64–8.

5. Sato Y, Asoh T, Kaji M, Oizumi K. Beneficial effect of intermittent cyclical etidronate therapy in hemiplegic patients following an acute stroke. J Bone Miner Res 2000;15:2487–94.

6. Sato Y, Honda Y, Kaji M, Asoh T, Hosokawa K, Kondo I Amelioration of osteoporosis by menatetrenone in elderly female Parkinson’s disease patients with vitamin D deficiency. Bone 2002;31:114–8.

7. Sato Y, Asoh T, Metoki N, Satoh K. Efficacy of methylprednisolone pulse therapy on neuroleptic malignant syndrome in Parkinson’s disease. J Neurol Neurosurg Psychiatry 2003;74:574–6.

8. Sato Y, Metoki N, Iwamoto J, Satoh K. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in stroke patients. Neurology 2003;61:338–42.

9. Sato Y, Kanoko T, Yasuda H, Satoh K, Iwamoto J. Beneficial effect of etidronate therapy in immobilized hip fracture patients. Am J Phys Med Rehabil 2004;83:298–303.

10. Iwamoto J, Takeda T, Sato Y, Uzawa M. Comparison of effect of treatment with etidronate and alendronate on lumbar bone mineral density in elderly women with osteoporosis. Yonsei Med J 2005;46:750–8.

11. Sato Y, Honda Y, Iwamoto J, Kanoko T, Satoh K. Effect of folate and mecobalamin on hip fractures in patients with stroke: a randomized controlled trial. JAMA 2005;293:1082–8.

12. Sato Y, Iwamoto J, Kanoko T, Satoh K. Low-dose vitamin D prevents muscular atrophy and reduces falls and hip fractures in women after stroke: a randomized controlled trial. Cerebrovasc Dis 2005;20:187–92.

13. Sato Y, Iwamoto J, Kanoko T, Satoh K. Risedronate sodium therapy for prevention of hip fracture in men 65 years or older after stroke. Arch Intern Med 2005;165:1743–8.

14. Sato Y, Iwamoto J, Kanoko T, Satoh K. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in hospitalized, elderly women with Alzheimer’s disease: a randomized controlled trial. J Bone Miner Res 2005;20:1327–33.

15. Sato Y, Iwamoto J, Kanoko T, Satoh K. Risedronate therapy for prevention of hip fracture after stroke in elderly women. Neurology 2005;64:811–6.

16. Sato Y, Kanoko T, Satoh K, Iwamoto J. The prevention of hip fracture with risedronate and ergocalciferol plus calcium supplementation in elderly women with Alzheimer disease: a randomized controlled trial. Arch Intern Med 2005;165:1737–42.

17. Sato Y, Kanoko T, Satoh K, Iwamoto J. Menatetrenone and vitamin D2 with calcium supplements prevent nonvertebral fracture in elderly women with Alzheimer’s disease. Bone 2005;36:61–8.

18. Sato Y, Honda Y, Iwamoto J. Etidronate for fracture prevention in amyotrophic lateral sclerosis: a randomized controlled trial. Bone 2006;39:1080–6.

19. Sato Y, Iwamoto J, Kanoko T, Satoh K. Alendronate and vitamin D2 for prevention of hip fracture in Parkinson’s disease: a randomized controlled trial. Mov Disord 2006;21:924–9.

20. Sato Y, Honda Y, Iwamoto J. Risedronate and ergocalciferol prevent hip fracture in elderly men with Parkinson disease. Neurology 2007;68:911–5.

21. Iwamoto J, Sato Y, Uzawa M, Takeda T, Matsumoto H. Comparison of effects of alendronate and raloxifene on lumbar bone mineral density, bone turnover, and lipid metabolism in elderly women with osteoporosis. Yonsei Med J 2008;49:119–28.

22. Sato Y, Iwamoto J, Honda Y. Beneficial effect of etidronate therapy in chronically hospitalized, disabled patients with stroke. J Stroke Cerebrovasc Dis 2010;19:198–203.

23. Sato Y, Honda Y, Umeno K, Hayashida N, Iwamoto J, Takeda T The prevention of hip fracture with menatetrenone and risedronate plus calcium supplementation in elderly patients with Alzheimer disease: a randomized controlled trial. Kurume Med J 2011;57:117–24.

24. Sato Y, Iwamoto J, Honda Y. Once-weekly risedronate for prevention of hip fracture in women with Parkinson’s disease: a randomized controlled trial. J Neurol Neurosurg Psychiatry 2011;82:1390–3.

25. Sato Y, Iwamoto J, Honda Y. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in Parkinson’s disease. Parkinsonism Relat Disord 2011;17:22–6.

26. Sato Y, Iwamoto J, Honda Y. An open-label trial comparing alendronate and alphacalcidol in reducing falls and hip fractures in disabled stroke patients. J Stroke Cerebrovasc Dis 2011;20:41–6.

27. Iwamoto J, Sato Y, Takeda T, Matsumoto H. Whole body vibration exercise improves body balance and walking velocity in postmenopausal osteoporotic women treated with alendronate: Galileo and Alendronate Intervention Trail (GAIT). J Musculoskelet Neuronal Interact 2012;12:136–43.

Appendix 2

– List of citing publications

Albert SG, Reddy S. Clinical evaluation of cost efficacy of drugs for treatment of osteoporosis: a meta-analysis. Endocrine Practice 2017;23(7):841–56.

Annweiler C, Schott A-M, Berrut G, Fantino B, Beauchet O. Vitamin D-related changes in physical performance: a systematic review. Journal of Nutrition Health & Aging 2009;13(10):893–8.

Annweiler C, Milea D, Whitson HE, Cheng C-Y, Wong T-Y, Ikram MK et al. Vitamin D insufficiency and cognitive impairment in Asians: a multi-ethnic population-based study and meta-analysis. Journal of Internal Medicine 2016;280(3):300–11.

Annweiler C, Llewellyn DJ, Beauchet O. Low serum vitamin D concentrations in Alzheimer’s disease: a systematic review and meta-analysis. Journal of Alzheimer’s Disease 2013;33(3):659–74.

Autier P, Gandini S. Vitamin D supplementation and total mortality – A meta-analysis of randomized controlled trials. Archives of Internal Medicine 2007;167(16):1730–7.

Baladia E, Frutos Perez-Surio A, Martinez-Rodriguez R. Summary of evidence-based nutritional recommendations of the Clinical Practice Guideline for the management of patients with Parkinson’s disease. Nutricion Hospitalaria 2016;33(3):749–60.

Batchelor F, Hill K, Mackintosh S, Said C. What works in falls prevention after stroke? A systematic review and meta-analysis. Stroke 2010;41(8):1715–22.

Beaudart C, Buckinx F, Rabenda V, Gillain S, Cavalier E, Slomian J et al. The effects of vitamin D on skeletal muscle strength, muscle mass, and muscle power: a systematic review and meta-analysis of randomized controlled trials. Journal of Clinical Endocrinology & Metabolism 2014;99(11):4336–45.

Bhogal SK, Teasell RW, Foley NC, Speechley MR. Quality of the stroke rehabilitation research. Topics in Stroke Rehabilitation 2003;10(1):8–28.

Bjelakovic G, Gluud LL, Nikolova D, Whitfield K, Wetterslev J, Simonetti RG et al. Vitamin D supplementation for prevention of mortality in adults. Cochrane Database of Systematic Reviews 2014, Issue 1. Art. No.: CD007470.

Burgos R, Breton I, Cereda E, Desport JC, Dziewas R, Genton L et al. ESPEN guideline clinical nutrition in neurology. Clinical Nutrition 2018;37(1):354–96.

Caffarelli C, Montagnani A, Nuti R, Gonnelli S. Bisphosphonates, atherosclerosis and vascular calcification: update and systematic review of clinical studies. Clinical Interventions in Aging 2017;12:1819–28.

Chowdhury R, Kunutsor S, Vitezova A, Oliver-Williams C, Chowdhury S, Kiefte-de-Jong JC et al. Vitamin D and risk of cause specific death: systematic review and meta-analysis of observational cohort and randomized intervention studies. BMJ 2014;348:g1903.

Crandall CJ, Newberry SJ, Diamant A, Limm YW, Gellad WF, Suttorp MJ et al. Treatment to prevent fractures in men and women with low bone density or osteoporosis: update of a 2007 report. Rockville, MA Agency for Healthcare Research and Quality; 2012.

Crandall M, Duncan T, Mallat A, Greene W, Violano P, Christmas AB et al. Prevention of fall-related injuries in the elderly: An Eastern Association for the Surgery of Trauma practice management guideline. Journal of Trauma and Acute Care Surgery 2016;81(1):196–206.

Da Cunha Sá-Caputo D, Ronikeili-Costa P, Carvalho-Lima RP, Bernardo LC, Bravo-Monteiro MO, Costa R et al. Whole body vibration exercises and the improvement of the flexibility in patient with metabolic syndrome. Rehabilitation Research and Practice 2014; 628,518.

Dennehy C, Tsourounis C. A review of select vitamins and minerals used by postmenopausal women. Maturitas 2010;66(4):370–80.

Dewansingh P, Melse-Boonstra A, Krijnen WP, van der Schans CP, Jager-Wittenaar H, van den Heuvel EGHM. Supplemental protein from dairy products increases body weight and vitamin D improves physical performance in older adults: a systematic review and meta-analysis. Nutrition Research 2018;49:1–22.

Edel Y, Avni T, Shepshelovich D, Reich S, Rozen-Zvi B, Elbaz M et al. The safety of pulse corticosteroid therapy – systematic review and meta-analysis. Seminars in Arthritis and Rheumatism 2020;50(3):534–45.

Fischer M, Vialleron T, Laffaye G, Fourcade P, Hussein T, Cheze L et al. Long-term effects of whole-body vibration on human gait: a systematic review and meta-analysis. Frontiers in Neurology 2019;10:627.

Fujiwara S, Hamaya E, Sato M, Graham-Clarke P, Flynn JA, Burge R. Systematic review of raloxifene in postmenopausal Japanese women with osteoporosis or low bone mass (osteopenia). Clinical Interventions in Aging 2014;9:1879–93.

Giangregorio LM, Papaioannou A, MacIntyre NJ, Ashe MC, Heinonen A, Shipp K et al. Too Fit To Fracture: exercise recommendations for individuals with osteoporosis or osteoporotic vertebral fracture. Osteoporosis International 2014;25(3):821–35.

Gillespie LD, Robertson MC, Gillespie WJ, Sherrington C, Gates S, Clemson L et al. Interventions for preventing falls in older people living in the community. Cochrane Database of Systematic Reviews 2012, Issue 9. Art. No.: CD007146.

Gnädinger M, Mellinghoff HU, Kaelin-Lang A. Parkinson’s disease and the bones. Swiss Medical Weekly 2011;141:w13154.

Gupta A, Thompson PD. The relationship of vitamin D deficiency to statin myopathy. Atherosclerosis 2011;215(1):23–9.

Healthcare Improvement Scotland. SIGN guideline 142: Management of osteoporosis and the prevention of fragility fractures; 2015.

Herrmann M, Schmidt JP, Umanskaya N, Wagner A, Taban-Shomal O, Widmann T et al. The role of hyperhomocysteinemia as well as folate, vitamin B(6) and B(12) deficiencies in osteoporosis a systematic review. Clinical Chemistry and Laboratory Medicine 2007;45(12):1621–32.

Hill KD, Suttanon P, Lin SI, Tsang WWN, Ashari A, Abd Hamid TA et al. What works in falls prevention in Asia: a systematic review and meta-analysis of randomized controlled trials. BMC Geriatrics 2018;18:3.

Inderjeeth CA, Chan K, Kwan K, Lai M. Time to onset of efficacy in fracture reduction with current anti-osteoporosis treatments. Journal of Bone and Mineral Metabolism 2012;30(5):493–503.

Invernizzi M, Carda S, Viscontini GS, Cisari C. Osteoporosis in Parkinson’s disease. Parkinsonism and Related Disorders 2009;15(5):339–46.

Karlsson MK, Karlsson C, Cöster M, Magnusson H, Rosengren BE. Prevention of falls in old people – a review. Reviews in Clinical Gerontology 2013;23(3):206–22.

Kernan WN, Ovbiagele B, Black HR, Bravata DM, Chimowitz MI, Ezekowitz MD et al. Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack. A guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2014;45(7):2160–236.

Latham NK, Anderson CS, Reid IR. Effects of vitamin D supplementation on strength, physical performance, and falls in older persons: A systematic review. Journal of the American Geriatrics Society 2003;51(9):1219–26.

Lee SY, Jung SH, Lee SU, Ha YC, Lim JY. Can bisphosphonates prevent recurrent fragility fractures? A systematic review and meta-analysis of randomized controlled trials. Journal of the American Medical Directors Association 2018;19(5):384–90.

Lerner PP, Sharony L, Miodownik C. Association between mental disorders, cognitive disturbances and vitamin D serum level: current state. Clinical Nutrition ESPEN 2018;23:89–102.

Li Y-T, Cai H-F, Zhang Z-L. Timing of the initiation of bisphosphonates after surgery for fracture healing: a systematic review and meta-analysis of randomized controlled trials. Osteoporosis International 2015;26(2):431–41.

Lin T, Yan SG, Cai XZ, Ying ZM, Yuan FZ, Zuo X. Alendronate versus raloxifene for postmenopausal women: a meta-analysis of seven head-to-head randomized controlled trials. International Journal of Endocrinology 2014:796,510.

Lock CA, Lecouturier J, Mason JM, Dickinson HO. Lifestyle interventions to prevent osteoporotic fractures: a systematic review. Osteoporosis International 2006;17(1):20–8.

Lv Z, Qi H, Wang L, Fan X, Han F, Wang H et al. Vitamin D status and Parkinson’s disease: a systematic review and meta-analysis. Neurological Sciences 2014;35(11):1723–30.

MacLean C, Alexander A, Carter J, Chen S, Desai SB, Grossman J et al. Comparative effectiveness of treatments to prevent fractures in men and women with low bone density or osteoporosis. Comparative effectiveness review number 12. Rockville, MA Agency for Healthcare Research and Quality; 2007.

MacLean C, Newberry S, Maglione M, McMahon M, Ranganath V, Suttorp M et al. Systematic review: comparative effectiveness of treatments to prevent fractures in men and women with low bone density or osteoporosis. Annals of Internal Medicine 2008;148(3):197–213.

Malihi Z, Wu Z, Lawes CMM, Scragg R. Adverse effects and withdrawals in randomized controlled trials of long-term vitamin D-2 or D-3 supplementation: a systematic review and meta-analysis. Nutrition Reviews 2017;75(12):1007–34.

Maltais F, Decramer M, Casaburi R, Barreiro E, Burelle Y, Debigaré R et al. An official American Thoracic Society/European Respiratory Society statement: update on limb muscle dysfunction in chronic obstructive pulmonary disease. American Journal of Respiratory and Critical Care Medicine 2014;189(9):E15-E62.

Mandema JW, Zheng J, Libanati C, Perez Ruixo JJ. Time course of bone mineral density changes with denosumab compared with other drugs in postmenopausal osteoporosis: a dose-response-based meta-analysis. Journal of Clinical Endocrinology & Metabolism 2014;99(10):3746–55.

Marsden J, Gibson LM, Lightbody CE, Sharma AK, Siddiqi M, Watkins C. Can early onset bone loss be effectively managed in post-stroke patients? An integrative review of the evidence. Age & Aging 2008;37(2):142–50.

McCarus DC. Fracture prevention in postmenopausal osteoporosis: a review of treatment options. Obstetrical & Gynecological Survey 2006;61(1):39–50.

Min C, Kang Y, Chunwei L, Rongrong L. Relationship between sarcopenia and the incidence of fall in elderly – a systematic review of advanced nutritional and physical intervention studies. Chinese Journal of Clinical Nutrition 2017; 25(5):278–85.

Muir SW, Montero-Odasso M. Effect of vitamin D supplementation on muscle strength, gait and balance in older adults: a systematic review and meta-analysis. Journal of the American Geriatrics Society 2011;59(12):2291–300.

Murad MH, Drake MT, Mullan RJ, Mauck KF, Stuart LM, Lane MA et al. Comparative effectiveness of drug treatments to prevent fragility fractures: a systematic review and network meta-analysis. Journal of Clinical Endocrinology & Metabolism 2012;97(6):1871–80.

Murad MH, Elamin KB, Abu Elnour NO, Elamin MB, Alkatib AA, Fatourechi MM et al. The effect of vitamin D on falls: a systematic review and meta-analysis. Journal of Clinical Endocrinology & Metabolism 2011;96(10):2997–3006.

Nehra D, Carlson SJ, Fallon EM, Kalish B, Potemkin AK, Gura KM et al. ASPEN clinical guidelines: nutrition support of neonatal patients at risk for metabolic bone disease. Journal of Parenteral and Enteral Nutrition 2013;37(5):570–98.

Nourhashémi F, Olde Rikkert MG, Burns A, Winblad B, Frisoni GB, Fitten J et al. Follow-up for Alzheimer patients: European Alzheimer Disease Consortium position paper. Journal of Nutrition Health & Aging 2010;14(2):121–30.

Orimo H, Nakamura T, Hosoi T, Iki M, Uenishi K, Endo N et al. Japanese 2011 guidelines for prevention and treatment of osteoporosis – executive summary. Archives of Osteoporosis 2012;7: 3–20.

Orr R. The effect of whole body vibration exposure on balance and functional mobility in older adults: A systematic review and meta-analysis. Maturitas 2015;80(4):342–58.

Page A, Etherton-Beer C, Seubert LJ, Clark V, Hill X, King S et al. Medication use to manage comorbidities for people with dementia: a systematic review. Journal of Pharmacy Practice and Research 2018;48(4):356–67.

Pardos-Mainer E, Morales SC, Sagarra L. Effects of vibratory platforms on bone health in postmenopausal women. Revista Cubana de Obstetricia y Ginecología 2019;45(1):118–36.

Peterson AL. A review of vitamin D and Parkinson’s disease. Maturitas 2014;78(1):40–4.

Qaseem A, Snow V, Shekelle P, Hopkins R, Forciea MA, Owens DK et al. Pharmacologic treatment of low bone density or osteoporosis to prevent fractures: a clinical practice guideline from the American College of Physicians. Annals of Internal Medicine 2008;149(6):404–15.

Qaseem A, Forciea MA, McLean RM, Denberg TD, for the Clinical Guidelines Committee of the American College of Physicians. Treatment of low bone density or osteoporosis to prevent fractures in men and women: a clinical practice guideline update from the American College of Physicians. Annals of Internal Medicine 2017;166(11):818–839.

Quinn TJ, Paolucci S, Sunnerhagen KS, Sivenius J, Walker MF, Toni D et al. Evidence-based stroke rehabilitation: an expanded guidance document from the European Stroke Organization (ESO) guidelines for management of ischemic stroke and transient ischemic attack 2008. Journal of Rehabilitation Medicine 2009;41(2):99–111.

Rejnmark L. Effects of vitamin D on muscle function and performance: a review of evidence from randomized controlled trials. Therapeutic Advances in Chronic Disease 2011;2(1):25–37.

Richy F, Ethgen O, Bruyere O, Reginster JY. Efficacy of alphacalcidol and calcitriol in primary and corticosteroid-induced osteoporosis: a meta-analysis of their effects on bone mineral density and fracture rate. Osteoporosis International 2004;15(4):301–10.

Richy F, Schacht E, Bruyere O, Ethgen O, Gourlay M, Reginster JY. Vitamin D analogs versus native vitamin D in preventing bone loss and osteoporosis-related fractures: a comparative meta-analysis. Calcified Tissue International 2005;76(3):176–86.

Ringleb P, Schellinger PD, Hacke W. European Stroke Organization 2008 guidelines for managing acute cerebral infarction or transient ischemic attack – part 1. Nervenarzt 2008;79(8):936–57.

Rogan S, de Bruin ED, Radlinger L, Joehr C, Wyss C, Stuck NJ et al. Effects of whole-body vibration on proxies of muscle strength in old adults: a systematic review and meta-analysis on the role of physical capacity level. European Review of Aging and Physical Activity 2015;12(1):12.

Rogan S, Taeymans J, Radlinger L, Naepflin S, Ruppen S, Bruelhart Y et al. Effects of whole-body vibration on postural control in elderly: an update of a systematic review and meta-analysis. Archives of Gerontology and Geriatrics 2017;73:95–112.

Rolland Y, Souto Barreto P, Van Kan GA, Annweiler C, Beauchet O, Bischoff-Ferrari H et al. Vitamin D supplementation in older adults: searching for specific guidelines in nursing homes. Journal of Nutrition, Health and Aging 2013;17(4):402–12.

Ruan J, Gong X, Kong J, Wang H, Zheng X, Chen T. Effect of B vitamin (folate, B6, and B12) supplementation on osteoporotic fracture and bone turnover markers: a meta-analysis. Medical Science Monitor 2015;21:875–81.

Salari P, Larijani B, Abdollahi M. Association of hyperhomocysteinemia with osteoporosis: a systematic review. Therapy 2008;5(2):215–22.

Scientific Advisory Committee on Nutrition. Vitamin D and health. 2016. Available: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/537616/SACN_Vitamin_D_and_Health_report.pdf (accessed 7 December 2021).

Shinohara Y, Yanagihara T, Abe K, Yoshimine T, Fujinaka T, Chuma T et al. VII. Rehabilitation. Journal of Stroke and Cerebrovascular Diseases 2011;20(4 Suppl):S145-S180.

Stockton KA, Mengersen K, Paratz JD, Kandiah D, Bennell KL. Effect of vitamin D supplementation on muscle strength: a systematic review and meta-analysis. Osteoporosis International 2011;22(3):859–71.

The European Stroke Organisation (ESO) Executive Committee and the ESO Writing Committee. Guidelines for management of ischemic stroke and transient ischemic attack 2008. Cerebrovascular Diseases 2008;25(5):457–507.

The Committee for Developing Guidelines for Prevention and Treatment of Osteoporosis: Japan Osteoporosis Society, Japanese Society for Bone and Mineral Research and Japan Osteoporosis Foundation. Japanese 2011 guideline for prevention and treatment of osteoporosis. http://www.josteo.com/ja/guideline/doc/11_2.pdf (accessed 7 December 2021).

The Committee for Developing Guidelines for Prevention and Treatment of Osteoporosis: Japan Osteoporosis Society, Japanese Society for Bone and Mineral Research and Japan Osteoporosis Foundation. Japanese 2015 guideline for prevention and treatment of osteoporosis. http://www.josteo.com/ja/guideline/doc/15_1.pdf (accessed 7 December 2021).

Tricco AC, Veroniki AA, Hamis JS, Cogo E, Strifler L et al. Comparisons of interventions for preventing falls in older adults. A systematic review and meta-analysis. JAMA Intern Med 2017;318:1687–1699.

van den Bos F, Speelman AD, Samson M, Munneke M, Bloem BR, Verhaar HJ. Parkinson’s disease and osteoporosis. Age & Aging 2013;42(2):156–62.

van Wijngaarden JP, Doets EL, Szczecińska A, Souverein OW, Duffy ME, Dullemeijer C et al. Vitamin B12, folate, homocysteine, and bone health in adults and elderly people: a systematic review with meta-analyses. Journal of Nutrition and Metabolism 2013; 486,186.

Wang G, Sui L, Gai P, Li G, Qi X, Jiang X. The efficacy and safety of vertebral fracture prevention therapies in postmenopausal osteoporosis treatment. Which therapies work best? A network meta-analysis. Bone & Joint Research 2017;6(7):452–63.

Wirth R, Dziewas R, Jäger M, Warnecke T, Smoliner C, Stingel K et al. Guideline of the German society for nutritional medicine (DGEM) in cooperation with the GESKES, the AKE, the DGN and the DGG: clinical nutrition in neurology – part of the ongoing S3-guideline project clinical nutrition. Aktuel Ernahrungsmed 2013;38(4):e49-e89.

Wu H, Pang Q. The effect of vitamin D and calcium supplementation on falls in older adults A systematic review and meta-analysis. Orthopade 2017;46(9):729–36.

Yang J, Hu X, Zhang Q, Cao H, Wang J, Liu B. Homocysteine level and risk of fracture: A meta-analysis and systematic review. Bone 2012;51(3):376–82.

Yang L, Kang N, Yang J-C, Su Q-J, Liu Y-Z, Guan L et al. Drug efficacies on bone mineral density and fracture rate for the treatment of postmenopausal osteoporosis: a network meta-analysis. European Review for Medical and Pharmacological Sciences 2019;23(6):2640–68.

Zhang W, Zhu C, Sun M, Ge Y, Yan G. Efficacy of bisphosphonates against hip fracture in elderly patients with stroke and Parkinson diseases: meta-analysis of randomized controlled trials. Journal of Stroke & Cerebrovascular Diseases 2014;23(10):2714–24.

Zhao Y, Shen L, Ji HF. Alzheimer’s disease and risk of hip fracture: a meta-analysis study. Scientific World Journal 2012:872,173.

Zhao Y, Sun Y, Ji HF, Shen L. Vitamin D levels in Alzheimer’s and Parkinson’s diseases: A meta-analysis. Nutrition 2013;29(6):828–32.

Zhong ZM, Chen JT. Anti-fracture efficacy of risedronic acid in men. A meta-analysis of randomized controlled trials. Clinical Drug Investigation 2009;29(5):349–57. [Retraction in Zhong ZM, Chen JT. Clinical Drug Investigation 2021;41:199.

Zhou J, Wang T, Zhao X, Miller DR, Zhai S. Comparative efficacy of bisphosphonates to prevent fracture in men with osteoporosis: a systematic review with network meta-analyses. Rheumatology and Therapy (2016) 3:117–128