https://orcid.org/0000-0002-1273-4779
ABSTRACT
Systematic reviews have become increasingly important for informing clinical practice and policy; however, little is known about the reporting characteristics and quality of SRs of interventions to improve immunization coverage in different settings. The aim of this study was to assess the reporting quality of systematic reviews of interventions aimed at improving vaccination coverage using the recommended Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guideline.
PubMed and Cochrane Library were searched to identify SRs of interventions to improve immunization coverage, indexed up to May 2016. Two authors independently screened the search output, assessed study eligibility, and extracted data from eligible SRs using a 27-item data collection form derived from PRISMA. Discrepancies in reviews assessments were resolved by discussion and consensus.
A total of 57 reviews were included in this study with a mean percentage of applicable PRISMA items that were met across all studies of 66% (range 19–100%) and median compliance of 70%. 39 out of the 57 reviews were published after the release of the PRISMA statement in 2009. Highest compliance was observed in items related to the “description of rational”, “description of eligibility criteria”, “synthesis of results” and “provision of a general interpretation of the results” (items #3, #6, #14 and #26, respectively). Compliance was poorest in the items “describing summary of evidence” (item 24, 19%), “describing indication of review protocol and registration” (item 5, 26%) and “describing results of risk of bias across studies (item 22, 33%).
The overall reporting quality of systematic reviews of interventions to improve vaccination coverage requires significant improvement. There remains a need for additional research targeted at addressing potential barriers to compliance and strategies to improve compliance with PRISMA guideline.
Introduction
Systematic reviews and meta-analyses have the highest possible level of evidence in medical literature by combining the results of a number of trialsCitation1. Poorly conducted primary trials, even randomized controlled trials (RCTs), could lead to the introduction of bias which may decrease the usefulness of systematic reviews and meta-analyses.Citation2 It is crucial to value findings in systematic reviews and meta-analyses because clinicians use the results directly in clinical practice guidelines and protocols.Citation3 Granting agencies may require a systematic review to ensure there is justification for further research (http://www.cihr-irsc.gc.ca/e/39187.html). As with all research, the value of a systematic review depends on the methodology, outcome, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews differs in quality and appraisal, limiting readers’ ability to assess the strength and weaknesses of those reviews and ultimately reducing its relevance to patient population and clinical relevance.Citation4
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was published in 2009 and consists of a 27-item checklist that covers each section of an article and includes a flow diagram. Its purpose is to help authors improve the quality of reporting in systematic reviews and meta-analyses.Citation5 Similar studies that have used the PRISMA statement in evaluating systematic reviews reporting quality have been reported in different fields, including orthorhino-laryngology,Citation2 orthodontics,Citation6 radiology,Citation7 gastroenterology and hepatology,Citation8 orthopedics,Citation9 plastic surgery, and ophthalmology.Citation10 Irrespective of the field of research, poorly reported systematic reviews can be misleading and potentially dangerous in clinical practice and policy implementation.
Studies in vaccinology may present additional challenges to both researchers and readers because of the need for complexity in study designs. For example, interventions to improve immunization are multifaceted coupled with the dozens of vaccines available for different groups and settings may require alternative designs and methods of analysis. As such, the results and conclusions of such studies may be confusing if they do not provide an appropriate level of explanation. Systematic reviews play a critical role in enabling the accurate appraisal of the literature because they allow large pools of data to be simultaneously interpreted, which can resolve some of the complexity in this field of vaccinology. Adequate reporting of the findings of systematic reviews and meta-analyses using an objective index such as the PRISMA statement is thus essential for clinical practice and patient care.Citation11 Inherently, authors of these studies are required to provide complete, clear and transparent information by using good reporting methods.Citation5,Citation12
To our knowledge, the quality of reporting of systematic reviews and meta-analyses has not been assessed in intervention reviews aimed at improving vaccination coverage. Therefore, our objective was to assess the quality of reporting in systematic reviews of interventions aimed at improving vaccination coverage.
Results
The search yielded a total of 2322 records to be screened and of these, 2247 were considered ineligible and therefore excluded. Subsequently, full texts of the remaining 75 studies were retrieved, 6 of which were protocols, 11 Cochrane reviews, 55 non-Cochrane reviews and 3 overviews; 57 systematic reviews met our predefined inclusion criteria ().
Figure 1. PRISMA selection of eligible studies flow diagram
Among the 57 included reviews, 11 (19.3%) were Cochrane reviews and 46 were non-Cochrane reviews. 18 reviews were published before the release of the PRISMA statement in 2009 while 39 were published post PRISMA release. Among the 57 included reviews, 11 (19%) were updates. The corresponding authors of the systematic reviews were from the following countries: UK (12 reviews, 21%),Citation13–Citation24 Canada (11 reviews, 19%),Citation25–Citation33 Norway (3 reviews, 5%),Citation34–Citation36 Greece (1 review, 2%),Citation37 Australia (2 reviews, 4%),Citation38–Citation40 Switzerland (2 reviews, 4%),Citation41,Citation42 USA (19 reviews, 33%),Citation32,Citation43–Citation60 Thailand (1 review, 2%),Citation61 China (2 reviews, 4%),Citation62,Citation63 Italy (2 reviews, 4%),Citation64,Citation65 India (1 review, 2%)Citation66 and Nigeria (1 review, 2%)Citation67 as shown in . A greater number of systematic reviews had impact factor ≤5 (36, 63.2%) (). Most of the systematic reviews of interventions to improve immunization coverage in this study focused on multifaceted interventions (34, 60%) compared to single recipient oriented (10, 18%) or single provider oriented (12, 21%) intervention.
Table 1. PubMed, and cochrane library search strategies
Table 2. PRISMA score based on the journal type, impact factor, years since publication, country of corresponding author, number of authors, type of immunization intervention
This study examined the quality of reporting of systematic reviews of interventions to improve vaccination coverage was assessed using the PRISMA statement. The mean percentage of applicable PRISMA items that were met across all studies was 66% (range 19–100%) and median compliance of 70%. 39 out of the 57 reviews were published after the release of the PRISMA statement in 2009. Highest compliance (100%) was observed in items related to the “description of rational”, “description of eligibility criteria”, “synthesis of results” and “provision of a general interpretation of the results” (items #3, #6, #14 and #26, respectively). Compliance was poorest in the items “describing summary of evidence” (item 24, 19%), “describing indication of review protocol and registration” (item 5, 26%) and “describing results risk of bias across studies (item 22, 33%) ().
Table 3. Compliance with PRISMA checklist items (adapted from Moher et al.)
In assessing the reporting quality of PRISMA items under the method section, our analysis shows that only three items (item 5, item 15 and item 16) had compliance less than 50% across all studies. These items were “protocol and registration” (15 reviews, 26%), “description of risk of bias across studies” (19 reviews, 33%), and “additional analysis” (20 reviews, 35%). This shows that most systematic reviews of intervention to improve immunization coverage have good reporting quality on methods. While in the results section, applicable PRISMA items that met across all studies with ≤50% compliance were four against three with compliance above 50% with the highest compliance observed in item 17 “describing the study selection criteria”, and in item 20 “describing results of individual studies” (50, 88% each), followed by item 18 “describing the study characteristic” (43, 75%). The summary of evidence item was the poorest PRISMA item that met across all studies with only 11 (19%) studies meeting this criterion. Overall, 18 out of 57 reviews had PRISMA items adherence of 50% and above ().
After selecting the three categories, i.e., Cochrane versus non-Cochrane, high impact versus low impact factor journals and reviews published before and after 2009, PRIMSA reporting quality was appraised. Amongst the Cochrane reviews, 10 reported that a protocol was available, two reported including papers published in English only, while nine considered all languages. In addition, nine made available the search strategy. Eight reviews reported that two authors screened the papers independently while all the 11 reported that data were extracted independently by two reviewers. Moreover, all these reviews assessed risk of bias in their included studies and five stated conducting meta-analysis while eight did subgroup analysis. Eight papers wrote the summary of findings; three wrote limitations in both the study and review level and 10 reported their sources of funding.
We also observed 17 reviews published in high impact factor journals, and these included the 11 Cochrane reviews. Among the six non-Cochrane reviews published in high impact factor journals, two reported using a protocol, one reported including papers published in English only and one provided a search strategy. In three of the non-Cochrane reviews, two authors screened papers and extracted data independently. Three assessed risk of bias in included studies, none conducted meta-analysis, additional analysis, and summary of evidence. Moreover, four wrote about limitations in both study and review level.
Regarding the 18 systematic reviews (which included two Cochrane reviews and two reviews published in high impact journals) published before PRISMA was released, four reported the use of a protocol, seven included papers published in English only and 15 provided their search strategy. Seven reviews reported having two authors to screen and extract data independently. Concerning analysis, 12 and two reported conducting meta-analysis and subgroup analysis, respectively. Only one of the studies included a summary of evidence, 10 mentioned limitations in both the study and review level and 14 reported their funding source.
An assessment of the descriptive characteristics of the search process in the included systematic reviews showed that most of the reviews searched mainly the MEDLINE/PubMed database (52, 91%), followed by EMBASE (34, 60%), Cochrane Library (32, 56%) and CINAHL (30, 53%) databases. It is important to note that in many reviews, authors also reported searching other databases ().
Table 4. Descriptive characteristics of search process in systematic reviews
Discussion
PRISMA statement is a tool to enhance full disclosure of valuable steps in reporting systematic reviews and meta-analyses thereby reducing reporting bias and ultimately permitting more accurate critical appraisal and more balanced and discerning clinical decision-making and policy implementation. The number of systematic reviews and meta-analyses published in the field of vaccines uptake has dramatically increased over the last few years as seen in the number of systematic reviews of interventions included in this study. Even though large data sets could be a positive indication for recognition of increased high-quality articles published in this field, this could also translate to increased room for error. This will, therefore, require that increased importance be placed on accurate and detailed reporting. Clear and transparent information not only allows an adequate interpretation of results but also helps authors to correct errors during earlier stages of the study.Citation5,Citation12
We conducted a study to assess the reporting quality of systematic reviews of interventions to improve vaccination coverage using the 27 items PRISMA checklist. Our review showed that the mean percentage of applicable PRISMA items that were met across all studies was 66% (range 19–100%) and median compliance of 70%. This compliance is relatively low in comparison with most previous studies that investigated the reporting quality of systematic reviews in different specialtiesCitation4,Citation7–Citation9,Citation68 but higher than that observed in the field of plastic surgery, ophthalmology, and nursing.Citation10,Citation69 All the Cochrane reviews met all 27 criteria of the PRISMA statement. Only a few of the non-Cochrane reviews reported working from a protocol provided a search strategy and assessed risk of bias in included studies. The reporting quality of these critical PRISMA checklist items among the non-Cochrane reviews was poor compared to Cochrane reviews thereby requiring improvement in reporting these intervention reviews in vaccination. Without a protocol that is publicly available, it is difficult to judge between proper and improper changes in the reviewCitation1. Among the 18 systematic reviews published before the release of the PRISMA checklist, we observed that only four reported working from a protocol, 15 provided a search strategy and 12 assessed risk of bias. We observed that six items with compliance less than or equal to 35% include items related to “describing the review protocol and registration”, “describing the risk of bias across studies”, “describing the methods of additional analyses”, “describing reporting additional analysis”, “reporting the risk of bias across studies” and “reporting summary of evidence” (items 5, 15, 16, 22, 23, and 24, respectively). Similar poor compliance with these items has also been observed in other previous studies that assessed the reporting quality of systematic reviews in different specialties.Citation2,Citation6–Citation10,Citation69
It was also common to find that, irrespective of their overall compliance with the PRISMA checklist, previous studies in specialties including otorhinolaryngology,Citation2 orthodontics,Citation6 and radiologyCitation7 all achieved compliance lower than 30% in items relating to describing the review protocol and registration and the risk of bias. A generally high median compliance above 80%, with the PRISMA checklist but poor compliance less than 20%, in describing the “protocol and registration” was observed in a study of journals in gastroenterology and hepatology.Citation8 Similar studies that evaluated orthopedic journals,Citation6 compliance of plastic surgery articles to PRISMA statement,Citation10 and a study of nursing journalsCitation69 produced overlapping results that showed there was low compliance regarding items related to the “protocol and registration”, “the description of bias”, and the “description of additional analyses”. The generally low compliance observed in this study is an indication that there is a lack of awareness regarding the importance of review protocols and registration information among authors. The use of protocol and review registration is important because a protocol allows more accurate comparisons to be made between authors and readers and supports better transparency during the review process.Citation5 Surprisingly, most studies generally achieve a low disappointing level of compliance in items related to the assessment of risk of bias. The attempt to eliminate bias is a very fundamental component of randomized controlled trials, as is the assessment of bias in systematic reviews and meta-analysis (https://methods.cochrane.org/bias/assessing-risk-bias-included-studies, assessed 02 October 2018). We suspect that this is caused by a lack of awareness regarding the essential points of research reporting among authors.
Generally, the reviews included in this review had varied literature search sources with only three (5%) reviews that did not report the data sources. PRISMA checklist item 7 states that systematic reviews should “describe all information sources in the search”.Citation5 Systematic reviews require a comprehensive search for published and unpublished studies to minimize bias. Failure to search multiple databases can increase the risk that relevant studies will be missed, which could bias the outcome of the review.Citation70,Citation71 Interestingly, most of the reviews included in this study searched more than one database thus giving more confidence to the readers of reviews of interventions in vaccination.
While improved awareness among stakeholders (i.e., journal reviewers and editors, funders, institutions, and readers) is required, there remains the glaring need for journals to make the PRISMA checklist mandatory for the electronic submission of systematic reviews.Citation10 The enforcement of these guidelines at the time of journal submission might improve the quality of reporting, and as their endorsement has been shown to improve adherence.Citation8,Citation72
PRISMA was designed to cover a wide range of specialties which might not necessarily address specialty specifics. Also, the weighting of items does not take into consideration the weight it gives to the reporting quality of systematic reviews. In this study, we assigned equal weights to all items in the PRISMA checklist and used the same number of denominators in each article. However, this imposed a limitation on our study in that certain items on the checklist may have more impact than others on the reporting quality of systematic reviews. Failure to declare some of the PRISMA items does not necessarily mean they were undertaken but may be due to the publishing style of the journals and some reviews may have lost a mark due to this type of limitation. Another limitation of our study is that there were insufficient numbers of comparable Cochrane versus non-Cochrane reviews, reviews published before versus those published after the PRISMA statement was released and high impact factor versus low impact factor reviews. Therefore, caution should be taken as to the relevance of the quality assessment as accumulation of sufficient comparable numbers of reviews will likely require so much elapse time that may raise debate on the relevance of this assessment.
The overall reporting quality of systematic reviews of interventions to improve vaccination coverage requires significant improvement. The PRISMA tool is a very important tool in assessing the reporting quality of systematic reviews of interventions in vaccination. There remains a need for additional research targeted at addressing potential barriers to compliance and strategies to improve compliance with PRISMA guideline.
Methods
A comprehensive literature search was conducted in 2016 in the Cochrane Database of Systematic Reviews (CDSR), the Database of Abstracts of Reviews of Effects (DARE) and PubMed to identify peer-reviewed systematic reviews. The search strategy was built using a combination of keywords including vaccination, immunization, vaccine, uptake, coverage, and rate ().
Two authors (VNN and AJ) independently screened titles and abstracts of systematic reviews for potential eligibility. Following this, full texts of potentially eligible reviews were retrieved. The authors (VNN and AJ) independently applied eligibility criteria to identify relevant reviews to be included in the study. We excluded reviews in which interventions to improve immunization coverage was not explicitly stated in the title or abstract. Where there were uncertainties, full-text articles were examined for further information. Disagreements regarding review selection were resolved through discussion and consensus, failing which a third author (CSW) arbitrated.
Two authors (VNN and AJ) independently extracted data such as participants, setting, interventions, outcomes, results, publications, and journal of publication. Also, data to assess the reporting quality of the eligible systematic reviews were independently extracted by two authors using the PRISMA checklist form. Discrepancies were discussed and resolved by agreement. Analysis of data was done using Microsoft excel. Continuous variables were presented as mean and range while categorical variables were presented as percentage values. PRISMA items adequately reported for all studies and their percentage compliant with each item of the PRISMA guidelines were analyzed.
Assessment of the reporting quality of the systematic reviews was done independently by two authors (VNN and AJ) for compliance against each item in the 27 items PRISMA checklist. Additionally, compliance was also assessed particularly for the data sources, journal type, impact factor, years since publication, country of corresponding author, number of authors, and type of immunization intervention. To assess the degree of compliance, every item was rated as “yes” for compliance, or ”no” for non-compliance. Differences in opinion were discussed until consensus was reached.
Three categories from the reviews were selected to assess their quality of reporting. These categories included Cochrane versus non-Cochrane reviews, journals of high (impact factor ≥ 5) versus low (impact factor <5) impact factor and reviews published before 2009 (i.e., reviews published before the PRISMA tool was released) versus post-2009 (i.e., reviews published after the PRISMA tool was released).
Disclosure of potential conflicts of interest
No potential conflicts of interest were disclosed.
Authors contributions
VNN and CSW conceived and designed the study, VNN and AJ screened reviews, extracted data, and data analysis. VNN and AJ drafted the manuscript, and all authors contributed in the write-up and final approval of the manuscript for submission.
Acknowledgments
We greatly appreciate the support given to us by Prof Taryn Young (Director of the Centre for Evidence Based Health Care) and the team at the Centre for Evidence Based Health Care, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town South Africa.
Additional information
Funding
Related Research Data
References
- Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma T. Preferred reporting items for systematic reviews and meta-analyses : the PRISMA statement. Int J Surg. 2010;8:336–41.
- Peters JPM, Hooft L, Grolman W, Stegeman I. Reporting quality of systematic reviews and meta-analyses of otorhinolaryngologic articles based on the PRISMA statement. PLoS One. 2015;10(8);1–11.
- Tian J, Zhang J, Ge L, Yang K, Song F. The methodological and reporting quality of systematic reviews from China and the USA are similar. J Clin Epidemiol. 2017;85:50–58. doi:10.1016/j.jclinepi.2016.12.004.
- Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines : review of literature. J Clin Neurosci. 2017;39:45–48. doi:10.1016/j.jocn.2017.02.016.
- Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses : the PRISMA statement. BMJ. 2009;2535(July):1–8.
- Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z. A PRISMA assessment of the reporting quality of systematic reviews in orthodontics. Angle Orthod. 2013;83(1):158–63.
- Tunis AS, Mcinnes MDF, Hanna R, Esmail K. Association of study quality with completeness of reporting : have completeness of reporting and quality of systematic reviews and meta- analyses in major radiology journals changed since publication of the PRISMA statement. Radiology 2013;269(2):413–26.
- Panic N, Leoncini E, De Belvis G, Ricciardi W, Boccia S. Evaluation of the endorsement of the Preferred Reporting Items for Systematic Reviews and Meta- Analysis (PRISMA) statement on the quality of Published Systematic Review and Meta-Analyses. PLoS One. 2013;8(12):e83138. doi: 10.1371/journal.pone.0083138.
- Gagnier JJ, Kellam PJ. Orthopaedic reviews in the orthopaedic literature. J Bone Joint Surg Am. 2013;95(11):e771–7.
- Lee S, Sagoo H, Whitehurst K, Wellstead G, Fowler AJ, Agha RA, et al. Compliance of systematic reviews in plastic surgery with the PRISMA statement. JAMA Facial Plast Surg. 2016;18(2):101–5.
- Hutton B, Salanti G, Caldwell DM, Chaimani A, Jansen JP, Mulrow C. Research and reporting methods the PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions : checklist and explanations. Ann Intern Med. 2015;161(11):777–84.
- Agha RA, Lee S, Jin K, Jeong L, Fowler AJ, Dennis P. Reporting quality of observational studies in plastic surgery needs improvement. Ann Plast Surg. 2016;76(5):585–89.
- Lagard M, Haines A, Palmer N. Conditional cash transfers for improving METHODS. Jama. 2007;298:1900–10.
- Lagarde M, Haines A, Palmer N. The impact of conditional cash transfers on health outcomes and use of health services in low and middle income countries (Review). Cochrane Database Syst Rev. 2009;4:CD008137.
- Williams N, Woodward H, Majeed A, Saxena S. Primary care strategies to improve childhood immunisation uptake in developed countries: systematic review. JRSM Short Rep. 2011;2:1–21.
- Castillo-Riquelme M, Health P, Complete M. The costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle-income countries: systematic review of the grey literature. Bull World Health Organ. 2004;82(9):689–96.
- Adams J, Bateman B, Becker F, Cresswell T, Flynn D, McNaughton R, Oluboyede Y, Roballino S, Ternet L, Sood BG, et al. Effectiveness and acceptability of parental financial incentives and quasi-mandatory schemes for increasing uptake of vaccinations in preschool children: systematic review, qualitative study and discrete choice experiment. Health Technol Assess (Rockv). 2015;119(94):1–176.
- Aigbogun NW, Hawker JI, Stewart A. Interventions to increase influenza vaccination rates in children with high-risk conditions-A systematic review. Vaccine. 2015;33(6):759–70. doi:10.1016/j.vaccine.2014.12.013.
- Elkan R, Kendrick D, Hewitt M, Robinson JJA, Tolley K, Blair M, Dewey M, Williams D, Brummell K. The effectiveness of domiciliary health visiting: A systematic review of international studies and a selective review of the British literature. Health Technol Assess (Rockv). 2000;4(13):i-v+1–330.
- Harvey H, Reissland N, Mason J. Parental reminder, recall and educational interventions to improve early childhood immunisation uptake: A systematic review and meta-analysis. Vaccine. 2015;33(25):2862–80. doi:10.1016/j.vaccine.2015.04.085.
- Kendrick D, Hewitt M, Dewey M, Elkan R, Blair M, Robinson J, Williams D, Brummell K. The effect of home visiting programmes on uptake of childhood immunization: A systematic review and meta-analysis. J Public Health Med. 2000;22(1):90–98.
- Pegurri E, Fox-Rushby JA, Damian W. The effects and costs of expanding the coverage of immunisation services in developing countries: A systematic literature review. Vaccine. 2005;23(13):1624–35. doi:10.1016/j.vaccine.2004.02.029.
- Whittaker K. Lay workers for improving the uptake of childhood immunization. Br J Community Nurs. 2002;7(9):474–79. doi:10.12968/bjcn.2002.7.9.10659.
- Wigham S, Ternent L, Bryant A, Robalino S, Sniehotta FF, Adams J. Parental financial incentives for increasing preschool vaccination uptake: systematic review. Pediatrics. 2014;134(4):e1117–28. doi:10.1542/peds.2013-3604.
- Lau D, Hu J, Majumdar S, Storie DA, Rees SE, Johnson JA. Interventions to improve influenza and pneumococcal vaccination rates among community-dwelling adults: a systematic review and meta-analysis. Ann Fam Med. 2012;10(6):538–47. doi:10.1370/afm.1405.
- Bassani DG, Arora P, Wazny K, Gaffey MF, Lenters L, Bhutta ZA. Financial incentives and coverage of child health interventions: A systematic review and meta-analysis. BMC Public Health. 2013;13(SUPPL.3):S30. doi:10.1186/1471-2458-13-S3-S30.
- Corace KM, Srigley JA, Hargadon DP, Yu D, MacDonald TK, Fabrigar LR, Garber GE. Using behavior change frameworks to improve healthcare worker influenza vaccination rates: A systematic review. Vaccine. 2016;34(28):3235–42. doi:10.1016/j.vaccine.2016.10.045.
- Gyorkos TW, Tannenbaum TN, Abrahamowicz M, Carsley J, Franco ED, Delage G, Miller MA, Lamping DL, Grover SA. Evaluation of the effectiveness of immunization delivery methods. Can J Public Health. 1994;85 Suppl 1:S14–30.
- Johri M, Pérez MC, Arsenault C, Sharma JK, Pai NP, Pahwa S, Sylvestre M-P. NP JMPMACSJP. Strategies to increase the demand for childhood vaccination in low-and middle-income countries: a systematic review and meta-analysis. TT -. Bull World Health Organ. 2015;93(5):339–46. doi:10.2471/BLT.14.146951.
- Lee C, Robinson JL. Systematic review of the effect of immunization mandates on uptake of routine childhood immunizations. J Infect. 2016;72(6):659–66. doi:10.1016/j.jinf.2015.11.003.
- Peacock S, Konrad S, Watson E, Nickel D, Muhajarine N. Effectiveness of home visiting programs on child outcomes: a systematic review. BMC Public Health. 2011;13:1.
- Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Emerg Infect Dis. 1996;3:399–409.
- Thomas Roger EMR, Lorenzetti Diane L. Interventions to increase influenza vaccination rates of those 60 years and older in the community. Cochrane Database Syst Rev. 2010;9:CD005188.
- Saeterdal I, Lewin S, Austvoll-Dahlgren A, Glenton C, Munabi-Babigumira S. Interventions aimed at communities to inform and/or educate about early childhood vaccination (Review). Cochrane Database Syst Rev. 2014;11. doi:10.1002/14651858.CD010232.pub2.
- Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, Van Wyk B, Odgaard-Jensen J, Johansen M, Aja GN, Zwarenstein M, et al. Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev. 2010;3(3):CD004015.
- Glenton C, Scheel IB, Lewin S, Swingler GH. Can lay health workers increase the uptake of childhood immunisation? Systematic review and typology. Trop Med Int Heal. 2011;16(9):1044–53. doi:10.1111/j.1365-3156.2011.02813.x.
- Lytras T, Kopsachilis F, Mouratidou E, Papamichail D, Bonovas S. Interventions to increase seasonal influenza vaccine coverage in healthcare workers: A systematic review and meta-regression analysis. Hum Vaccin Immunother. 2016;12(3):671–81. doi:10.1080/21645515.2016.1208328.
- Kaufman J, Synnot A, Ryan R, Hill S, Horey D, Willis N, Lin V, Robinson P. Face to face interventions for informing or educating parents about early childhood vaccination. Cochrane Database Syst Rev. 2013;5. doi:10.1002/14651858.CD010038.pub2.
- Lassi ZS, Haider BA, Bhutta ZA. Community-based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes. Cochrane Database Syst Rev. 2010;11:CD007754.
- Rashid H, Yin JK, Ward K, King C, Seale H, Booy R. Assessing interventions to improve influenza vaccine uptake among health care workers. Health Aff. 2016;35(2):284–92. doi:10.1377/hlthaff.2015.1087.
- Arditi C, Rège-Walther M, Wyatt J, Durieux P, Burnand B. Computer-generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes (Review). Cochrane Database Syst Rev. 2012;12:CD001175.
- Hollmeyer H, Hayden F, Mounts A, Buchholz U. Review: interventions to increase influenza vaccination among healthcare workers in hospitals. Influenza Other Respi Viruses. 2013;7(4):604–21. doi:10.1111/irv.12002.
- Maglione MA, Stone EG, Shekelle PG. Mass mailings have little effect on utilization of influenza vaccine among Medicare beneficiaries. Am J Prev Med. 2002;23(1):43–46. doi:10.1016/S0749-3797(02)00443-9.
- Voss DS, Wofford LG. Human papillomavirus vaccine uptake in adolescent boys: an evidence review. Worldviews Evidence-Based Nurs. 2014;13(5):390–395.
- Bordley WC, Chelminski A, Margolis PA, Kraus R, Szilagyi PG. The effect of audit and feedback on immunization. Am J Prev Med. 2000;18(4):343–50. doi:10.1016/S0749-3797(00)00126-4.
- Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, Carande-Kulis VG, Yusuf HR, Ndiaye SM, Williams SM. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force community preventive services. Am J Prev Med. 2000;18(1):97–140. doi:10.1016/S0749-3797(99)00118-X.
- Chamla D, Luo C, Adjorlolo-Johnson G, Vandelaer J, Costales MO, Mcclure C. Integration of HIV infant testing into immunization programmes: a systematic review. Paediatr Int Child Health. 2015;35(4):298–304. doi:10.1080/20469047.2015.1109233.
- Groom H, Hopkins DP, Pabst LJ, Murphy Morgan J, Patel M, Calonge N, Coyle R, Dombkowski K, Groom AV, Kurilo MB, et al. Immunization information systems to increase vaccination rates: a community guide systematic review. J Public Health Manag Pract. 2015;21(3):227–48. doi:10.1097/PHH.0000000000000069.
- Ndiaye SM, Hopkins DP, Shefer AM, Hinman AR, Briss PA, Rodewald L, Willis B. Interventions to improve influenza, pneumococcal polysaccharide, and hepatitis B vaccination coverage among high-risk adults: A systematic review. Am J Prev Med. 2005;28(5SUPPL):248–79. doi:10.1016/j.amepre.2005.02.016.
- Niccolai LM, Hansen CE. Practice- and community-based interventions to increase human papillomavirus vaccine coverage. JAMA Pediatr. 2015;169(7):686. doi:10.1001/jamapediatrics.2015.0310.
- Stone EG, Morton SC, Hulscher ME, Maglione MA, Roth EA, Grimshaw JM, Mittman BS, Rubenstein LV, Rubenstein LZ, Shekelle PG. Interventions that increase use of adult immunization and cancer screening services. Ann Intern Med. 2002;136(9):641. doi:10.7326/0003-4819-136-9-200205070-00006.
- Szilagyi PG, Bordley C, Vann JC, Chelminski A, Kraus RM, Margolis PA, Rodewald LE. Effect of patient reminder/recall interventions on immunization rates. JAMA. 2000;284(14):1820. doi:10.1001/jama.284.14.1820.
- Smulian EA, Mitchell KR, Stokley S. Interventions to increase HPV vaccination coverage: A systematic review. Hum Vaccin Immunother. 2016;12(6):1566–88. doi:10.1080/21645515.2016.1208328.
- Walling EB, Benzoni N, Dornfeld J, Bhandari R, Sisk BA, Garbutt J, Colditz G. Interventions to improve HPV vaccine uptake: A systematic review. Pediatrics. 2016;138(1):e20153863–e20153863. doi:10.1542/peds.2015-3863.
- Fu LY, Bonhomme LA, Cooper SC, Joseph JG, Zimet GD. Educational interventions to increase HPV vaccination acceptance: A systematic review. Vaccine. 2014;32(17):1901–20. doi:10.1016/j.vaccine.2014.01.091.
- Giuffrida A, Gosden T, Forland F, Kristiansen I, Sergison M, Leese B, Pedersen L, Sutton M. Target payments in primary care: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2000;3:CD000531.
- Jacob N, Coetzee D. Missed opportunities for immunisation in health facilities in Cape Town, South Africa. South African Med J. 2015 Oct 10;105(11):917. doi:10.7196/SAMJ.2015.v105i11.10194.
- Poorman E, Gazmararian J, Parker RM, Yang B, Elon L. Use of text messaging for maternal and infant health: A systematic review of the literature. Matern Child Health J. 2015;19(5):969–89. doi:10.1007/s10995-014-1595-8.
- Ryman TK, Dietz V, Cairns KL. Too little but not too late: results of a literature review to improve routine immunization programs in developing countries. BMC Health Serv Res. 2008;8:1–11. doi:10.1186/1472-6963-8-134.
- Targonski PV, Poland GA. Review: patient reminder or recall systems improve immunization rates. ACP J Club. 2003;139:18.
- Mureed S, Somronghtong R, Kumar R, Ghaffar A, Chapman RS. Enhanced immunization coverage through interventions for childhood cluster diseases in developing countries. J Ayub Med Coll Abbottabad. 2015;27(1):223–27.
- Wong VWY, Lok KYW, Tarrant M. Interventions to increase the uptake of seasonal influenza vaccination among pregnant women: A systematic review. Vaccine. 2016;34(1):20–32. doi:10.1016/j.vaccine.2016.10.045.
- Wang S, Smith H, Peng Z, Xu B, Wang W. Increasing coverage of hepatitis B vaccination in China: A systematic review of interventions and implementation experiences. Med (United States). 2016;95:1–15.
- Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, Signorelli C. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage: A systematic review. Hum Vaccines Immunother. 2015;11(1):72–82. doi:10.4161/hv.34313.
- Schmidt S, Saulle R, Di Thiene D, Boccia A, La Torre G. Do the quality of the trials and the year of publication affect the efficacy of intervention to improve seasonal influenza vaccination among healthcare workers? Results of a systematic review. Hum Vaccines Immunother. 2013;9(2):349–61. doi:10.4161/hv.22736.
- Gera T, Shah D, Garner P, Richardson M, Hs S. Integrated management of childhood illness (IMCI) strategy for children under five (Review). Cochrane Database Syst Rev. 2016;6:CD010123.
- Oyo-Ita A, Wiysonge C, Oringanje C, Nwachukwu CEC, Oduwole O, Meremikwu MMM. Interventions for improving coverage of childhood immunisation in low- and middle-income countries (Review). Cochrane Database Syst Rev. 2016;(7):CD008145. doi: 10.1002/14651858.CD008145.pub3.
- Pidgeon TE, Wellstead G, SagooH, Jafree DJ. An assessment of the compliance of systematic review articles published in craniofacial surgery with the PRISMA statement guidelines: A systematic review. J Cranio-Maxillo-Facial Surg. 2016;44(10):1522–1530.
- Tam WWS, Lo KKH, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta- analyses published in nursing journals : a cross-sectional study. BMJ Open. 2017;7(2):e013905. doi:10.1136/bmjopen-2016-013905.
- Relevo R, Balshem H. Finding evidence for comparing medical interventions : AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77. doi:10.1016/j.jclinepi.2010.11.022.
- Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.
- Smith TA, Kulatilake P, Brown LJ, Wigley J, Hameed W, Shantikumar S. Do surgery journals insist on reporting by CONSORT and PRISMA? A follow-up survey of ‘ instructions to authors’. Ann Med Surg. 2015;4(1):17–21. doi:10.1016/j.amsu.2014.12.003.