ABSTRACT
One of the challenges facing the success of immunization programs is shortages of vaccines at health facilities, which could result from inadequate vaccine stock management. Several approaches have been designed by countries to improve vaccine stock management. This review summarizes currently available information on interventions for vaccine stock management.
We considered both randomized trials and non-randomized studies eligible for inclusion in this review. The following databases were searched: PubMed, Embase, Cochrane Central Register of Controlled Trials, World Health Organization Library Information System, Web of Science, and PDQ-Evidence. We searched the websites of the World Health Organization, Global Alliance for Vaccine and Immunization, PATH’s Vaccine Resources Library, and United Nations Children’s Fund. The reference lists of all the included studies were also searched. Two authors independently screened search outputs, reviewed full texts of potentially eligible articles, evaluated risk of bias, and extracted data; resolving disagreements through consensus.
Four studies met our inclusion criteria (three before–after studies and one randomized trial). Three studies were conducted in low- and middle-income countries while one was conducted in Canada (a high-income country). All the studies had various limitations and were classified as having a high risk of bias. Study findings suggest that the use of digital information systems to improve information and stock visibility, coupled with other interventions (such as training of health-care workers on the use of innovative tools and redesign of the supply chain to tackle certain bottlenecks), has the potential to increase vaccine availability, reduce response times, and improve the quality of vaccine records.
Introduction
Vaccine supply chains play an important role in ensuring functional immunization services and are therefore an important element of ensuring access to primary health-care services.Citation1 The vaccine supply chain is fundamental to a good health system, and it should be designed to ensure consistent and uninterrupted supply of vaccines.Citation2 However, many children, especially in low and middle-income countries (LMICs), do not always get vaccines when they present to health facilities.Citation3,Citation4 Shortages of vaccines at service delivery points have serious consequences for the health-care system and its users. Vaccine shortages undermine the commitment by many countries to the benefits of life saving vaccines and universal access to quality primary health-care services.Citation5
Immunization programs in many countries are currently characterized by the new vaccine introduction and efforts by countries to improve vaccine coverage.Citation6 Whilst the introduction of new vaccines is a progressive step toward achieving the Sustainable Development Goals for health, it creates new challenges. A smooth introduction of new vaccines, which often is an expansion of national schedules with additional antigens, demands upgrading of systems, processes, and infrastructure. However, in many countries, these upgrades are not in place before new vaccines are introduced.
New vaccines place a huge demand on cold chain capacity. For example, in Ethiopia, the introduction of pentavalent, pneumococcal conjugate, and rotavirus vaccines resulted in a fivefold increase in cold chain capacity requirement.Citation7 Some LMICs have reported a 10-fold increase in value of vaccines.Citation5 Many LMICs have vaccine supply chains and management systems that over decades have remained unchanged, with no corresponding increase in capacity and with no upgrade of the technology and information systems.Citation7,Citation8 The interplay of these factors, which includes increased cold chain requirements, the increase in contextual factors such as increase in targeted populations as well as a significant number of outreach (mobile) service points, requires vigilance in management of vaccine stock, to avoid vaccine shortages at service delivery points. Effective management of vaccine supply chains may be better achieved with the support of technology and upgrade of other systems. It is due to these underlying factors that several countries experience vaccine shortages with the introduction of new vaccines.Citation9-Citation11
Vaccine stock-out is said to occur when there is complete absence of a particular vaccine or different vaccines.Citation12 The frequency and duration of stock-outs are indicators commonly used to measure vaccine availability.Citation13 When stock-outs occur, children often miss their vaccinations which in turn reduce immunization coverage.Citation14 In LMICs, vaccine stock-outs are usually caused by poor stock management which includes poor recording of vaccine inventory, poor forecasting of demands, delays, and/or mistakes when placing orders as well as failure to establish actual stock levels before placing orders.Citation14,Citation15
To improve vaccine availability at health facility level, detailed and correct information on stock levels and information on other records are required.Citation14 Innovations in the vaccine supply chain are critically needed to reduce vaccine stock-outs,Citation16 and to this effect, countries have designed approaches to improve vaccine stock management. One of such innovations is the use of digital tools to improve information or visibility of vaccine stock within the supply chain. These devices readily provide information such as stock-on-hand and the quantity received, dispensed, and wasted.Citation17 For example, dash boards and mobile devices are used by LMICs to improve vaccine stock management and also monitor vaccine availability.Citation18-Citation20 Other interventions have been used to improve vaccine stock management include redesigning the supply chain and training of health personnel.Citation21 This scoping review is aimed at identifying and assessing the effectiveness of interventions for improving vaccine stock management in primary health-care facilities. To the best of our knowledge, no review has been conducted to assess the effectiveness of these interventions.
Results
Our literature search yielded 5462 records, out of which 5459 were found through database searching and 3 through other sources. We excluded 5455 clearly irrelevant records and assessed the full texts of the remaining 7 articles for eligibility. Four studies met our inclusion criteria—three before–after studies and one randomized trial.Citation4,Citation22-Citation24 The reasons for excluding the three remaining studiesCitation21,Citation25,Citation26 are provided in . is a flow diagram showing the process of study selection.Citation29
Table 1. List of excluded studies.
One before–after study by Gilbert et al., in 39 facilities in 2 districts of Uttar Pradesh in India assessed the effects of a combination of 3 interventions.Citation4 The first intervention was the use of a standardized data collection process. This data collection process involved the use of papers to record the quantities of vaccines used in every immunization session. This information was then pooled together and captured into a digital software for monitoring of vaccine stock levels in real time using mobile phones, tablets, or computers. For the second intervention, the district-level staff were assigned the responsibility of overseeing vaccine stock management activities across various facilities, to ensure that vaccines do not run out of stock or are not stocked in excess. The third intervention involved the training of the health-care workers at the facilities on the use of the both the standardized manual and digital tools. The interventions improved vaccine availability and reduced replenishment time. The impact was more in low-performing facilities, where vaccine availability increased from 91% to 99%. The average replenishment time reduced from 5 to 2 days.Citation4
The second before–after study that was conducted in Benin and Mozambique by Prosser et al. involved redesigning the supply chain system.Citation22 This redesign process involved the reduction of a four-level supply chain structure to a three-level structure. In a typical four-level system, supply of vaccines begins from the national store (level 1), through the provincial level (level 2) and districts or subdistricts (level 3), to the health facilities (level 4) where they are administered to the target population. In both countries, level 3 was removed and vaccines were delivered directly from level 2 to the health-care facilities (37 in Benin and 111 in Mozambique). Additional transport systems were created, and logisticians were assigned to supervise these activities and ensure that vaccines were successfully delivered to the health facilities. The study observed a reduction in vaccine stock-outs from 79% to less than 1% in Mozambique and an increase in the effective vaccine management score (EVM) from 52% to 65% in Benin. The study conducted in Mozambique measured the effect of the intervention on vaccine stock-outs rates and the cost of implementing the intervention. In addition, the intervention was reported to be 17% more cost effective in Mozambique; but the cost per vaccine dose increased from US$0.009 to US$0.16 in Benin.
The third before–after study by Ramanujapuram and AkkihalCitation23 was conducted in 29 primary health-care centers in Karnataka in India.Citation23 This study assessed the use of “digital bulletin boards”Citation23 by health facility managers who oversee the placing of new orders for vaccines. Vaccine stock levels, derived from real time data generated from primary health-care facilities, were displayed on television screens strategically positioned in managers’ offices or in places that are visible to them, to prompt them into acting. The study reported a steady improvement in vaccine availability up to 99%, though the baseline score was not mentioned. There was also a reduction in replenishment time from 14 to 5 days.Citation23
The fourth study, a randomized trial by Pereira et al.,Citation24 assessed the effect of using barcode scanning technology for vaccine record compared to the manual entry of vaccine records. The barcode method involved scanning the barcode on each vaccine vial such that all the information of the vaccine (such as lot number and expiry date) is captured in an electronic data recording system. The other approach involved manually entering all the vaccine information into a predesigned drop-down menu on the computer system.Citation24 This study measured the completeness and accuracy of data entry. The study observed fewer errors when using barcode scanning, although the exact values were not reported.Citation24
All included studiesCitation4,Citation22-Citation24 were judged to have high-risk of bias since they scored high at least in one of the three domains, that is, completeness of outcome data, similarity of baseline characteristics, and similarity of outcome characteristics. The summary of our risk of bias assessment is shown in .
Table 2. Risk of bias in included randomized trials.
Discussion
The rise in occurrence of stock-outs suggests a need to improve vaccine stock management in countries, especially LMICs.Citation15 We conducted a scoping review to evaluate the effects of interventions for improving vaccine stock management.
Our comprehensive literature search identified a total of 5462 records, out of which 4 were found eligible for inclusion. Three of these studies were before–after studies conducted in LMICs and one was a randomized trial conducted in a high-income country. The first studyCitation4 showed that the use of a combination of interventions, including use standard data collection instruments, digital tools, training of health-care workers, and assigning specific roles to personnel, can increase vaccine availability and reduce replenishment time. The second study showed that redesigning a supply chain from a four-level system to a three-level system can improve vaccine availability and reduce stock-outs.Citation22 The redesigned process reduces the complexity of getting vaccines from the highest level in the supply chain to health facilities. This process resulted in an informed push system, in which vaccines are supplied from the higher levels of the supply chain to the health facilities, which is often informed by the level of consumption in these facilities. The findings from this study are consistent with another study conducted in Senegal, where introducing an informed push system reduced the rate of stock-outs of contraceptives.Citation27
The third studyCitation23 showed that the use of screens to display abnormal stock levels to vaccine managers not only improve vaccine availability but can also reduce replenishment time. The fourth study showed that integrating barcode scanning on vaccine vials as opposed to manual method or drop-down menu to record vaccine inventory could reduce errors that occur when inventory data are captured.
Most of the interventions described were effective in LIMICsCitation4,Citation22,Citation23 implying that they are likely be effective in similar settings, especially hard to reach areas.
Three studiesCitation4,Citation23,Citation24 in this review included a component of digital technology, to improve visibility of vaccine stock levels. Many countries have also adopted the use of digital systems to monitor vaccine levels, making them visible for managers to make informed decisions. Countries such as India, Mozambique, and Nigeria use these systems to improve vaccine availability.Citation18 South Africa, for example, has developed the “Stock Visibility Solution” which is a mobile application designed to enable health facilities to monitor and capture stock levels daily. The information is stored in a cloud-based data management system and can send alerts when the stock is running low.Citation28 The use of digital technologies as interventions for stock management has advantages over manual methods and should be embraced by LMICs as they have the potential to strengthen health systems.Citation30,Citation31 Furthermore, data from manual stock records are not usually efficient to support decision-making, due to poor data quality, wrong estimates of vaccine consumption, and late arrival of data at managerial levels.Citation32 Manual reporting systems are also labor intensive and tend to overburden health-care workers. This in turn affects the motivation of health-care workers who may end up resenting the responsibilities of vaccine stock management because they often also have other duties including clinical duties.Citation9,Citation32 When adequate data are not available, it affects the placing orders and could lead to shortages.Citation31 In addition, findings from this review have underscored the importance of assigning a specific person at health facility level to manage vaccine stock, as highlighted in the studies by Gilbert et al.Citation4 and Prosser et al.Citation22.
Current evidence shows that the use of digital solutions is effective for solving public health problems, but the ones that target the vaccine supply chain are still scarce.Citation32,Citation33 To the best of our knowledge, this study is the first to assess the effects of interventions for vaccine stock management in health facilities where vaccines are administered.
One of the limitations to this study is that most studies included in this review had high risk of bias mostly due to how they were designed and conducted. Also, the related outcomes in all studies were measured differently; there were no consistent methods for measuring outcomes such as stock-out rates and vaccine availability.
Conclusion
Interventions, such as the use of digital information systems (e.g., use of mobile devices, display screens, and barcode scanning of vaccines for recording), training of workers, assigning specific roles to specific personnel, and redesign of the supply chain, have the potential to improve vaccine stock management which eventually can increase vaccine availability. However, well-designed studies are urgently required to increase the certainty of the current evidence base.
Methods
Inclusion criteria
This review was registered in the International Prospective Register of Systematic ReviewsCitation34 and the protocol was published in a peer-reviewed journal.Citation35 The following study designs were eligible for inclusion in this review, randomized trials (with randomization at either individual or cluster levels), controlled before–after studies, interrupted time series studies, and repeated cross-sectional studies with no restrictions on language and publication status. The target population is health-care facilities where vaccines are administered, and health-care workers involved in providing immunization services.
We considered the following interventions in our inclusion criteria:
Interventions directed at providers of immunization services such as education or training, audit and feedback, use of prompts and reminders, and supportive supervision.
Interventions for monitoring vaccine stock level at facilities, e.g., using mobile devices including handheld devices and cellular phones or hotline platforms.
Interventions targeting the health system offering immunization services such as redesigning vaccine supply chain system and integration of interventions with other services.
Other interventions intended to reduce vaccine stock-outs, including multicomponent interventions.
We considered the following as eligible comparisons: standard vaccine stock management practices in the study setting, alternative interventions, and similar interventions implemented with different degrees of intensity. We were cognizant of the fact that a standard vaccine stock management practice in one context may differ in another setting.
Our primary outcomes of interest were as follows:
Vaccine availability: The proportion of vaccination days in which all vaccines in the national schedule were available, and no one eligible for vaccination was turned back because of stock out of vaccines, i.e., did not reach zero-stock or as defined by the authors.
Stock-outs: The percentage of facilities experienced a stock-out of a specific vaccine(s) that a site is expected to provide, at any point, within a defined period.
We also considered the following as our secondary outcomes:
Inventory accuracy rates: The accuracy of data on product stock levels at a facility, that is, a measure of how stock balances recorded on a stock ledger, stock cards, or automated systems are similar to the actual inventory on hand.
Response time: The average time it takes between placing an order from a health facility to a higher facility (e.g., district or provincial store) and when the order is received. Response time is also referred to as order lead time or replenishment time.
EVM score: The EVM assessment is an approach used to assess the performance of the immunization supply chain at different levels from the national to the service delivery levels. The threshold for good performance is a score of 80% World Health Organization (WHO)/United Nations Children’s Fund (UNICEF) 2014.Citation36
Cost of intervention.
Adverse outcomes of the intervention.
Acceptability of the intervention, as defined by the authors of the study.
Search strategy
We developed a comprehensive search strategy for peer-reviewed studies and gray literature. We searched the following databases between June and August 2018 in PubMed and Embase. In addition, we searched the websites of the WHO, Global Alliance for Vaccine and Immunization, PATH’s Vaccine Resources Library, UNICEF, and the International Clinical Trials Registry Platform for trials. We screened the reference lists of all the included studies and related systematic reviews for other potentially eligible primary studies. We also performed a citation search for all studies that have accessed included studies.
Data collection and analysis
Two authors (CJI and AJ) independently screened the titles and abstracts of the retrieved records to identify potentially eligible studies. The full texts of these potentially eligible studies were assessed using the prespecified eligibility criteria. The two authors met to compare lists of included studies and resolved discrepancies by discussion and consensus. A data collection form was designed and used independently by two review authors (CJI and AJ) to extract data from the included studies. Disagreements were resolved through discussion, and a third author (CSW) arbitrated when the two authors failed to reach consensus. The following information was extracted from each included study; study setting (city and country), type of study, study participants, types and description of the intervention, comparator, and study outcomes.
Two authors (CJI and AJ) independently assessed risk of bias in the included studies, using the criteria suggested by the Cochrane Effective Practice and Organization of Care risk of bias tool.Citation37 This tool is based on nine domains, namely, random sequence generation, allocation concealment, similar baseline outcome measurements, similar baseline characteristics, incomplete outcome data, adequate prevention of knowledge of the allocated interventions, protection against contamination, selective reporting, and other potential sources of bias. Of these nine domains, random sequence generation and allocation concealment were not used to assess risk of bias in the non-randomized studies. Each domain was judged as either low-risk, high-risk, or unclear risk. Each non-randomized study was judged to have low-risk of bias if it scored “low-risk” for at least one of the following three domains: completeness of outcome data, similarity of baseline characteristics, and similarity of outcome characteristics. A non-randomized study was considered to have high-risk of bias if it scored “high risk” for at least one of the three domains mentioned above. Differences in judgment were resolved by discussion between the authors and consensus, and there was no need for arbitration by a third author (CSW). Due to heterogeneity in the methods and outcome measures between the included studies, we could not conduct meta-analysis; hence, we synthesized our data narratively.
Authors’ contributions
The study was conceived by CJI and CSW. CJI and AJ were involved in screening of articles for eligibility and data extraction. All authors contributed to the analysis of the data. CJI drafted the review while AJ, CSW, LM, and NN made additions where necessary, reviewed the layout the meaning and interpretation of results. All authors contributed to the final draft and approved the final version of the manuscript.
Disclosure of potential conflicts of interest
No potential conflicts of interest were disclosed.
Acknowledgments
We are grateful to Elizabeth Pienaar for assisting in developing the search strategy.
Additional information
Funding
References
- WHO/UNICEF. Achieving immunization targets with the comprehensive effective vaccine management (EVM) framework [Internet]. WHO/UNICEF Jt. statement2016: 1–5. [accessed 2017 Sep 5]. http://www.who.int/immunization/programmes_systems/supply_chain/evm/en/index1.html
- World health organisation (WHO). Sixty-ninth world health assembly addressing the global shortage of medicines and vaccines [Internet]. 2016 [accessed 2018 Mar 6]. http://apps.who.int/medicinedocs/documents/s22423en/s22423en.pdf
- Lydon P, Raubenheimer T, Arnot-Krüger M, Zaffran M. Outsourcing vaccine logistics to the private sector: the evidence and lessons learned from the Western Cape Province in South-Africa. Vaccine. 2015;33:3429–34. doi:10.1016/j.vaccine.2015.03.042.
- Gilbert SS, Thakare N, Ramanujapuram A, Akkihal A. Assessing stability and performance of a digitally enabled supply chain: retrospective of a pilot in Uttar Pradesh, India. Vaccine. 2017;35:2203–08. doi:10.1016/j.vaccine.2016.11.101.
- Brison M, LeTallec Y. Transforming cold chain performance and management in lower-income countries. Vaccine. 2017;35:2107–09. doi:10.1016/j.vaccine.2016.11.067.
- Guillermet E, Alfa DA, Gbodja R, Jaillard P. Professional changes induced by a redesigned immunization supply chain in the Com? Health Zone, Benin. Vaccine. 2017;35:2189–94. doi:10.1016/j.vaccine.2016.12.074.
- Ashok A, Brison M, LeTallec Y. Improving cold chain systems: challenges and solutions. Vaccine. 2017;35:2217–23. doi:10.1016/j.vaccine.2016.08.045.
- Humphreys G. Vaccination: rattling the supply chain. [Internet]. Bull World Health Organ. 2011; 89:324–25. doi: 10.2471/BLT.11.030511.
- Ngcobo NJ, Kamupira MG. The status of vaccine availability and associated factors in Tshwane government clinics. South African Med J. 2017;107:535. doi:10.7196/SAMJ.2017.v107i6.12149.
- Rao R, Schreiber B, Lee BY. Immunization supply chains: why they matter and how they are changing. Vaccine. 2017;35:2103–04. doi:10.1016/j.vaccine.2017.02.062.
- Lee BY, Assi T-M, Rookkapan K, Wateska AR, Rajgopal J, Sornsrivichai V, Chen S-I, Brown ST, Welling J, Norman BA, et al. Maintaining vaccine delivery following the introduction of the rotavirus and pneumococcal vaccines in Thailand. PLoS One. 2011;6:24673. doi:10.1371/journal.pone.0024673.
- SAGE. Pre-empting and responding to vaccine supply shortages. SAGE April 2016 executive summary [Internet]. 2016 [accessed 2018 Feb 22]. http://www.who.int/immunization/sage/meetings/2016/april/1_Mariat_shortages_SAGE_2016.pdf
- World Health Organization (WHO). Global vaccine action plan. 2015; (Monitoring, Evaluation and accountability Secretariat Annual Report 2014 [Internet]). [accessed 2018 Feb 19]. http://www.who.int/immunization/global_vaccine_action_plan/gvap_secretariat_report_2014.pdf
- Anderson R, Perrier T, Pervaiz F, Sisouveth N, Kumar B, Phongphila S, Rahman A, Dhiman R, Newland S. Supporting immunization programs with improved vaccine cold chain information systems. IEEE Global Humanitarian Technology Conference (GHTC 2014), October 10–13, 215–22. doi:10.1109/GHTC.2014.6970284
- World Health Organization. Assessment report of the global vaccine action plan. Strategic Advisory Group of Experts on Immunization [Internet]. 2017; 1–5. [accessed 2018 Dec 25]. https://www.who.int/immunization/sage/meetings/2017/october/1_GVAP_Assessment_report_web_version.pdf
- World Health Organization/PATH. Collaborating with countries to improve supply chains [Internet]. 2010 [accessed 2018 Mar 13]. https://www.path.org/publications/files/TS_opt_country_collab.pdf
- World Health Organization (WHO). Immunization supply chain and Logistics. A neglected but essential system for national immunization programmes [Internet]. WHO/IVB/14.052014 [accessed 2018 May 14]. https://www.who.int/immunization/documents/WHO_IVB_14.05/en/
- GAVI, WHO, UNICEF, Melina B andGates foundation. Guidance on dashboards for immunization supply chains [Internet]. 2015 [accessed 2017 Aug 25]. https://www.technet-21.org/iscstrengthening/index.php/en/data-for-management-documents-and-downloads/guidance-on-dashboards
- Oliver-Williams C, Brown E, Devereux S, Fairhead C, Holeman I. Using mobile phones to improve vaccination uptake in 21 low-and middle-income countries: systematic review. JMIR Mhealth Uhealth. 2017;5:e148. doi:10.2196/mhealth.7792.
- Africa HS. eHealth strategy South Africa 2012–2016. 2017. (Implementation Review Report).
- Brown ST, Schreiber B, Cakouros BE, Wateska AR, Dicko HM, Connor DL, Jaillard P, Mvundura M, Norman BA, Levin C, et al. The benefits of redesigning Benin’s vaccine supply chain. Vaccine. 2014;32:4097–103. doi:10.1016/j.vaccine.2014.04.090.
- Prosser W, Jaillard P, Assy E, Brown ST, Matsinhe G, Dekoun M, Lee BY. System redesign of the immunization supply chain: experiences from Benin and Mozambique. Vaccine. 2017;35:2162–66. doi:10.1016/j.vaccine.2016.09.073.
- Ramanujapuram A, Akkihal A Improving performance of rural supply chains using mobile phones reducing information asymmetry to improve stock availability in low-resource environments. ACM DEV-5 ’14 Proceedings of the Fifth ACM Symposium on Computing for Development San Jose; 2014 Dec 05–06; California (USA); 11–20.
- Pereira JA, Quach S, Hamid JS, Heidebrecht CL, Quan SD, Nassif J, Diniz AJ, Van Exan R, Malawski J, Gentry A, et al. Exploring the feasibility of integrating barcode scanning technology into vaccine inventory recording in seasonal influenza vaccination clinics. Vaccine. 2012;30:794–802. doi:10.1016/j.vaccine.2011.11.043.
- Shieshia M, Noel M, Andersson S, Felling B, Alva S, Agarwal S, Lefevre A, Misomali A, Chimphanga B, Nsona H, et al. Strengthening community health supply chain performance through an integrated approach: using mHealth technology and multilevel teams in Malawi. J Glob Health. 2014;4:020406. doi:10.7189/jogh.04.020406.
- Kapuria B, Talukdar J, Muthusamy N, Gera R. Designing and implementing an intelligent vaccine logistics management system for India’s Universal Immunisation Programme (UIP) - ‘The eVIN Model. J Pharm Policy Pract. 2014;7:O3. doi:10.1186/2052-3211-7-S1-O3.
- Daff BM, Seck C, Belkhayat H, Sutton P. Informed push distribution of contraceptives in Senegal reduces stockouts and improves quality of family planning services. Glob Heal Sci Pract. 2014;2:245–52. doi:10.9745/GHSP-D-13-00171.
- Médecins Sans Frontières (MSF). The Rural Doctors Association of Southern Africa (RuDASA), the Rural Health Advocacy Project (RHAP), the Treatment Action Campaign (TAC) S and the SAHCS. Stop Stockouts Project (SSP) Stockouts National Survey [Internet]. 2016; http://www.groundup.org.za/media/uploads/documents/StopStockoutsSurvey2016.pdf
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. doi:10.1371/journal.pmed.1000097.
- World Health Organisation. Ensuring contraceptive security through effective supply chains [Internet]. Evid Br. 2017. [accessed 2018 Oct 5]. https://www.popcouncil.org/uploads/pdfs/FP_Evidence_supply_chains_FINAL_07.10.17.pdf.
- Van Velthoven MH, Car J, Zhang Y, Marušic A. New ideas for mHealth data collection implementation in low – and middle – income countries. J Glob Health. 2013;3:10–12. doi:10.7189/jogh.03.020304.
- PATH/World Health Organization. Optimize: immunization systems and technologies for tomorrow. A case for better immunization information systems [Internet]. Evid. Br. Ser. 2010. [accessed 2018 Oct 4]. http://www.who.int/immunization/programmes_systems/supply_chain/optimize/better_immunization_information_systems.pdf.
- Mechael PN. The case for mhealth in developing countries. Mobilizing Mark Spec Ed MIT Innov J GSMA Mob World Congr 2009 Mark Spec Ed MIT Innov J GSMA Mob World Congr. 2009;2009:103–18. Cambridge MIT Press.
- Chinwe J, Anelisa J, Leila Hussein A, Ntombenhle Judith N, Charles SW. A systematic review of interventions for vaccine stock management [Internet]. PROSPERO2018 [accessed 2018 Apr 14]. http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018092215
- Iwu CJ, Jaca A, Abdullahi LH, Ngcobo NJ, Wiysonge CS. Protocol for a systematic review of the effects of interventions for vaccine stock management 11 medical and health sciences 1117 public health and health services. Syst Rev. 2019;8:14.
- WHO/UNICEF. WHO/UNICEF joint statement draft version. effective vaccine management [Internet]. 2014 [accessed 2016 Mar 9]. http://www.who.int/immunization/sage/meetings/2014/april/2_EVM_JS_DraftStatement_5.2.pdf
- Cochrane Effective Practice and Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews. EPOC Resources for review authors. [Internet]. 2017 [accessed 2018 Aug 29]. http://epoc.cochrane.org/resources/epoc-resources-review-authors
Appendix: Search strategy
Database: PubMed
Date searched: 02 August 2018
Database: Embase
Date searched: 18 June 2018