A scoping review of interventions for vaccine stock management in primary health-care facilities

ABSTRACT

One of the challenges facing the success of immunization programs is shortages of vaccines at health facilities, which could result from inadequate vaccine stock management. Several approaches have been designed by countries to improve vaccine stock management. This review summarizes currently available information on interventions for vaccine stock management.

We considered both randomized trials and non-randomized studies eligible for inclusion in this review. The following databases were searched: PubMed, Embase, Cochrane Central Register of Controlled Trials, World Health Organization Library Information System, Web of Science, and PDQ-Evidence. We searched the websites of the World Health Organization, Global Alliance for Vaccine and Immunization, PATH’s Vaccine Resources Library, and United Nations Children’s Fund. The reference lists of all the included studies were also searched. Two authors independently screened search outputs, reviewed full texts of potentially eligible articles, evaluated risk of bias, and extracted data; resolving disagreements through consensus.

Four studies met our inclusion criteria (three before–after studies and one randomized trial). Three studies were conducted in low- and middle-income countries while one was conducted in Canada (a high-income country). All the studies had various limitations and were classified as having a high risk of bias. Study findings suggest that the use of digital information systems to improve information and stock visibility, coupled with other interventions (such as training of health-care workers on the use of innovative tools and redesign of the supply chain to tackle certain bottlenecks), has the potential to increase vaccine availability, reduce response times, and improve the quality of vaccine records.

Keywords:

• Vaccine stock management
• vaccine stock-out
• vaccine availability
• primary health-care facilities
• e-health
• supply chain

Introduction

Vaccine supply chains play an important role in ensuring functional immunization services and are therefore an important element of ensuring access to primary health-care services.¹ The vaccine supply chain is fundamental to a good health system, and it should be designed to ensure consistent and uninterrupted supply of vaccines.² However, many children, especially in low and middle-income countries (LMICs), do not always get vaccines when they present to health facilities.^3,4 Shortages of vaccines at service delivery points have serious consequences for the health-care system and its users. Vaccine shortages undermine the commitment by many countries to the benefits of life saving vaccines and universal access to quality primary health-care services.⁵

Immunization programs in many countries are currently characterized by the new vaccine introduction and efforts by countries to improve vaccine coverage.⁶ Whilst the introduction of new vaccines is a progressive step toward achieving the Sustainable Development Goals for health, it creates new challenges. A smooth introduction of new vaccines, which often is an expansion of national schedules with additional antigens, demands upgrading of systems, processes, and infrastructure. However, in many countries, these upgrades are not in place before new vaccines are introduced.

New vaccines place a huge demand on cold chain capacity. For example, in Ethiopia, the introduction of pentavalent, pneumococcal conjugate, and rotavirus vaccines resulted in a fivefold increase in cold chain capacity requirement.⁷ Some LMICs have reported a 10-fold increase in value of vaccines.⁵ Many LMICs have vaccine supply chains and management systems that over decades have remained unchanged, with no corresponding increase in capacity and with no upgrade of the technology and information systems.^7,8 The interplay of these factors, which includes increased cold chain requirements, the increase in contextual factors such as increase in targeted populations as well as a significant number of outreach (mobile) service points, requires vigilance in management of vaccine stock, to avoid vaccine shortages at service delivery points. Effective management of vaccine supply chains may be better achieved with the support of technology and upgrade of other systems. It is due to these underlying factors that several countries experience vaccine shortages with the introduction of new vaccines.^9-11

Vaccine stock-out is said to occur when there is complete absence of a particular vaccine or different vaccines.¹² The frequency and duration of stock-outs are indicators commonly used to measure vaccine availability.¹³ When stock-outs occur, children often miss their vaccinations which in turn reduce immunization coverage.¹⁴ In LMICs, vaccine stock-outs are usually caused by poor stock management which includes poor recording of vaccine inventory, poor forecasting of demands, delays, and/or mistakes when placing orders as well as failure to establish actual stock levels before placing orders.^14,15

To improve vaccine availability at health facility level, detailed and correct information on stock levels and information on other records are required.¹⁴ Innovations in the vaccine supply chain are critically needed to reduce vaccine stock-outs,¹⁶ and to this effect, countries have designed approaches to improve vaccine stock management. One of such innovations is the use of digital tools to improve information or visibility of vaccine stock within the supply chain. These devices readily provide information such as stock-on-hand and the quantity received, dispensed, and wasted.¹⁷ For example, dash boards and mobile devices are used by LMICs to improve vaccine stock management and also monitor vaccine availability.^18-20 Other interventions have been used to improve vaccine stock management include redesigning the supply chain and training of health personnel.²¹ This scoping review is aimed at identifying and assessing the effectiveness of interventions for improving vaccine stock management in primary health-care facilities. To the best of our knowledge, no review has been conducted to assess the effectiveness of these interventions.

Results

Our literature search yielded 5462 records, out of which 5459 were found through database searching and 3 through other sources. We excluded 5455 clearly irrelevant records and assessed the full texts of the remaining 7 articles for eligibility. Four studies met our inclusion criteria—three before–after studies and one randomized trial.^4,22-24 The reasons for excluding the three remaining studies^21,25,26 are provided in Table 1. Figure 1 is a flow diagram showing the process of study selection.²⁹

Table 1. List of excluded studies.

Study	Study design	Reasons for exclusion
Shieshia^Citation27	Before–after study	The study assessed the effectiveness of an SMS and web-based reporting system used to report stock data using SMS through mobile phones. We excluded this study because its focus was on the health products and did not specifically mention vaccines
Kapuria^Citation28	Before–after study	The study assessed the implementation of vaccine logistic management system that enables the visibility of stock levels in real time coupled with strengthening of human resources capacity. We excluded this study because at the time of assessing the article, results of the study were not out yet
Brown^Citation18	Modeling study	This study assessed the effects of redesigning the vaccine supply chain. However, the authors conducted the study using computational modeling, a basis for exclusion in this review

Figure 1. Process of study selection.

One before–after study by Gilbert et al., in 39 facilities in 2 districts of Uttar Pradesh in India assessed the effects of a combination of 3 interventions.⁴ The first intervention was the use of a standardized data collection process. This data collection process involved the use of papers to record the quantities of vaccines used in every immunization session. This information was then pooled together and captured into a digital software for monitoring of vaccine stock levels in real time using mobile phones, tablets, or computers. For the second intervention, the district-level staff were assigned the responsibility of overseeing vaccine stock management activities across various facilities, to ensure that vaccines do not run out of stock or are not stocked in excess. The third intervention involved the training of the health-care workers at the facilities on the use of the both the standardized manual and digital tools. The interventions improved vaccine availability and reduced replenishment time. The impact was more in low-performing facilities, where vaccine availability increased from 91% to 99%. The average replenishment time reduced from 5 to 2 days.⁴

The second before–after study that was conducted in Benin and Mozambique by Prosser et al. involved redesigning the supply chain system.²² This redesign process involved the reduction of a four-level supply chain structure to a three-level structure. In a typical four-level system, supply of vaccines begins from the national store (level 1), through the provincial level (level 2) and districts or subdistricts (level 3), to the health facilities (level 4) where they are administered to the target population. In both countries, level 3 was removed and vaccines were delivered directly from level 2 to the health-care facilities (37 in Benin and 111 in Mozambique). Additional transport systems were created, and logisticians were assigned to supervise these activities and ensure that vaccines were successfully delivered to the health facilities. The study observed a reduction in vaccine stock-outs from 79% to less than 1% in Mozambique and an increase in the effective vaccine management score (EVM) from 52% to 65% in Benin. The study conducted in Mozambique measured the effect of the intervention on vaccine stock-outs rates and the cost of implementing the intervention. In addition, the intervention was reported to be 17% more cost effective in Mozambique; but the cost per vaccine dose increased from US$0.009 to US$0.16 in Benin.

The third before–after study by Ramanujapuram and Akkihal²³ was conducted in 29 primary health-care centers in Karnataka in India.²³ This study assessed the use of “digital bulletin boards”²³ by health facility managers who oversee the placing of new orders for vaccines. Vaccine stock levels, derived from real time data generated from primary health-care facilities, were displayed on television screens strategically positioned in managers’ offices or in places that are visible to them, to prompt them into acting. The study reported a steady improvement in vaccine availability up to 99%, though the baseline score was not mentioned. There was also a reduction in replenishment time from 14 to 5 days.²³

The fourth study, a randomized trial by Pereira et al.,²⁴ assessed the effect of using barcode scanning technology for vaccine record compared to the manual entry of vaccine records. The barcode method involved scanning the barcode on each vaccine vial such that all the information of the vaccine (such as lot number and expiry date) is captured in an electronic data recording system. The other approach involved manually entering all the vaccine information into a predesigned drop-down menu on the computer system.²⁴ This study measured the completeness and accuracy of data entry. The study observed fewer errors when using barcode scanning, although the exact values were not reported.²⁴

All included studies^4,22-24 were judged to have high-risk of bias since they scored high at least in one of the three domains, that is, completeness of outcome data, similarity of baseline characteristics, and similarity of outcome characteristics. The summary of our risk of bias assessment is shown in Table 2.

Table 2. Risk of bias in included randomized trials.

Discussion

The rise in occurrence of stock-outs suggests a need to improve vaccine stock management in countries, especially LMICs.¹⁵ We conducted a scoping review to evaluate the effects of interventions for improving vaccine stock management.

Our comprehensive literature search identified a total of 5462 records, out of which 4 were found eligible for inclusion. Three of these studies were before–after studies conducted in LMICs and one was a randomized trial conducted in a high-income country. The first study⁴ showed that the use of a combination of interventions, including use standard data collection instruments, digital tools, training of health-care workers, and assigning specific roles to personnel, can increase vaccine availability and reduce replenishment time. The second study showed that redesigning a supply chain from a four-level system to a three-level system can improve vaccine availability and reduce stock-outs.²² The redesigned process reduces the complexity of getting vaccines from the highest level in the supply chain to health facilities. This process resulted in an informed push system, in which vaccines are supplied from the higher levels of the supply chain to the health facilities, which is often informed by the level of consumption in these facilities. The findings from this study are consistent with another study conducted in Senegal, where introducing an informed push system reduced the rate of stock-outs of contraceptives.²⁷

The third study²³ showed that the use of screens to display abnormal stock levels to vaccine managers not only improve vaccine availability but can also reduce replenishment time. The fourth study showed that integrating barcode scanning on vaccine vials as opposed to manual method or drop-down menu to record vaccine inventory could reduce errors that occur when inventory data are captured.

Most of the interventions described were effective in LIMICs^4,22,23 implying that they are likely be effective in similar settings, especially hard to reach areas.

Three studies^4,23,24 in this review included a component of digital technology, to improve visibility of vaccine stock levels. Many countries have also adopted the use of digital systems to monitor vaccine levels, making them visible for managers to make informed decisions. Countries such as India, Mozambique, and Nigeria use these systems to improve vaccine availability.¹⁸ South Africa, for example, has developed the “Stock Visibility Solution” which is a mobile application designed to enable health facilities to monitor and capture stock levels daily. The information is stored in a cloud-based data management system and can send alerts when the stock is running low.²⁸ The use of digital technologies as interventions for stock management has advantages over manual methods and should be embraced by LMICs as they have the potential to strengthen health systems.^30,31 Furthermore, data from manual stock records are not usually efficient to support decision-making, due to poor data quality, wrong estimates of vaccine consumption, and late arrival of data at managerial levels.³² Manual reporting systems are also labor intensive and tend to overburden health-care workers. This in turn affects the motivation of health-care workers who may end up resenting the responsibilities of vaccine stock management because they often also have other duties including clinical duties.^9,32 When adequate data are not available, it affects the placing orders and could lead to shortages.³¹ In addition, findings from this review have underscored the importance of assigning a specific person at health facility level to manage vaccine stock, as highlighted in the studies by Gilbert et al.⁴ and Prosser et al.²².

Current evidence shows that the use of digital solutions is effective for solving public health problems, but the ones that target the vaccine supply chain are still scarce.^32,33 To the best of our knowledge, this study is the first to assess the effects of interventions for vaccine stock management in health facilities where vaccines are administered.

One of the limitations to this study is that most studies included in this review had high risk of bias mostly due to how they were designed and conducted. Also, the related outcomes in all studies were measured differently; there were no consistent methods for measuring outcomes such as stock-out rates and vaccine availability.

Conclusion

Interventions, such as the use of digital information systems (e.g., use of mobile devices, display screens, and barcode scanning of vaccines for recording), training of workers, assigning specific roles to specific personnel, and redesign of the supply chain, have the potential to improve vaccine stock management which eventually can increase vaccine availability. However, well-designed studies are urgently required to increase the certainty of the current evidence base.

Methods

Inclusion criteria

This review was registered in the International Prospective Register of Systematic Reviews³⁴ and the protocol was published in a peer-reviewed journal.³⁵ The following study designs were eligible for inclusion in this review, randomized trials (with randomization at either individual or cluster levels), controlled before–after studies, interrupted time series studies, and repeated cross-sectional studies with no restrictions on language and publication status. The target population is health-care facilities where vaccines are administered, and health-care workers involved in providing immunization services.

We considered the following interventions in our inclusion criteria:

• Interventions directed at providers of immunization services such as education or training, audit and feedback, use of prompts and reminders, and supportive supervision.

• Interventions for monitoring vaccine stock level at facilities, e.g., using mobile devices including handheld devices and cellular phones or hotline platforms.

• Interventions targeting the health system offering immunization services such as redesigning vaccine supply chain system and integration of interventions with other services.

• Other interventions intended to reduce vaccine stock-outs, including multicomponent interventions.

We considered the following as eligible comparisons: standard vaccine stock management practices in the study setting, alternative interventions, and similar interventions implemented with different degrees of intensity. We were cognizant of the fact that a standard vaccine stock management practice in one context may differ in another setting.

Our primary outcomes of interest were as follows:

• Vaccine availability: The proportion of vaccination days in which all vaccines in the national schedule were available, and no one eligible for vaccination was turned back because of stock out of vaccines, i.e., did not reach zero-stock or as defined by the authors.

• Stock-outs: The percentage of facilities experienced a stock-out of a specific vaccine(s) that a site is expected to provide, at any point, within a defined period.

We also considered the following as our secondary outcomes:

• Inventory accuracy rates: The accuracy of data on product stock levels at a facility, that is, a measure of how stock balances recorded on a stock ledger, stock cards, or automated systems are similar to the actual inventory on hand.

• Response time: The average time it takes between placing an order from a health facility to a higher facility (e.g., district or provincial store) and when the order is received. Response time is also referred to as order lead time or replenishment time.

• EVM score: The EVM assessment is an approach used to assess the performance of the immunization supply chain at different levels from the national to the service delivery levels. The threshold for good performance is a score of 80% World Health Organization (WHO)/United Nations Children’s Fund (UNICEF) 2014.³⁶

• Cost of intervention.

• Adverse outcomes of the intervention.

• Acceptability of the intervention, as defined by the authors of the study.

Search strategy

We developed a comprehensive search strategy for peer-reviewed studies and gray literature. We searched the following databases between June and August 2018 in PubMed and Embase. In addition, we searched the websites of the WHO, Global Alliance for Vaccine and Immunization, PATH’s Vaccine Resources Library, UNICEF, and the International Clinical Trials Registry Platform for trials. We screened the reference lists of all the included studies and related systematic reviews for other potentially eligible primary studies. We also performed a citation search for all studies that have accessed included studies.

Data collection and analysis

Two authors (CJI and AJ) independently screened the titles and abstracts of the retrieved records to identify potentially eligible studies. The full texts of these potentially eligible studies were assessed using the prespecified eligibility criteria. The two authors met to compare lists of included studies and resolved discrepancies by discussion and consensus. A data collection form was designed and used independently by two review authors (CJI and AJ) to extract data from the included studies. Disagreements were resolved through discussion, and a third author (CSW) arbitrated when the two authors failed to reach consensus. The following information was extracted from each included study; study setting (city and country), type of study, study participants, types and description of the intervention, comparator, and study outcomes.

Two authors (CJI and AJ) independently assessed risk of bias in the included studies, using the criteria suggested by the Cochrane Effective Practice and Organization of Care risk of bias tool.³⁷ This tool is based on nine domains, namely, random sequence generation, allocation concealment, similar baseline outcome measurements, similar baseline characteristics, incomplete outcome data, adequate prevention of knowledge of the allocated interventions, protection against contamination, selective reporting, and other potential sources of bias. Of these nine domains, random sequence generation and allocation concealment were not used to assess risk of bias in the non-randomized studies. Each domain was judged as either low-risk, high-risk, or unclear risk. Each non-randomized study was judged to have low-risk of bias if it scored “low-risk” for at least one of the following three domains: completeness of outcome data, similarity of baseline characteristics, and similarity of outcome characteristics. A non-randomized study was considered to have high-risk of bias if it scored “high risk” for at least one of the three domains mentioned above. Differences in judgment were resolved by discussion between the authors and consensus, and there was no need for arbitration by a third author (CSW). Due to heterogeneity in the methods and outcome measures between the included studies, we could not conduct meta-analysis; hence, we synthesized our data narratively.

Authors’ contributions

The study was conceived by CJI and CSW. CJI and AJ were involved in screening of articles for eligibility and data extraction. All authors contributed to the analysis of the data. CJI drafted the review while AJ, CSW, LM, and NN made additions where necessary, reviewed the layout the meaning and interpretation of results. All authors contributed to the final draft and approved the final version of the manuscript.

Disclosure of potential conflicts of interest

No potential conflicts of interest were disclosed.

Acknowledgments

We are grateful to Elizabeth Pienaar for assisting in developing the search strategy.

Additional information

Funding

This work is based on research supported by the South African Medical Research Council and the National Research Foundation of South Africa (Grant Number: 106035).

Appendix: Search strategy

Database: PubMed

Date searched: 02 August 2018

Table

No	Query	Results
#1	Search “vaccine management” or “vaccine stock management”	142
#2	Search “drug storage” and (vaccine or vaccines)	648
#3	Search “vaccine stock-out” or “vaccines stock-out”	3
#4	Search (vaccine or vaccines) and “supply chain management”	15
#5	Search (vaccine or vaccines) and “supply and distribution”	2521
#6	Search #1 or #2 or #3 or #4 or #5	3204

Database: Embase

Date searched: 18 June 2018

Table

No	Query	Results
#1	“vaccine management” or “vaccine stock management”	74
#2	(“drug storage”/exp or “drug storage”) and (“vaccine”/exp or vaccine or “vaccines”/exp or vaccines)	1073
#3	“vaccine stock-out” or “vaccines stock-out”	1
#4	(“vaccine”/exp or vaccine or “vaccines”/exp or vaccines) and (“supply chain management”/exp or “supply chain management”)	17
#5	(“vaccine”/exp or vaccine or “vaccines”/exp or vaccines) and (“supply and distribution”/exp or “supply and distribution”)	130
#6	#1 or #2 or #3 or #5	1255