Evaluating the vaccination coverage: validity of household-hold vaccination booklet and caregiver’s recall

ABSTRACT

Background

We compared results from household data sources to medical record sources by using data from a vaccination coverage survey.

Methods

Vaccination coverage (VC) was calculated based on parental recall, household vaccination booklet, and Zhejiang provincial immunization information system (ZJIIS). We evaluated the accuracy of VC based on household sources (vaccination booklet and recall) assuming the medical record was accurate. Concordance, sensitivity, specificity, positive predictive value, and negative predictive value were estimated as well as the Kappa statistic was also used to evaluate the agreement between data sources.

Results

Among the 1,800 children identified in the household survey, all were registered in ZJIIS. VC estimated using the vaccination booklet alone was substantially lower than that based on medical records (net bias 3.4–16.7% in different age groups). VC based on parental recall ranged from 2.5% below (among children aged 1 year) to 16.7% points above (among children aged 6 years) than those based on medical records. Concordance was lowest for card estimates (32.5–45.5%). Sensitivity was <60% for all household sources, except for recall source. Specificity was lowest for recall estimates (14.5–42.6%). Positive predictive value was >75%, while negative predictive value was <50%, for all household sources. Kappa statistics generally indicated poor agreement between household and medical record sources.

Conclusions

Household-retained vaccination booklets and parental recall were insufficient sources for evaluating the VC. Our findings emphasized the importance of taking interventions to make the vaccination booklet more consistent with the records from medical resource.

KEYWORDS:

• Immunization
• vaccination coverage
• survey bias
• validity
• recall

Introduction

Assessing the vaccination coverage is one of the critical aspects of national or local immunization programs for programmatic and policy decision-making, as it is to measure the vaccination service, to enhance the service to underserved groups, and to evaluate the success of immunization in improving the health to population. Additionally, achieving or maintaining a high vaccination coverage (VC) is an essential component of international, national, or local public health goals, such as the United Nation’s Millennium Development Goals,¹ the World Health Organization (WHO) and United Nations Children’s Fund’s Global Immunization Vision and Strategy,² and the Outline of Healthy China 2030 Plan.³ Ensuring accuracy and completeness of the VC information sources is critical to reduce the potential bias and thus provide a complete and accurate parameter of the current situation.

Calculating the VC by using administrative data (by dividing the number of doses administered by the estimated target population) is common in most countries,^4–6 but the results may be unreliable when the target population size is incorrect or when the reporting is incomplete. Proportion probability survey is considered independent of these concerns and is generally considered as the “gold standard” for evaluating the VC.⁷ In these surveys, vaccination status is typically categorized on the basis of the documentation of vaccination dates on household vaccination booklet, sometimes supplemented through the caregiver’s recall.^8–10 However, the validity of estimates derived from these household data sources is not thoroughly evaluated.

Chinese Expanded Program on Immunization (EPI) was launched in 1978 with 4 vaccines, and today it continues to ten vaccines (22 doses).¹¹ The latest vaccination schedule includes one dose of Bacillus Calmette–Guerin vaccine (BCG) scheduled at birth; three doses of hepatitis B vaccine (HBV) scheduled at birth, 1 month, and 6 months of age; four doses of diphtheria-tetanus-pertussis combined vaccine (DTP) scheduled at 3, 4, 5 and 18 months of age; four doses of polio vaccine (PV) scheduled at 2, 3, and 4 months and 4 years of age; two doses of measles-containing vaccine (MCV) scheduled at 8 and 18 months of age; two doses of Japanese encephalitis vaccine (JEV) scheduled at 8 months and 2 years of age; two doses of meningococcal polysaccharide vaccine-type a (MenV-a) scheduled at 6 and 9 months of age; two doses of meningococcal polysaccharide vaccine-type a and c (MenV-ac) scheduled at 3 and 6 years of age; one dose of diphtheria-tetanus combined vaccine (DT) scheduled at 6 years of age; one dose of hepatitis A vaccine (Hep A) scheduled at 18 months of age.¹² All vaccinations are administered in vaccination clinic set in public community health service center or some private hospitals. Child-level vaccination records are maintained in an electronic registry referred as Zhejiang provincial immunization information system (ZJIIS) in which all vaccination clinics are included.¹³ ZJIIS is the only official immunization information system in Zhejiang province, which is a part of the provincial health record system and its function can be found elsewhere.¹³ The caregivers are also given a vaccination booklet that records the vaccination information and are encouraged to well maintain this booklet at home.

In this study, the immunization status of children aged 1, 2 and 6 years in six counties in Zhejiang province was evaluated, based on the household vaccination booklet and caregiver’s recall as well as the vaccination records of the same children retained by the vaccination. The age-specific VC derived from the different data sources was compared. The accuracy of VC from household data was evaluated assuming the vaccination records retained by the vaccination clinic were accurate.

Methods

Study areas

This study was conducted in 2019 in 6 out of the 90 counties in Zhejiang province, including Yinzhou, Yuhang, Jiashan, Changxing, Jingning, and Suichang. Zhejiang province is located at the east coastline of China, with a population of 65 million in 2018 and an area of 1055 km². The average annual birth rate was around 1.1% in the last 5 years, with an estimated 639,652 births in 2018, respectively. The six counties were randomly selected according to the socioeconomic development level in 2018, which was categorized by the index of gross domestic product (GDP) per capita. Yinzhou and Yuhang belonged to the high GDP per capita stratum (≥15,000 USD). Jiashan and Changxing were of middle stratum for GDP per capita, at between 10,000 and 15,000 USD. Jingning and Kecheng were of the low GDP per capita stratum under 10,000 USD. The population of the selected counties were obtained from the 2018 census of Zhejiang province. The total population size of Yinzhou, Yuhang, Jiashan, Changxing, Jingning and Suichang of 2018 was 1891560, 1429257, 592395, 675261, 112523, and 192377, respectively.

Outcome

The main outcome of this study was designed to estimate the percentages of children aged 1, 2, and 6 years who had been administered the full immunization series recommended at the relevant age. Specifically, children aged 1 year (12–23 months) were considered fully immunized if they had received all vaccine doses recommended by age 12 months; children aged 2 years (24–35 months) were considered fully immunized if they had received all vaccine doses recommended by age 24 months; children aged 6 years (72–83 months of age) were considered fully immunized if they had received all vaccine doses required according to the vaccination schedule. In this study, VC was defined as the percentage of children fully immunized for each age group.

Sample size and survey procedures

The survey applied the household-based cluster survey method recommended by the WHO.¹⁴ The sample size of the two surveys was calculated based on the formula as follows:

$\mathit{N}=\mathit{deff}\times \frac{z_{\left(1-\frac{\alpha }{2}\right)}^2\times \mathit{p}\times \left(1-\mathit{p}\right)}{d^2}$

The parameters were set as a two-tailed α error of 5%, a permissible error (d) of 0.1, a design effect (deff) of 2, and expected coverage of VC at 0.85 As such, the final sample size required for each county and each age group was 98, and we finally determined the sample size as 100 children per each county and each age group (300 children per each county) for the operational convenience.

The field survey included three steps: first, ten clusters (villages or communities) were selected by systematic random sampling with probability of selection proportional to estimated size of populations in each county and ten eligible children for each age group needed to be surveyed in each cluster. Second, the household, which was defined as a group of persons who live and eat together and was considered eligible if there was at least one child aged 1, 2, or 6 years currently residing there, was chosen by random sampling in each selected cluster. Third, the adjacent household right to the previous household was selected till all ten children per each age group were surveyed. There were three other criterions to follow: (1) all eligible children in selected households were included; (2) households should be arranged another visit if there was somebody living but without any response; (3) children who usually resided in one household but were temporarily staying in another were included in the household where they usually resided; (4) the closest community or village was selected if adequate sample could not be obtained in the selected community.

Data source

Vaccination information was extracted from three separate sources. First, each respondent was asked the question as “In your opinion, has this child received all recommended shots for his/her age?” If the respondent gave a positive reply, the child was considered as fully immunized per “recall.” Second, household vaccination booklet was reviewed and transcribed if available during the interview. If the vaccination booklet indicated that the child had received all vaccines recommended for his/her age, the child was considered as fully immunized per card. Otherwise, if no vaccination booklet was available, he/she was considered as not fully immunized per card as the standard practice. Third, the electronic vaccination records were extracted from ZJIIS by matching the surveyed child’s name, date of birth, and identification number of vaccination booklet. If these records indicated that the child had received all of the recommended vaccine doses for his/her age, he/she was considered fully immunized as determined by “medical records.” In addition, the card and recall indicators were combined into “card+recall.” Finally, since the VC would be underestimated as the assumption that a child without the vaccination booklet was considered as not fully immunized, we assessed immunization status per card among those children with the vaccination booklets, called “card among vaccination booklets holders”.

Statistical analysis

We used Epi Info 2002 version 3.3.2 (Centers for Disease Control and Prevention, GA) for data entry and cleaning and used Stata 14.0 (Stata Corp. 2015, Stata statistical software, college station, TX, USA) for data analysis. The sampling design and finite population were accounted for in the analysis, and the analytic process was weighted to adjust for differences in probability of selection.

Bivariate analysis was applied to assess the VC and its 95% confidence interval (CI) for children in each age group by different data sources. We evaluated the accuracy of VC based on household sources (vaccination booklet and recall) assuming the medical record was accurate. Five statistical measures of agreement were estimated through the following 2 × 2 table (Table 1).

Table 1. The definitions of measurements in this study

Agreement		Percentage of fully immunized children based on medical records
		Yes	No
Percentage of fully immunized children based on household source (immunization booklet and recall)	Yes	True positive (TP)	False positive (FP)
	No	False negative (FN)	True negative (TN)

Concordance was defined as the percentage of children with accurate household source immunization status [(TP+TN)/(TP+FN+FP+TN)]. Sensitivity was defined as the percentage of fully immunized children who were found to be fully immunized in the household source [TP/(TP+FN)]. The specificity was defined as the percentage of under-immunized children who were found to be under immunized in the household source [TN/(FP+TN)]. Positive predictive value was defined as the percentage of children found to be fully immunized based on the household source who were actually fully immunized [TP/(TP+FP)]. Negative predictive value was defined as the percentage of children found to be under immunized based on the household source who were actually under immunized [TN/(FN+TN)]. The Kappa statistic was also used to evaluate the agreement between data sources. Conventionally, a Kappa score ≤0.20 indicated slight to poor agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement, and 0.81–1.00 almost perfect agreement.¹⁵

Results

A total of 1800 children (600 per each age group) were interviewed and all the surveyed children were identified in ZJIIS. Of them, 92.5% of children aged 1 year, 89.8% of children aged 2 years and 73.2% of children aged 6 years had the household-retained vaccination booklet.

Estimates of VC varied by data source within each age group (Table 2), with the observed difference between different sources highest in the age group of 6 years and lowest in the age group of 1 year. VC estimated using vaccination booklet alone was substantially a little lower than that based on medical records (net bias 3.4–16.7% among the different age groups). Even among children with the vaccination booklets, the card-estimated coverage was 2.6–11.2% lower than that based on medical records, except for the age group of 1 year. VC based on parental recall was also lower, and ranged from 2.5% below (among children aged 1 year) to 16.7% points above (among children aged 6 years) than those based on medical records. The combined card+ recall measure produced results were still 1.0–13.2% lower than medical records.

Table 2. Estimated percentage (95% CI) of children fully immunized as estimated by various data sources

Data source	VC and 95% CI for age group (%)
	1 year old	2 years old	6 years old
Medical records	93.7(82.3-97.2)	91.5(86.0-93.4)	83.2(69.5-89.6)
Card	90.3(85.9-94.6)	84.7(78.2-88.7)	66.5(57.2-79.3)
Recall	91.2(86.4-93.9)	82.3(75.5-89.3)	58.7(50.2-62.4)
Card+recall	92.7(87.2-95.1)	86.5(80.3-91.0)	70.0(61.6-79.2)
Card among holders	94.1(90.3-97.6)	88.9(82.5-93.4)	72.0(63.8-80.5)

Notes (1) Children aged 1 year should receive 1 dose of BCG, 3 doses of HBV, 3 doses of DTP, 3 doses of PV, 1 dose of MCV, 1 dose of JEV and 2 doses of MenV-a; Children aged 2 years should receive 1 dose of BCG, 3 doses of HBV, 4 doses of DTP, 3 doses of PV, 2 dose of MCV, 2 dose of JEV, 2 doses of MenV-a and 1 dose of HAV; Children aged 6 years should receive 1 dose of BCG, 3 doses of HBV, 4 doses of DTP, 4 doses of PV, 2 dose of MCV, 2 dose of JEV, 2 doses of MenV-a, 1 dose of HAV, 2 doses of MenV-ac and 1 dose of DT. (2) Medical records meant the immunization status was determined based on ZJIIS; Card meant the immunization status was determined based on the vaccination booklet; Recall meant the immunization status was determined based on the response to survey question “In your opinion, has this child received all recommended shots for his/her age?”; Card+recall meant the immunization status was determined based on the vaccination booklet or recall for children without a vaccination booklet; Card among holders meant the immunization status was determined based on the vaccination booklet among children with vaccination booklets.

Estimated VC varied significantly between demographic characteristics, such as siblings, maternal age, maternal education, maternal employment status, residence and immigration status (Table 3). Differences observed between medical records and household sources were consistent among different demographic characteristics and different age groups.

Table 3. Estimated percentage of children fully immunized by demographic characteristic and data sources

Demographic characteristic	1 year old						2 years old						6 years old
	n	%	Medical records	Card	Recall		n	%	Medical records	Card	Recall		n	%	Medical records	Card	Recall
Overall	600	-	93.7	90.3	91.2		600	-	91.5	84.7	82.3		600	-	83.2	66.5	58.7
Child’s gender
Male	304	50.7	93.1	89.8	90.5		301	50.2	91.4	84.4	82.4		297	49.5	83.5	66.7	58.6
Female	296	49.3	94.3	90.9	91.9		299	49.8	91.6	84.9	82.3		303	50.5	82.8	66.3	58.7
Siblings
1	388	64.7	97.7	95.1	95.9		457	76.2	96.3	89.1	87.7		519	86.5	86.5	69.2	60.9
2	144	24.0	91.0	86.8	88.2		108	18.0	82.4	75.9	69.4		62	10.3	72.6	56.5	50.0
≥3	68	11.3	76.5	70.6	70.6		35	5.8	62.9	54.3	51.4		19	3.2	26.3	26.3	26.3
Maternal age (years)
<30	103	17.2	95.1	94.2	94.2		138	23.0	94.2	91.3	90.6		387	64.5	85.0	68.5	62.0
≥30	497	82.8	93.4	89.5	90.5		462	77.0	90.7	82.7	79.9		213	35.5	79.8	62.9	52.6
Maternal education
No/primary	8	1.3	62.5	62.5	62.5		26	4.3	46.2	42.3	46.2		82	13.7	51.2	46.3	46.3
Second level	72	12.0	84.7	79.2	80.6		109	18.2	88.1	64.2	63.3		195	32.5	83.1	66.7	58.5
College level	520	86.7	95.6	92.3	93.1		465	77.5	94.8	91.8	88.8		323	53.8	91.3	71.5	61.9
Maternal employment status
Home fulltime	199	33.2	95.0	93.5	94.0		225	37.5	94.2	88.0	87.1		75	12.5	89.3	77.3	66.7
Employed	401	66.8	93.0	88.8	89.8		375	62.5	89.9	82.7	79.5		525	87.5	82.3	65.3	57.5
Residence
Urban	318	53.0	90.3	86.2	87.4		286	47.7	85.3	79.4	76.9		306	51.0	74.8	61.4	52.9
Rural	282	47.0	97.5	95.0	95.4		314	52.3	95.5	89.5	87.3		294	49.0	91.8	71.8	64.6
Immigration status
Resident	198	33.0	96.5	93.4	93.4		196	32.7	96.4	89.3	89.8		255	42.5	89.8	74.1	67.5
Migrant	402	67.0	92.3	88.8	90.0		404	67.3	89.1	82.4	78.7		345	57.5	78.3	60.9	52.2

Notes (1) Children aged 1 year should receive 1 dose of BCG, 3 doses of HBV, 3 doses of DTP, 3 doses of PV, 1 dose of MCV, 1 dose of JEV and 2 doses of MenV-a; Children aged 2 years should receive 1 dose of BCG, 3 doses of HBV, 4 doses of DTP, 3 doses of PV, 2 dose of MCV, 2 dose of JEV, 2 doses of MenV-a and 1 dose of HAV; Children aged 6 years should receive 1 dose of BCG, 3 doses of HBV, 4 doses of DTP, 4 doses of PV, 2 dose of MCV, 2 dose of JEV, 2 doses of MenV-a, 1 dose of HAV, 2 doses of MenV-ac and 1 dose of DT. (2) Medical records meant the immunization status was determined based on ZJIIS; Card meant the immunization status was determined based on the vaccination booklet; Recall meant the immunization status was determined based on the response to survey question “In your opinion, has this child received all recommended shots for his/her age?”; (3) Values in bold indicate P < 0.05.

Statistical analyses to assess the validity of household sources are presented in Table 4. Concordance was the highest for recall (76.5–79.6%) and lowest for card estimates (32.5–45.5%). Sensitivity was <60% for all household sources, except for the recall source. Specificity was highest for card (89.2–90.4%), and lowest for recall estimates (14.5–42.6%). Positive predictive value was >75%, while negative predictive value was <50%, for all household sources. Kappa statistics generally indicated slight to poor agreement between household and medical sources, with fair agreement for recall and card among cardholders among children aged 1 year.

Table 4. Agreement of vaccination status between household sources and medical records

Age group	Household sources	Concordance	Sensitivity	Specificity	Positive predictive value	Negative predictive value	Kappa
1 year old	Card	36.4	32.7	90.4	92.6	11.9	0.13
	Recall	79.6	81.6	39.5	93.9	12.7	0.24
	Card+recall	55.7	54.2	70.9	91.5	20.5	0.16
	Card among holders	46.5	46.8	63.8	95.7	11.8	0.42
2 years old	Card	45.5	33.6	89.2	92.5	22.6	0.09
	Recall	77.6	85.4	42.6	88.3	46.1	0.05
	Card+recall	60.5	57.7	68.4	87.5	30.6	0.12
	Card among holders	59.2	49.3	86.7	92.1	33.9	0.17
6 years old	Card	32.5	16.8	89.2	83.6	22.0	0.02
	Recall	76.5	96.8	14.5	75.5	40.5	0.07
	Card+recall	52.6	55.1	51.2	78.6	22.8	0.04
	Card among holders	35.8	27.5	72.5	80.5	19.5	0.15

Notes (1) Children aged 1 year should receive 1 dose of BCG, 3 doses of HBV, 3 doses of DTP, 3 doses of PV, 1 dose of MCV, 1 dose of JEV and 2 doses of MenV-a; Children aged 2 years should receive 1 dose of BCG, 3 doses of HBV, 4 doses of DTP, 3 doses of PV, 2 dose of MCV, 2 dose of JEV, 2 doses of MenV-a and 1 dose of HAV; Children aged 6 years should receive 1 dose of BCG, 3 doses of HBV, 4 doses of DTP, 4 doses of PV, 2 dose of MCV, 2 dose of JEV, 2 doses of MenV-a, 1 dose of HAV, 2 doses of MenV-ac and 1 dose of DT. (2) medical records from ZJIIS were considered as the gold standard; (3) Card meant the immunization status was determined based on the vaccination booklet; Recall meant the immunization status was determined based on the response to survey question “In your opinion, has this child received all recommended shots for his/her age?”; Card+recall meant the immunization status was determined based on the vaccination booklet or recall for children without a vaccination booklet; Card among holders meant the immunization status was determined based on the vaccination booklet among children with vaccination booklets.

Discussion

All the 1,800 children identified in the household survey were registered in ZJIIS, which indicated that ZJIIS was able to determine true coverage since all children were registered with ZJIIS. In this study, an optimal VC among children aged 1 year was observed; however, the trend of VC declined as the age of children increased. We also found some demographic factors impacted the VC from medical or household resource. High concordance and sensitivity were found from the parental recall resource. A high specificity was observed from the card resource. Poor agreements in VC were found between medical resource and other resources. In most of the developing countries, VC survey based on household-retained cards and/or parental recall provided what was generally thought to be the most accurate estimate of VC available. However, there were some limitations of these estimates should be kept in mind. In many circumstances, the studies only accounted for the degree of uncertainty due to the sampling but not for the potential misclassification of immunization status, when the point estimate of VC was along with the widely standard error or confidence interval.^9,16,17 As such, the results from VC survey might not appear to be reliable than they truly were.

Our findings demonstrated that the household-retained vaccination booklet was not a sufficient data source for evaluating VC among children. Furthermore, the common VC survey could produce the biased results without using or comparing with the medical records and it could potentially induce inappropriate public health action. The difference in results from the different data sources was striking, which indicated the importance of identifying some reliable sources of immunization information. In each age group, a trend of a lower VC found in vaccination booklet was observed when it was compared with the medical records for the same children. Even when children without vaccination booklets were excluded from the analyses, the VC was underestimated in both age groups of 2 and 6 years. The specificity and positive predictive value were high for card (vaccination booklet), suggesting the card could identify most of the under-immunized children, and most of those who were found to be fully immunized based on their vaccination booklets were actually fully immunized. However, sensitivity and negative predictive values were low in both all children and those with cards, suggesting that data recorded on cards were sometimes incomplete. The reasons for incomplete information from vaccination booklet might be due to the parents forgot to bring the vaccination booklets to one or more visits or if the provider failed to record the vaccination information appropriately on the vaccination booklets for the doses administered.^18–20 Furthermore, there was another situation that the difference in the format of vaccination booklet from other provinces could not support to print the vaccination records on the vaccination booklet. As a result, a single missing date could cause a child to be misclassified as under-immunized because the missing data were indistinguishable from the missing vaccinations.²¹

In this survey, parental recall was estimated by asking whether their child had received all the recommended vaccinations for their children’s s age, which assumed that parents understood what vaccines and how many doses were recommended or that they had been told by vaccination providers that their children need not to be given any additional vaccine doses. However, questions soliciting more specific information on which vaccines had been given or the number of doses might have produced more interesting results. The estimated VC based on the parental recall was closer to that based on the vaccination booklet for children aged 1 year and 2 years, but not for the children aged 6 years. It suggested that the recall of immunization status was unreliable when the vaccination had occurred before a long time. Since routine vaccination is comprised of a series of vaccination events spanning a period of several months or even years other than a single event, it is particularly susceptible to induce the bias of misclassification.^22–24 Furthermore, the concordance, sensitivity, and positive predictive values were high, while the specificity and negative predictive values were <40%. It indicated that most parents of under-immunized children assumed that their children had received all recommended vaccinations for the relevant age, while most children whose parents reported not fully immunized were actually fully immunized.

The completeness and accuracy of data are critical to the validity of any survey. On the basis of our findings, we recommended using the multiple sources of immunization information to improve the validity of results on VC survey if it was feasible. We suggested that the medical records should be used to supplement the household information as a measure of cross-check or the record check should be conducted for a sample of respondents for validation at a minimum level. In settings where medical records are not available, other means of validation, such as comparing the estimates with those based on the administrative data, should be explored. In all cases, we suggested to make effort to scrutinize and describe the degree of accuracy and completeness of the household-retained vaccination booklet and recall through the expert’s s opinion and experience. Alternatively, statistical methods for analysis of the missing data such as the latent class analysis or the use of multiple imputation, based on the characteristics of children with complete household-retained records, might be valuable for assessing the VC among children without any written records. Investigators were encouraged to keep in mind the importance of ensuring the data accuracy and representation when the policymaker use these data to formulate the public health actions and policy.

Our findings also indicated some risk factors for the VC of children among all age groups. First, more siblings in the household significantly decreased the probability of being fully immunized. It was in line with other previous reports. Previous reports found children born in households with two or three children had lesser chance of being fully immunized, compared with children without siblings.^25,26 It was assumed that parents with several children would develop the confidence and believe modern health care is not necessary due to the experience accumulated from previous children and the household resource would be depleted as the number of children increases. Second, we found the maternal age was presented as an inverse predictor of the VC, as found in one study from Africa. The younger mother would possibly have a better utilization of the antenatal care and post-natal visits, which might lead to a better completion of immunization. Third, children with highly educated mothers had a more optimal VC than those with lower educated mothers, which was consistent with the reports from many countries.^27,28 In terms of common sense, high education background would facilitate mothers communicate with vaccination providers more effectively and help mothers have a better understanding and acceptance of vaccination service. Fourth, we found that children with employed mothers had a lower VC compared with those with home full-time mother which was in line with the previous reports. We assumed that mothers with jobs would not have enough time to spare for the childhood immunization and are less aware of the information on immunization.²⁹ Fifth, we found living in rural areas was a positive impact factor for immunization status in this study and it was consistent with the free seasonal influenza vaccination program among elders over 60 years of age in Beijing.³⁰ It was explained as the difference in the acceptability and the confidence in vaccination service between the urban and the rural areas. In rural areas, the vaccination providers had a closer relationship with their target population, making it convenient to recommend immunization policies. Due to this people-provider trust, accurate information was easily accepted by parents. The close-knit nature of the community, which was lacking in urban areas, resulted in the rapid spread of information and encourages parents bring their children to get vaccinations. At last, the association between the migrant children and the lower VC had been well established in Zhejiang province. It could be explained as the migrants would face the challenges of adapting to the new sociocultural environment, while the residents were better able to avail themselves of the public health services, as they were familiar with the living areas.^31,32

These results should be considered in light of potential limitations and considerations. First, we used the medical records information as the gold standard; however, these data might not have been perfect. Vaccine doses that were not administered might have been inadvertently recorded in ZJIIS, reducing the apparent differences between VC based on vaccination booklet and ZJIIS. Second, this study focused on six out of 90 counties in Zhejiang province. The results would not be generalized to the whole population as household-keeping records could be highly variable depending on local conditions and the factors influencing household record retention and the range of typical retention rates were not well known. Third, the vaccination coverage cluster survey manual had been updated in 2018 by the WHO; however, we did not apply the new recommendation in the calculation of sample size and survey procedures. It might affect the calculation of sample size as we did not estimate the spatial correlation of responses within clusters in the vaccination cluster survey, and this limitation should be addressed in future studies.

Conclusions

Household-retained vaccination booklets and parental recall were insufficient sources of information for evaluating the VC among children. Our findings emphasized the importance of identifying the reliable sources of vaccination records and reinforced the need for the awareness of the limitations of VC calculated based on the household-retained records or parental recall. The medical record such as ZJIIS will become a more common resource of VC estimate in a VC coverage survey, and we need to take interventions to make the vaccination booklet more consistent with the records from ZJIIS.

Author contributions

Y.H. and HK.L. conceived and designed the experiments; H.L. and Y.C. performed the experiments; Y.H. and Y.W. analyzed the data; Y.W. contributed reagents/materials/analysis tools; and Y.H. wrote the paper.

Disclosure of potential conflicts of interest

The authors declare that they have no competing interests.

Ethics approval and consent to participate

This study was approved by the ethical review board of Zhejiang provincial CDC. Written informed consent was obtained from a legal caregiver of each eligible child and health workers enrolled in this study.

Acknowledgments

We appreciate the vaccination staff from six CDCs at county level for their investigation and data collection.

Additional information

Funding

This study was funded by medical and health science and technology project of Zhejiang province [Grant number: 2020KY522].