Reliability and validity of a survey to identify vaccine hesitancy among parents in Changxing county, Zhejiang province

ABSTRACT

Background: To evaluate the validity and reliability of a survey to identify vaccine hesitancy among parents.

Methods: Cross-sectional survey of parents of 19–35 month old children was conducted in Changxing County, Zhejiang Province, through a questionnaire developed for the survey of the vaccine hesitancy. Construct validity was assessed by linking parental responses to their child’s immunization record. The association between mean% of days of under-immunization and the parental socio-demographics and the individual item response was explored via the univariate and multivariate analyses. Factor analysis was applied to confirm survey sub-domains and Cronbach’s α to determine the internal consistency reliability of sub-domain scales.

Results: We approached 336 households while 285 of them agreed to participate in this study. Education level and the parental ‘score of vaccination hesitancy’ were significantly associated with the mean% of days of under-immunization. Cronbach’s coefficients for the 3 sub-domain scales created by re-grouping the questionnaire’s items were 0.71, 0.83, and 0.72, respectively.

Conclusions: The survey represented a valid and reliable instrument to identify VHPs and it could help to screen parents to receive an intervention aimed at increasing acceptance of vaccinations.

KEYWORDS:

• Immunization
• survey
• reliability
• validity
• hesitancy

Introduction

Parental acceptance of childhood vaccination is declining in China after several negative median coverage on immunization in recent years. This is similar to the situation observed in US, where 12% of caregivers refused at least one recommended vaccine and an increasing number of parents filling philosophical exemptions for their children’s immunization check of school-entry.¹ The rise in vaccination resistance had driven several studies on parental decision-making on childhood vaccinations.^2-7 Vaccine-hesitant parents (VHPs) are a heterogeneous group who tend to have attitudes that fall between those of vaccine acceptors and rejecters on the vaccination acceptance. VHPs are defined as parents who only refuse one or two vaccines while agree to all others, or purposefully delay vaccines or have moderate concern on vaccine safety, and yet still want to trust and receive information on the immunization^8,9.

VHPs has soon become a focus of research target on improving the acceptance on vaccination among parents for two reasons¹⁰: first, VHPs are a much larger group than those completely reject vaccines. Second, they are potentially more amenable to behavior change as their attitude are not extreme and tend to seek information on childhood vaccines. To our knowledge, a comprehensive understanding on how to address VHP’s concerns is critical to convert the benefits of vaccine into practice. However, it is difficult for providers to communicate benefits and risk of vaccines with parents which may address their concerns and foster trust because of the lack of time materials and knowledge. For example, one study suggested parental concerns on vaccination are often neglected and another investigation found 40% of the surveyed physicians indicated that they dismissed a child if his/her parent refused one or more recommended vaccines.¹¹ These existing gaps would result in missed opportunities or drop-outs in vaccination, which resulted the consequent risk of prevalence of vaccine preventable diseases. Prior studies had used some survey instruments to explore the parental vaccination attitudes.^12-14 However, these studies were not explicitly designed for identifying VHPs and lack sensitivity. For making future policies to increase the immunization coverage, it is vital to have a reliability and validity survey method for identifying VHPs.

In this study, we sought to develop a survey questionnaire to accurately evaluate parental vaccine hesitancy by referring to the existing surveys, and to assess the internal consistency reliability of the questionnaire, and to evaluate the validity of the items of the questionnaire to discriminate between vaccine-hesitant and non-hesitant parents, through associating the response to items with the child’s immunization status.

Results

Socio-demographic characteristics

Totally, 336 households were approached while 285 of them agreed to participate in this study, with a response rate of 84.8%. Of the surveyed participants, 83.9% were mothers, 69.1% were under 30 years of age, 52.6% had a college education background, and 36.1% were migrant. Of the surveyed households, 28.8% had a household income of over 10000 RMB per month and 66.3% had only one child (Table 1).

Table 1. Characteristics of the participants (N = 285).

Characteristic	n	%	Characteristic	n	%
Relationship to child			Immigration status
Mother	239	83.9	Migrant	103	36.1
Father	46	16.1	Resident	182	63.9
Parent Age (years)			Household income per month (RMB)
< 30	197	69.1	< 5000	68	23.9
≥ 30	88	30.9	5000–10000	135	47.4
Education level			≥ 10000	82	28.8
≤ Junior school	39	13.7	Number of children in household
Senior school	96	33.7	1	189	66.3
College	150	52.6	2	79	27.7
			≥ 3	17	6.0

The mean% of days of under-immunization and its risk factors

Of the surveyed parents, 22.1% reported delaying and 24.9% had the hesitancy of get all the recommended vaccinations if they had another child. While over half of parents (53%) trusted the information they received about immunizations. In addition, 35.8% of the parents agreed that it would be better for children to get fewer vaccines at the same time, and 34.7% were concerned about the serious side effect from a vaccine, and 33.7% were concerned that childhood vaccines might not be effective (Table 2). In bivariate analyses, the hesitant response was significantly associated with a higher mean% of days of under-immunization from birth to 19 months old than the non-hesitant response for 11 out of 15 individual items in the questionnaire.

Table 2. Descriptive statistics, item association with immunization status, and factor analysis (N = 285).

Item	Response	n	%	Mean% of days under-immunized	p*	Factor loadings (>0.3)
						Behavior	Safety and efficacy	Attitude
Have you ever delayed your child’s vaccination for reasons other than illness or allergy?	Hesitate	63	22.1	13.1	<0.05	0.531
	Not sure	38	13.3	9.5
	Non-hesitate	184	64.6	1.5
How sure that following the recommended schedule is good for your child?	Hesitate	77	27.0	17.5	<0.01			0.611
	Not sure	42	14.7	8.2
	Non-hesitate	166	58.2	2.1
It is my role as a parent to question vaccinations.	Hesitate	94	33.0	10.4	>0.05		0.382
	Not sure	85	29.8	9.5
	Non-hesitate	106	37.2	9.7
If you had another infant today, would you intend to get all the recommended vaccinations?	Hesitate	71	24.9	14.7	<0.05	0.322		0.425
	Not sure	37	13.0	9.2
	Non-hesitate	177	62.1	1.7
Overall, how hesitant about childhood vaccinations would you consider yourself to be?	Hesitate	106	37.2	10.6	>0.05			0.325
	Not sure	60	21.1	9.7
	Non-hesitate	119	41.8	8.9
Children receiving more vaccinations are good for them.	Hesitate	93	32.6	10.5	>0.05			0.357
	Not sure	83	29.1	9.4
	Non-hesitate	109	38.2	8.8
I believe that many of the illnesses prevented by vaccines are severe.	Hesitate	62	21.8	12.4	<0.05			0.466
	Not sure	34	11.9	6.5
	Non-hesitate	189	66.3	1.1
It is better for my child to develop immunity by getting sick than vaccination.	Hesitate	98	34.4	19.6	<0.01			0.507
	Not sure	36	12.6	7.9
	Non-hesitate	151	53.0	2.4
It is better for children to get fewer vaccines at the same time.	Hesitate	102	35.8	20.7	<0.01	0.552	0.592
	Not sure	33	11.6	7.1
	Non-hesitate	150	52.6	2.5
How concerned about the potential serious side effect following vaccination?	Hesitate	99	34.7	19.6	<0.01		0.582
	Not sure	38	13.3	8.2
	Non-hesitate	148	51.9	2.2
How concerned about the vaccine efficacy?	Hesitate	96	33.7	18.8	<0.01		0.612
	Not sure	32	11.2	7.2
	Non-hesitate	157	55.1	2.0
The only reason for get childhood vaccinations is they can enter daycare or school.	Hesitate	94	33.0	10.5	>0.05	0.333
	Not sure	81	28.4	9.6
	Non-hesitate	110	38.6	8.7
I trust the information received about vaccine.	Hesitate	97	34.0	19.7	<0.01			0.632
	Not sure	37	13.0	8.2
	Non-hesitate	151	53.0	2.4
I can discuss my concern on vaccines with my doctors openly.	Hesitate	93	32.6	10.9	>0.05			0.316
	Not sure	83	29.1	9.5
	Non-hesitate	109	38.2	8.9
Overall, how much do you trust your child’s vaccination doctor?	Hesitate	77	27.0	15.2	<0.01			0.517
	Not sure	34	11.9	6.4
	Non-hesitate	174	61.1	1.3

^a Linear regression of mean% days of under-immunization and item responses (non-hesitant response as referent group).

Three parental socio-demographic determinants were found to be associated with the percentage of days of under-immunization. Children with a parent aged ≥ 30 years old, who had a college education, and who had only one child were under-immunized for a greater mean% of days of under-immunization than those with parents <30 years old (19.5% vs. 6.2%, p < 0.01), who had a junior school education or less (17.2% vs. 3.5%, p < 0.01), and who had ≥ 3 children (13.8% vs. 7.3%, p < 0.05), respectively. Other parent socio-demographics (relationship to child, immigration status and income) were not significantly associated with the mean% of days of under-immunization.

A significant linear association between the parental ‘score of vaccination hesitancy’ and their child’s vaccination status was observed, with a β-coefficient of 0.9. It meant that one point of the parental ‘score of vaccination hesitancy’ would increase 0.9% of the days of under-immunization (corresponding to 27 days under-immunized for all 14 vaccinations).

In multivariate regression models adjusted for the parental age, education level, and number of children in household, which were found significant in the univariate analysis, we found education level and the parental ‘score of vaccination hesitancy’ were significantly associated with the mean% of days of under-immunization (Table 3).

Table 3. Determinants associated with the mean% of days of under-immunization from birth to 19 months old.

Variable	β	t	p
Education level	1.112	3.065	<0.01
Score of vaccination hesitancy	1.023	2.992	<0.01
Constant	6.553	2.886	<0.01

Factor analysis

We revised the original 4 content domains into 3 similar sub-domain scales using the items that loaded most highly under each factor: a ‘behavior’ sub-scale with 2 items, a ‘safety and efficacy’ sub-scale with 4 items, and a ‘attitudes’ sub-scale with 9 items (Table 2). The Cronbach’s α-coefficients of the three scales were 0.71, 0.83, and 0.72, respectively.

Discussion

This study evaluated the construct validity and internal consistency reliability of the newly-developed questionnaire on the parental vaccine hesitancy. In this study, we combined the original ‘Attitudes’ domain with the ‘Trust’ domain as the items under both of these a priori domains loaded under the same factor. Although two of the Cronbach’s α coefficients of the three scales were at the lower limit of acceptability (0.71 and 0.72), our findings still indicated a reasonable initial structure and the revisions made to the groupings of the survey items based on the factor analysis should strengthen this structure. We also gave some potential explanations on the low coefficients found in two scales. The sample of this study might be homogeneous as they were selected from one county, which could underestimated the reliability of the questionnaire. Second, the questionnaire contained only 15 items and limited items would reduce the reliability of the questionnaire. Our results from multivariate analysis also confirmed that an increasing ‘score of vaccination hesitancy’ was significantly associated with increasing under-immunization. It obviously appeared that the questionnaire could validly measure the potential vaccine hesitancy. To our knowledge, the existing studies on vaccination hesitancy were not explicitly designed to identify VHPs and the validity and reliability were almost unclear.^8,13,15 As such, the most important advantage of this study was the development of a useful tool for identifying the parents with vaccination concerns. Besides, it may also be used to screen parents who receiving interventions aimed at increasing the acceptance of vaccination.

Additionally, the mean% of days of under-immunization from birth to 19 months old was associated with the ‘score of vaccination hesitancy’. It was noteworthy in the risk of contracting and transmitting a vaccine-preventable disease. The risk of infection of the vaccine-preventable disease was likely is known to increase with the longer delays in vaccination, especially for those highly contagious diseases. For instance, Glanz¹⁶ had found that refusal of ≥ 1 dose of pertussis containing vaccine had been associated with a 23-fold increased risk of infecting pertussis. One socio-demographic factors found to be associated with the mean% of days of under-immunization was the parental education level. It was consistent with the previous reports from home and abroad and it suggested that the higher education level dose not necessarily correlate with the positive health behaviour related to immunization.^17-21 The possible explanations included: first, parents with higher education background were more likely to have a career and they might have less time to spare for their child’s immunization. Second, parent with a higher education level might had an increased concern on the quality and safety as important measures for vaccines, leading to a lower rate or an untimely of vaccination uptake.

This study also yielded interesting results on the associations between the response of some individual items and childhood immunizations, most of which were demonstrated in the previous reports. For example, 22.1% of the participants stated that they had delayed their children’s immunization for reasons other than illness or allergy, which was higher than previous studies. Gust²² found 13% of the surveyed parents had delayed a vaccination for their child in a 2003–2004 national survey and Mokdad²⁰ found 14% of the surveyed parents had delayed childhood immunizations for reasons other than their child being ill. Additionally, 24.9% of the surveyed parents would refuse to have their children vaccinated with all the recommended vaccinations. It was higher than the results Freed¹ and Gust²² found, which were only 12% and 6%, respectively.

In this study, we found a lower proportion of parents who were concerned on the vaccine safety as that Freed¹ found in 2009 in U.S.(34.7% vs. 54%). The difference between two studies were not clear, however, the effective way to address the unnecessary safety concern was the delivery of information on vaccination in a convincing manner. According our previous experience, communication channel made an important role in delivering the knowledge on immunization.²³ Hence, local public health officials might redesign the immunization information program to be more tailored to the specific subgroups of parents and might address the current safety concerns. Otherwise, the continued high vaccination coverage might be risky due to the refuse of vaccination among parents. Of the surveyed parents, 35.8% agreed that children should get fewer vaccines at the same time, which was higher that the other reports.^24,25 As we know, simultaneous administration of all vaccines is an essential component of childhood immunization programs and it is particularly important when return of the recipient for further vaccination was uncertain, or imminent exposure to several vaccine-preventable diseases (VPDs) is expected.²⁶ The World Health Organization supported multiple vaccine injections in a single visit, and encouraged this practice based on the benefits they confer.²⁷ As such, a refresh-training target vaccination physicians and a health education program target parents were needed to confer the importance of getting more vaccine at a single visit. We found the concern of the vaccine efficacy was associated with the under-immunization, which was similar to the previous studies in other settings. These studies found parents who were not convinced of the efficacy of vaccines were more likely to consider claiming an exemption from vaccination for their children^19,20,28,29.

Furthermore, there were other items of attitudes that influence the under-immunization. If parents did not consider the VPDs as severe enough to take preventive action or they did not consider any benefit from immunization, they would not to have their children get vaccination in a timely manner. For example, the mean% of days of under-immunized was highest among children whose parents considered the VPDs were not severe in this study. Similarly, Gellin had reported that lack of the firsthand knowledge on VPDs among parents was an immediate threat to the health of their children.³⁰ Parents may also be hesitant with the immunization as they perceive that the natural immunity is preferable to vaccine-induced immunity. A preference for disease-induced immunity had been suggested as one reason for vaccine refusal among parents in a previous study.³¹ Educating parents about VPDs, together with the vaccines, might be one way to impart the importance of the childhood immunization program. We found that parents generally trusted the vaccination information they received (53.0%) and the vaccination physicians (61.1%). Vaccination providers were widely considered by parents as an important factor in their decision-making on childhood immunization.^19,28 Not only did the majority of parents state that their common information on vaccination were from providers, but also they changed minds of delaying or refusing a vaccination after receiving the information. In this study, our results reinforced the fact that provider-parent communication on immunizations was pivotal in transferring the benefits of vaccination to practice.

This study had several limitations. First, our results might reflect current perceptions of immunizations other than perceptions at the time they were making immunization decisions, because we surveyed the parental attitudes, beliefs and behaviors on vaccination after the timeframe in which they were making the relevant decisions. As such, we might not be able to know whether the association of a child’s vaccination status with their parent’s responses reflects their current or past vaccination attitudes or beliefs since the perceptions might change over time. Second, the association between responses to individual items and the mean% of days of under-immunization detected 4 out of 15 items that did not discriminate between hesitant and non-hesitant responses. There might be limitations when we directly used these items in surveys to category parents as a hesitant group to understand how to improve their vaccination behavior. Third, using the education background to assess the association with the mean% of days of under-immunization might not be exact and appropriate since the health (or vaccination) literacy skill would be more detailed and specific. We will try to re-evaluate the vaccination hesitancy through specific questionnaires in future studies, considering health (or vaccination) literacy as a potential factor. Fourth, the participants were limited to the population of a northwest county in Zhejiang province. Therefore, the results might not be generalizable to populations outside of the target geographic area.

Conclusions

The survey represented a valid and reliable instrument to identify VHPs and it could be a useful tool for identifying the parents with vaccination concerns and help to screen parents to receive an intervention aimed at increasing acceptance of vaccinations.

Methods

Development of the questionnaire

We used two steps to develop the questionnaire for the survey of the vaccine hesitancy. First, we drafted a preliminary questionnaire through identifying the domains in previous studies on parental vaccination hesitancy. The items were borrowed or modified from existing survey questionnaires .^25,32,33 Second, the draft questionnaire was reviewed by seven experts through a seminar. We convened an expert panel that included two experts from national immunization program of Chinese center for disease control and prevention (CDC), three experts from local immunization program of CDCs at city level, and two expert from department of epidemiology, Zhejiang University. We also conducted a pre-test among five parents to evaluate the content validity and feasibility in the field. Finally, the questionnaire contained 15 items under 4 content domains, including vaccination behavior (5 items), beliefs on vaccine safety and efficacy (6 items), attitudes on vaccine mandates (1 item) and trust (3 items). The questionnaire utilized 3 different response formats: 5-point Likert scale (e.g. strongly agree, agree, not sure, disagree, and strongly disagree), and an 11-point scale (from ‘0: not sure at all’ to ‘10: completely sure’) and the ‘yes/not sure/no’. Besides, the parental socio-demographic information was included in the questionnaire. The questionnaire took less than 10 minutes to complete in the pre-test.

Overall, the responses for survey items were collapsed into 3 categories: ‘hesitant’, ‘not sure’, and ‘non-hesitant’, which were assigned a score of 2, 1 and 0, respectively. For the items with a 5-point Likert-scale ranging from ‘strongly disagree to strongly agreed’, hesitant responses corresponded to the collapsed responses of ‘strongly disagree and disagree’ or ‘strongly agree and agree’, according to the content of items. For the items with a 5-point Likert-scale ranging from ‘not at all concerned to very concerned’, hesitant responses corresponded to the collapsed responses of ‘somewhat and very concerned’ or ‘not at all or not too concerned’, according to the content of items. For the item with a 5-point Likert-scale ranging from ‘not at all hesitant to very hesitant’, the hesitant response corresponded to the collapsed responses of ‘somewhat or very hesitant’, while the non-hesitant response corresponded to ‘not at all or not too hesitant’. Lastly, for the items with an 11-point Likert-scale, hesitant responses corresponded to the collapsed responses 0–5, not sure responses to 6–7, and non-hesitant responses to 8–10 (Table 4).

Table 4. The content of the survey questionnaire.

Content domain	Item	Response format and category
		Hesitant	Not sure	Non-hesitant
Behavior	1. Have you ever delayed your child’s vaccination for reasons other than illness or allergy?	Yes	Not sure	No
	2. How sure that following the recommended schedule is good for your child?	0–5 points	6–7 points	8–10 points
	3. It is my role as a parent to question vaccinations.	Strongly Agree/Agree	Not sure	Disagree/Strongly Disagree
	4. If you had another infant today, would you intend to get all the recommended vaccinations?	No	Not sure	Yes
	5. Overall, how hesitant about childhood vaccinations would you consider yourself to be?	Somewhat hesitant/Very hesitant	Not sure	Not at all hesitant/Not too hesitant
Beliefs	6. Children receiving more vaccinations are good for them.	Disagree/Strongly Disagree	Not sure	‘Strongly Agree/Agree
	7. I believe that many of the illnesses prevented by vaccines are severe.	Disagree/Strongly Disagree	Not sure	‘Strongly Agree/Agree
	8. It is better for my child to develop immunity by getting sick than vaccination.	‘Strongly Agree/Agree	Not sure	Disagree/Strongly Disagree
	9. It is better for children to get fewer vaccines at the same time.	‘Strongly Agree/Agree	Not sure	Disagree/Strongly Disagree
	10. How concerned about the potential serious side effect following vaccination?	Somewhat concerned/Very concerned	Not sure	Not at all concerned/Not too concerned
	11. How concerned about the vaccine efficacy?	Somewhat concerned/Very concerned	Not sure	Not at all concerned/Not too concerned
Attitudes	12. The only reason for get childhood vaccinations is they can enter daycare or school.	Yes	Not sure	No
Trust	13. I trust the information received about vaccine.	Disagree/Strongly Disagree	Not sure	‘Strongly Agree/Agree
	14. I can discuss my concern on vaccines with my doctors openly.	Disagree/Strongly Disagree	Not sure	‘Strongly Agree/Agree
	15. Overall, how much do you trust your child’s vaccination doctor?	0–5 points	6–7 points	8–10 points

Study area and target population

The survey was conducted in Changxing County, Zhejiang Province, East China in May, 2015. The total population of Changxing was 625,325 according to the census data of 2014 from Zhejiang provincial bureau of statistics and Changing consisted of 19 townships. The survey sample was parent of child aged 19–35 months (born between June, 2012 and October, 2013). According to the Chinese expanded program on immunization (EPI)³⁴, a child needs to get 11 vaccines or 22 vaccine doses before 7 years old (Table 5). The main reason for choosing this age range was that it represented an appropriate time period to parental viewpoints on immunization as it occurred after the 14 dose-series before 18 months old, which contained the vaccinations that most often prompt a parent to question, delay, or refuse its administration. Furthermore, it could also maximally reduce the possibility of recall bias. Children lived continuously in Changxing County since the birth were included, which helped ensure that the number of vaccinations administered out of Zhejiang province among those children would be negligible.

Table 5. Immunization schedule in Chinese expanded program on immunization and minimum ages and intervals for specific vaccinations.

Vaccine	Dose	Recommended age (months)	Minimum acceptable age (days)	Minimum interval (days)	Age when under-immunized count initiated (days)	Maximum number of days under-immunized at 19 months (days)
BCG	-	Birth	0	-	32	581–32 = 549
HepB	1	Birth	0	-	32	581–32 = 549
	2	1	28	28	93
	3	6	168	56	215
PV	1	2	56	-	93	581–93 = 488
	2	3	84	28	123
	3	4	112	28	154
DTP	1	3	84	-	123	581–123 = 458
	2	4	112	28	154
	3	5	140	28	184
MPV-A	1	6	168	-	215	581–215 = 366
	2	9	252	84	306
MR	-	8	224	-	276	581–276 = 305
JEV	-	8	224	-	276	581–276 = 305
Total	14					3020

Note: BCG: Bacillus-Calmette-Guerin vaccine; HepB: hepatitis B vaccine; PV: polio vaccine; DTP: diphtheria-tetanus-pertussis vaccine; MPV-A: meningococcal polysaccharide vaccine-type A; MR: measles-rubella vaccine; JEV: Japanese encephalitis vaccine; MMR: measles-mumps-rubella vaccine; HepA: hepatitis A Vaccine; MPV-AC: meningococcal polysaccharide vaccine-type A and C; DT: diphtheria-tetanus vaccine.

Data collection and item scoring

Parents of the selected children were visited at home by interviewers, who were trained on the questionnaire by the study team. Demographic information and socio-economic characteristics of the selected child, the mother, and the household were collected. Immunization records were transcribed from the immunization cards and validated through Zhejiang provincial immunization information system and its detail functions could be found elsewhere³⁴.

Overall, the responses for survey items were collapsed into 3 categories: ‘hesitant’, ‘not sure’, and ‘non-hesitant’, which were assigned a score of 2, 1 and 0, respectively. The definitions for three categories of each survey items could be found in Table 4. The rules of combinations of different response formats were discussed by the experts before conducting the field survey, basing on the previous studies and the experts’ experience. The total raw score of each parent was obtained by aggregating each item. After that, we converted this raw score to a 0–100 scale through a simple linear transformation and the index was named as ‘score of vaccination hesitancy’.

Primary measurement

The primary measurement of interest was the under-immunization. In this study, the timeliness of each of 14 vaccinations recommended through 19 months of age was chosen as we considered this estimate of under-immunization was more sensitive by accounting for delayed vaccination, compared with the traditional outcome such as the number of missed doses.

The method to calculate the estimate of timeliness of vaccination included three steps: first, the age of receiving the specific vaccination was determined in days. Second, we compared the actual age of receiving the specific vaccine dose with the Chinese EPI schedule for that dose, accounting for both the minimum acceptable age of each dose and the minimum interval between doses. If a specific dose received 5 days prior to the minimum acceptable age or the interval for that vaccine, it was considered as too early and not counted. If a specific dose received beyond the recommended age or interval, the difference between the age the dose was received and the latest age in which it should have been received was calculated and covert to days. Third, the days of under-immunization was added up for each of the 14 doses per child till 19 months old or 580 days of age (30.5 days per month in average). The under-immunization was expressed as the percentage of days that a child was under-immunized from birth date to 19 months old for all 14 vaccinations combined. To do so, we summed together the calculated days under-immunized for each vaccine and divided this total number by the maximum number of days a child could be under-immunized in this interval (3020 days). For example, if a child received only MR and the third dose of PV late at 300 days of age but received other vaccinations on time, the percentage of under-immunization from birth to 19 months was 5.6%: [BCG(0)+HepB(0)+DTP(0)+ MPV-A(0)+JEV(0)+PV(300–154)+MR(300–276)]/3020 = 0.056 (Table 5).

Sample size

The sample size was estimated based on the formula as follows³⁵:${\mathrm{N}}_{\mathrm{m}\mathrm{i}\mathrm{n}}=deff\times \frac{z_{\left(1-\alpha /2\right)}^2\times p\times \left(1-p\right)}{d^2}$. To reach the estimates of coverage at the significance level of a two-tailed α error of 5% and a permissible error (d) of 0.05, assuming the expected percentage of days under-immunized of 10% and a design effect (deff) of 2, the minimum sample size required for each city was 277 eligible children. For the convenience and feasibility of the field work, we determined 15 eligible children for every 19 township, corresponding to 285 children, as our final sample size.

Survey procedures

First, one community or village was elected in every township by the simple ballot from the list of all communities and villages. Second, we selected the first household by using the table of random numbers in the selected community or villages. Only one eligible child per household was randomly selected for the survey. Household in which somebody was living, but without any response, was re-scheduled for another attempt till three times failed. Third, we selected the subsequent households, by turning to the right and visiting the adjacent households. If we could not find enough sample in the selected community or village, we moved to the closest one in the same town to survey the remaining children by following the steps above.

Data analysis

The responses of each items were analyzed by using the descriptive statistics. In the univariate analyses, the parental socio-demographics and the mean% of days of under-immunization was assessed by the analysis of variance. The bivariate association between the individual item response and the mean% of days of under-immunization was evaluated by the linear regression model. In the multivariate analyses, the association between the ‘score of vaccination hesitancy’ and the mean% of days of under-immunization at 19 months using multivariate linear regression models (stepwise method) while adjusting for the significant parental socio-demographics in the univariate analysis (p < 0.1).

Factor analysis was used to determine the number of latent constructs of the questionnaire, the items that loaded on these constructs, to reduce the number of items. The number of constructs was considered as those occurring before the break in the curve on a scree test of the eigenvalues. A factor loading threshold of 0.3 was applied to explore which item belonged to the identified construct. When an item loaded on more than one factor, it was placed under the factor in which it loaded highest. Sub-domains consisted of all the items that loaded highest under a particular construct. The internal consistency of each sub-domain was evaluate by using Cronbach’s α. We used Stata 11.0 (Stata Corp. 2009, Stata statistical software, college station, TX, USA) for all our analyses and a p-value of 0.05 or less was considered to be significant.

Disclosure of potential conflicts of interest

No potential conflict of interest were disclosed.

Author Contributions

Yu Hu conceived and designed the study; Yu Hu and Yaping Chen performed the study; Yu Hu and Ying Wang analyzed the data; Hui Liang and Ying Wang contributed reagents/materials/analysis tools; Yu Hu wrote the paper.

Ethical considerations

This study was approved by the ethical review board of Zhejiang provincial CDC. Written informed consent was obtained from a parent of each eligible child enrolled in this survey.

Acknowledgments

Great acknowledgement go to the immunization staff from Changxin CDC for their investigation and data collection.