Effectiveness of a smartphone app to increase parents' knowledge and empowerment in the MMR vaccination decision: A randomized controlled trial

ABSTRACT

Researchers are trying to build evidence for mhealth effectiveness in various fields. However, no evidence yet is showing the effectiveness of mhealth on parents' attitudes and behavior with regard to recommended vaccination of their children. The aim of this study was to look into the effects of 2 smartphone-based interventions targeting MMR vaccination knowledge and psychological empowerment respectively. The interventions used gamification features and videos in combination with text messages. We conducted a 2x2 between-subject factorial randomized controlled trial (absence/presence of knowledge intervention X absence/presence of empowerment intervention) with parents of young children in Italy. We randomly allocated 201 eligible participants to one of the 4 conditions. Data were collected by questionnaires at baseline and posttest. Primary outcomes were MMR vaccination knowledge, psychological empowerment, risk perception, and preferred decisional role; secondary outcomes included MMR vaccination intention, attitude, confidence, and recommendation intention. A significant gain in vaccination knowledge was reported by all experimental groups compared with the control (F(3,179) = 48.58, p < .000), while only those receiving both interventions reported a significant increase in their psychological empowerment (t(179) = −2.79, p = .006). Participants receiving the intervention targeting knowledge reported significantly higher intention to vaccinate (t(179) = 2.111; p = .03) and higher confidence in the decision (t(179) = 2.76; p = .006) compared with the control group. Parent-centered, gamified mobile interventions aimed at providing parents with vaccination-related information can be used to increase their knowledge, their intention to vaccinate as well as their confidence in the vaccination decision.

KEYWORDS:

• empowerment
• intervention
• knowledge
• mHealth
• MMR vaccination
• smartphone app

Introduction

The number of smartphone applications designed for health purposes has grown exponentially in the past 15 y and is still rapidly rising.¹ Mobile apps can provide tremendous opportunities to influence people's health behavior thanks to a combination of unique characteristics.^2,3 They are at the same time personal, connected, easy to use, customizable, empowering, increasingly technological and always at hand.^{1,4, 5} Their range of application is extremely wide and, more recently, they have made an appearance for vaccination-related purposes as well. Immunization apps include features: they can provide information on different vaccinations and on disease activity in a given area,^6-8 calculate one's risk of catching a disease,⁹ offer a reminder about vaccines,^7,10-13 and track, record and update immunization information.¹⁴ The Immunization Action Coalition (IAC) lists 19 free immunization apps directed either to healthcare/immunization providers or to patients/parents and offered by recognized institutions,¹⁵ but a search with the keyword “vaccin*” on Google Play generates as many as 249 results.¹⁶

Despite coming from reliable and certified organizations, a major limitation of almost all immunization apps is the lack of evidence of their effectiveness, as a recent review also concluded.¹⁷ As a matter of fact, only one immunization app directed at parents was tested empirically.¹⁸ This is the first study aiming at testing in a randomized controlled trial 2 versions of a smartphone-based application, one to increase parents' knowledge about the MMR vaccination and the other to augment empowerment in the MMR vaccination decision. The theoretical background is provided by the Health Empowerment Model. In this model, Schulz and Nakamoto¹⁹ suggested that acceptance or refusal to vaccinate one's child might arise from several factors including beliefs based on completely or partly incorrect information (knowledge), in addition to a more or less strong sense of autonomy (empowerment).

The intervention targeting knowledge used the device of gamification. Gamification is defined as “the use of game design elements in non-game contexts.”^20,21 It represents an increasingly popular field of research and application due to its potential to increase users' engagement,^22-25 satisfaction,^23,26,27 enjoyment of activities,^28-30 task performance,^30-32 participation,^22,33 empowerment,²⁵ learning,^27,34-39 attitude,,^22,35,40 and in reinforcing a behavior.³⁷

The intervention targeting empowerment used a narrative presented by a video format and interpersonal communication elements through text messages.⁴¹ Recent studies found that web-based interventions to increase patient empowerment had positive effects^42-45 and that the use of both narratives and interpersonal communication may influence health outcomes and one's vaccination decision, as well as facilitate decision-making.^46-52

Grounded in the theoretical model, the goal of this study is to target the 2 constructs of knowledge and empowerment through a smartphone app and enhance their effects on MMR vaccination future behavior, attitude and recommendation, while testing, in a similar approach, the use of interpersonal communication and gamification as boosters. This is the first RCT that includes gamification, visual narrative, and interpersonal communication features as part of an experimental manipulation and studies the effect of a smartphone app targeting vaccination knowledge, respectively literacy, and empowerment on vaccination-related decisional and behavioral outcomes of parents of young children.

Results and discussion

Participants' characteristics

Initially, 255 participants agreed to participate to the study and 233 accessed the baseline questionnaire. Of these, 26 did not meet the inclusion criteria and 5 did not complete the baseline questionnaire. We randomly allocated the resulting 202 subjects to one of the 3 experimental groups or the control group. After the post-test survey was closed, 5 subjects were removed from the control group as they reported having known the app. We further removed 13 subjects who did not access the app or did not complete the post-test survey. The final sample (N = 184) was mainly composed by mothers (94.6%), highly educated parents (60.4%) and Italian nationals (98.4%). The average age was 34.2 y (SD = 4.66; range = 21–47) and most participants had only one child (77%) compared with official statistics indicating that the average number of children per woman is 1.43.⁸³ Participants' characteristics can be found in Table 1. Italian nationals, as well as highly educated parents, were overrepresented in our sample, as the migrant population in Italy is estimated at 11.5% at January 1st, 2016, while the number of Italian residents owning an academic degree is 12.3% of the total population.⁸³

Table 1. Participants' characteristics.

	Experimental group
	1	2	3	4	Tot.
	n = 48 (26%)	n = 45 (24%)	n = 47 (26%)	n = 44 (24%)	N = 184 (100%)
Age	M = 33.44, SD = 4.27	M = 34.49; SD = 4.46	M = 33.98, SD = 4.86	M = 35, SD = 5.06
Gender
Women	n = 43 (25%)	n = 43 (25%)	n = 46 (26%)	n = 42 (24%)	174 (95%)
Men	n = 5 (50%)	n = 2 (20%)	n = 1 (10%)	n = 2 (20%)	10 (5%)
Nationality
Italy	n = 45 (25%)	n = 45 (25%)	n = 46 (26%)	n = 43 (24%)	179 (97%)
Brazil	n = 0	n = 0	n = 1 (100%)	n = 0	1 (1%)
Morocco	n = 1 (100%)	n = 0	n = 0	n = 0	1 (1%)
Mexico	n = 1 (100%)	n = 0	n = 0	n = 0	1 (1%)
Education
Middle school	n = 3 (75%)	n = 0	n = 1 (25%)	n = 0	4 (2%)
Professional school	n = 4 (36%)	n = 2 (18%)	n = 2 (18%)	n = 3 (28%)	11 (6%)
High school	n = 17 (30%)	n = 13 (23%)	n = 13 (23%)	n = 14 (24%)	57 (31%)
University	n = 23 (21%)	n = 30 (27%)	n = 31 (28%)	n = 26 (24%)	110 (60%)
Number of children
1	n = 40 (27%)	n = 35 (25%)	n = 35 (25%)	n = 33 (23%)	143 (78%)
>1	n = 8 (20%)	n = 10 (24%)	n = 12 (29%)	n = 11 (27%)	41 (22%)

Randomization Check

We found no significant differences across the 4 groups in terms of participants' age (F(4,179) = 0.94; p = .42), gender (χ² = 3.47; p =.32), educational level (F(4,179) = 2.24; p = .08), number of children (χ² = 6.18; p =.72), control preference (χ² = 10.90; p = .54), nationality (χ² = 8.67; p = .47), age of youngest child (F(4,179) = .634; p = .59), empowerment (F(4,179) = .431; p = .73) and knowledge (F(4,179) = .79; p = .5).

Primary Outcomes

The covariate, pre-experiment knowledge, was significantly related to the post-experiment knowledge (F(4,179) = 82.07; p < .000). There was a significant main effect of the experimental group on the level of post-experiment knowledge after controlling for the effect of pre-experiment knowledge (F(3,179) = 48.58; p < .000). Planned contrasts revealed that all 3 experimental groups increased post-experiment knowledge compared with the control group (knowledge intervention only t(179) = 9.11; p < 0.000; empowerment intervention only (t(179) = 4.40; p < .000; both interventions (t(179) = 11.00; p < .000). Interestingly, all pairwise comparisons between experimental groups also showed significant differences. The group receiving both interventions has the highest knowledge gain, followed by the group receiving the knowledge intervention only, the empowerment intervention only and, finally, the control group with the lowest knowledge level. This indicates that empowering parents will increase their information seeking and favor learning; giving the information also increases their knowledge, but it is only by giving the information and pushing them to search for more information that the highest gain is generated. Further results confirmed between groups differences in terms of online information seeking (F(3,180) = 11; p = <.000). A t-test revealed that participants receiving both interventions searched information more often compared with those in the knowledge intervention only group (t(93) = 2.09; p = .04).

The covariate, pre-experiment psychological empowerment (a summative score of the 4 subdimensions of meaningfulness, impact, competence and self-determination), was significantly related to the post-experiment psychological empowerment (F(4,179) = 77.750; p < .000). There was a significant main effect of the experimental group on the level of post-experiment empowerment after controlling for the effect of pre-experiment empowerment (F(3,179) = 2.74; p = .04). Planned contrasts revealed that the knowledge intervention only did not increase post-experiment empowerment compared with belonging to the control group (t(179) = 1.68; p = .09), as well as belonging to the empowerment only group (t(179) = 1.03; p = .302). However, receiving both interventions increased post-experiment empowerment against the control group (t(179) = -2.79; p = .006). This suggests that a shift in empowerment can take place only when empowering interventions also offer tangible information about the domain where empowerment is advocated.

Secondary outcomes

The ANCOVA that was conducted to determine any between groups difference in terms of “post-experiment intention to vaccinate” showed a significant main effect (F(3,179) = 4.287; p = .006). Planned contrasts revealed that the group receiving the intervention addressing vaccination knowledge showed a stronger post-experiment intention compared with the control group (t(179) = 2.111; p = .03). On the other hand, the group receiving the intervention addressing empowerment (t(179) = −1.156; p = .24) and the group receiving both interventions (t(179) = -.737; p = .46.) showed similar intention to vaccinate compared with the control group. As expected, the pre-experiment intention was significantly related to the post-experiment intention (F(4,179) = 71.83; p < .000).

Similar results were found analyzing the “post-intervention confidence in the vaccination decision." There was a significant main effect of the experimental conditions on the level of post-experiment confidence after controlling for the effect of pre-experiment confidence (F(3,179) = 4.44; p = .005). Planned contrasts revealed that belonging to the group receiving the knowledge intervention increased the post-experiment confidence compared with belonging to the control group (t(179) = 2.76; p = .006). On the other hand, belonging to the group receiving the empowerment intervention (t(179) = −0.665; p = .5) or to the group receiving both interventions (t(179) = .056; p = .62) did not have an impact on the post-experiment confidence compared with the control group. The covariate, pre-experiment confidence, was significantly related to the post-experiment confidence (F(4,179) = 156.04; p < .000).

Discussion

First of all, these findings suggest that increasing parents' knowledge about the vaccination using gamification can lead to an increase in their vaccination intention. This is in line with the literature, which found an association between poor objective knowledge of the vaccination and delayed or refused vaccination status^53-56 as well as lower intention to vaccinate as predicted by poor subjective knowledge about the vaccine.⁵⁷

Second, the empowerment intervention did not have the desired effect on vaccination intention, and neither did the combination of both interventions. The latter result is unexpected, though there could be possible explanations. First, considering that the empowerment intervention invited participants to search information and make an autonomous and informed decision, it could be that an information overload might have confused them. Secondly, in light of many parents being aware that the vaccinations in question were officially recommended, a call for an autonomous decision might have been understood as a call for a decision against the official recommendation.

A third and simpler explanation is that the intervention combining both strategies is excessively complicated as well as cognitively and emotionally demanding. Research has found that combined interventions are not always more efficient than simple interventions using one strategy.⁵⁸

Another finding is that there was no significant main effect of the experimental group on the level of post-experiment opinion after controlling for the effect of pre-experiment opinion (F(3,179) = 1.02; p = .38). As expected, the pre-experiment opinion, was significantly related to the post-experiment opinion (F(4,179) = 99.76; p < .000). Similarly, there was no main effect of experimental conditions on the post-experiment recommendation intention (F(3,179) = 1.54; p = .24). Also in this case, the pre-experiment recommendation intention was significantly related to the post-experiment recommendation intention (F(4,179) = 98.8; p < .000).

The first insignificant findings could be ascribed to the operationalization of opinion in the survey. The response options (“Against” to “In favor”) differed significantly from those proposed for intention, which rather measured probability. The lack of significant results in the second case are to ascribed to the fact that, while intention and opinion are theoretically related, the concept of referral or recommendation might depend on personality factors or from parents' previous experience with the vaccination staff and facilities.^59,60

Materials and methods

Sample

Recruitment of the participants lasted from April until November 2016. A marketing agency was contacted to send the study invitation to potential participants by email. Potential participants were identified on the basis of the personal information they provided when they subscribed to one or more websites managed by the marketing agency. To be eligible, participants had to (a) have at least one child born after September 1, 2015¹ , (b) be resident in Lombardy² , one of the 20 administrative regions of Italy, and (c) own a smartphone with Internet connection. The final sample was composed by 184 subjects divided into 4 groups.

Experimental design

A smartphone application was developed to deliver 2 interventions, one targeting MMR vaccination literacy and the other targeting empowerment in the MMR vaccination decision. The study design was a 2x2 between-subject factorial randomized controlled trial. The factors studied were presence or absence of 2 interventions, resulting in 4 possible experimental groups. Participants were randomly allocated to one of the 3 experimental groups or to the control condition. The first group received the app containing only the intervention targeting the MMR literacy, the second one received the app containing only the intervention targeting empowerment and the third one received the app containing both the knowledge and empowerment interventions. The control group did not receive the app.

In the first intervention aimed at increasing participants' literacy about the MMR vaccination, users received 35 questions distributed on a time span of 10 d. Once answered, each question unblocked an explanation of the answer through textual content. Each correct answer would earn participants several points according to the weight of each question, while no points were given for wrong answers or if no answer was given by midnight of the day.⁶¹ To provide a gamified experience, participants could see their score and compare it to that of the other participants through a leaderboard. Furthermore, participants were awarded a shopping voucher the value of which increased with their performance in the quiz.⁶² All questions, answers, and contents were developed following a review of the scientific literature on parents' decision on the MMR vaccination^63-69 and of major public health websites (CDC, ECDC, NHS), and later validated by a panel of medical experts based in Italy. We asked the panel members to identify any inaccurate, inappropriate or incomplete information and suggest possible alternatives, as well as decide on each question's weight (from 1 to 5 points) in terms of importance.

In the second intervention, aimed at enhancing psychological empowerment, users received 2 videos (one on the first day lasting 4 minutes and one on the last day lasting approximately one minute) and 8 messages. We developed the script of messages and the 2 videos following Spreitzer's conceptualization of empowerment as a set of 4 sub-dimensions: competence, self-determination, importance and impact. In addition, we included active information orientation as a fifth sub-dimension of empowerment, following our previous qualitative work on parental psychological empowerment in the vaccination decision.⁷⁰ In the 2 videos, an actress acting as a mother reports that she became able to make an empowered decision about MMR by collecting reliable information from multiple sources, and by thinking about the importance and the impact of the decision. In the end, she addresses her audience encouraging them to make an informed, empowered decision. The video's viewer was addressed in the second person to increase participant's involvement.⁷¹ Text messages were designed to reinforce the messages delivered in the video. We designed the app to send up to 3 notifications per day as a reminder to complete the quiz, watch the video/s or read the messages.

We developed our application according to Cugelman's gamification tactics,^41,72 and to several related techniques (see Appendix 1). Since gamification should offer a long-term resource to be considered effective,⁷² without reaching a point of saturation where its appeal decraeses, we set the duration of the application's tasks at 10 d.

MorbiQuiz sections and features

The smartphone app, called Morbiquiz, was entirely developed by researchers with expertise in health communication, mHealth and psychology, and it was created with the collaboration of an agency specialized in native app development. The application is in Italian, it runs on the 2 operating systems iOS and Android, and can be downloaded free of charge on the Italian and Swiss Google Play and App Store.

The app consists of 3 main screens: a main screen, a lateral menu, and a leaderboard. In the intervention targeting vaccination literacy, the main screen displays the participant's path, dotted by 10 points, each representing a daily quiz (Fig. 1). The dot lights up when the quiz is completed and allows participants to visualize the questions answered, the correct answers, the score for each answer, and the textual content associated with the answer (Fig. 2). In the second intervention, targeting vaccination empowerment, the participant's path has only 2 dots, standing for the 2 videos. The dots light up when a video was watched and, by clicking on it, the participants can see the video again. The main screen of the group receiving both interventions displays a 10-dot path with the 2 videos integrated in the first and last dots respectively.

Figure 1. Apzp's main screen.

Figure 2. App's screen showing the questions answered, the correct answers, the score for each answer, and the textual content associated with the answer.

The lateral menu (Fig. 3) has the following features: (a) a profile section – where participants can select their gender, nickname and profile picture; (b) a message section, where participants can read all messages received until that point (only available for the 2 experimental groups receiving them); (c) the option to recommend the app by e-mail; (d) the option to evaluate the app in the official store; (e) the option to share the app (e.g. via WhatsApp or other social media); (f) an “about” section; (g) a disclaimer; (h) the list of all institutions working on the project; (i) the option to contact the developers; and (l) a logout option. The leaderboard (Fig. 4) displays all participants' nicknames, profile pictures, and respective scores, with the highest scores on top. Finally, the researchers had the access to a dashboard, which allowed to constantly keep track of participants' usage of the app and download usage-related data in real time.

Figure 3. App's lateral menu.

Figure 4. App's leaderboard.

Procedure

Before starting the experiment, participants received an invitation containing a unique ID number, an online questionnaire (baseline survey), and a consent form. We sent up to 2 reminders to fill out the survey, which closed on November 20. Once the questionnaire was closed, we randomly allocated all eligible participants to one of the 4 conditions.

On November 23, we provided participants in the 3 experimental conditions with further instructions on how to download and access the app through generated accounts with unique username and password. We included information on the (maximum) voucher amount they would receive as a compensation for their participation. We set the amount at 10 euros for the group receiving only the empowerment intervention and the control group, while participants receiving the knowledge intervention or both interventions could obtain up to 50 euros according to their final score (approximately 43 eurocents per point, for a maximum of 117 points). The control group was informed that they would only receive a second questionnaire after 2 weeks.

Application of the 2 interventions lasted 10 d and took place simultaneously for all participants between December 1 and December 10, 2016. Once logged in, participants were asked to select their gender, include a public nickname and upload a profile picture. Once the experiment was finished, all participants received a posttest questionnaire aiming to measure the same primary and secondary variables assessed during the baseline survey. The questionnaire was closed on January 15th, 2017.

Measures

The baseline questionnaire and the posttest used the same questions and exact wording for all primary and secondary measures. In the pretest, we also assessed subjective health of the participant and of his/her child as an ice-breaking question, and we added 4 extra items for those participants who had more than one child to compensate for recall bias.⁷³ These 4 items asked about: (a) vaccination status of older children (vaccinated with 1, 2 or no doses), (b) past experience with MMR side effects (on a 5-point scale anchoring at “mild” and “severe”), (c) if participant uses the same or different criteria to make an MMR vaccination decision for the youngest child compared with the older one(s), and (d) to evaluate the MMR vaccination decision for the youngest child compared with the older ones (on a 9-point scale anchoring at “easier” and “more difficult”). The posttest also included questions regarding: (a) information seeking in the past 30 days; (b) any events that had prevented active participation in the experiment (such as child's or own sickness, travel, lack of Internet or smartphone access for one or more days); (c) any conversation on the MMR vaccination or other vaccinations with other people in the past 30 d (pediatrician or other medical professional, homeopath or other CAM professional, friends or relatives); (d) user's evaluation of the app. To evaluate the app, we adapted 11 items from the Mobile App Rating Scale.^74,75

Primary outcomes

Psychological empowerment was measured with 12 items developed by Diviani and colleagues.⁷⁶ The scale follows Spreitzer's conceptualization of psychological empowerment as a set of 4 sub-dimensions (meaningfulness, impact, competence and self-determination) and includes 4 items per dimensions (e.g.,: “I feel able to make an MMR vaccination decision for my child” to measure perceived competence).⁷⁷ Response was recorded on a 7 point-scale measuring agreement. The final score is the sum of all answers, with a range from 12 to 84. MMR vaccination knowledge was measured with 15 items. Eight items were adapted from the previously validated Vaccination Knowledge Scale⁷⁸ while 7 items were created ad hoc to cover several notions included in the app, such as current vaccination coverage in Italy and typology of vaccination facilities. Response was recorded as either “True,” “False” or “I don't know." Correct answers were scored as 1, while other options obtained no score. The final score is the sum of all correct answers, ranging from 0 to 15. Risk perception of the MMR vaccination side effects and of measles was measured with 4 items, 2 about severity and 2 about susceptibility. Furthermore, we asked participants to compare the risks and benefits of the MMR vaccination against those of its target diseases with a single item adapted from a risk perception scale.⁷⁹

Secondary outcomes

The secondary outcomes were MMR vaccination attitude (What is your opinion on the MMR vaccination?), intention to vaccinate against MMR (How likely is it that you will take your child for his/her next MMR vaccination?), intention to recommend the MMR vaccination to other parents (How likely are you to recommend the MMR vaccination to other parents?) and confidence in the decision (How confident are you with your MMR vaccination decision?). All variables were measured with a single item scale and response was recorded on a 5-point scale. We decided to measure vaccination attitude and confidence with single item scales both because the questionnaire contained an extensive number of questions and because we were specifically interested in participants' opinion toward the vaccination (rather than vaccine hesitancy more broadly which also entails risk perception) and the extent to which they felt certain about the decision they had made about vaccination. Furthermore, since attitude has often been found to be a predictor of intention, we decided to rely on an extra measurement to find confirmation. Furthermore, participants' preferred role in the MMR vaccination decision was measured with the Control Preference Scale (CPS), adapted to the vaccination context by replacing “doctor” with “pediatrician” and asking subjects to indicate their preferred role in their child's MMR vaccination decision-making (ranging from the individual making the decision alone, through the individual making the decision jointly with the pediatrician, to the pediatrician making the decision alone).⁸⁰

Control variables

Socio-demographic information included both parents' age, level of education, nationality and ZIP code. In addition, parents indicated the date of birth of their only child or, if they had more than one child, their youngest one.

Other information

The Ethical Committee of the University of Milan approved the study on April 18, 2016 (decision no. 14/16).

Statistical analysis

Data analysis was performed using the Statistical Package for Social Science (SPSS; Version 21.0). After entering the collected data into the software, missing data and outlier checks were performed, as well as shape of distribution analyses. To ensure that results could be ascribed to the experimental conditions rather than to baseline between-group differences, randomization checks were performed using ANOVAs and contingency coefficients. ANCOVAs were performed for each primary and secondary outcome to determine whether there were differences among the experimental conditions in terms of “post-experiment outcome” after controlling for its “pre-experimental” level. Where appropriate, planned contrasts were conducted to analyze significant differences across the experimental conditions.

Limitations

This study is not without limitations. First of all, the self-selected nature of our sample resulted in a low number of parents contrary to or undecided about their child's MMR vaccination. Second, as significant between-group differences were detected for intention and confidence, it might be that insignificant findings are to be ascribed to limitations related to the operationalization and measurement of the other secondary outcomes. Third, since the study was advertised by academic institutions, it may have mostly attracted the attention of educated parents. Therefore, sampling among less educated participants might generate different findings. Fourth, generalizability of our results is limited by the homogeneity of our samples' characteristics. Finally, there's the possibility of confoundingbecause the groups receiving the knowledge intervention were potentially offered a higher monetary incentive compared with the empowerment only group.

Implications and conclusions

Our work suggests that multi-component mHealth interventions aimed at providing parents with vaccination-related information can be effective in boosting their knowledge and increasing their intention to vaccinate.^81,82 Furthermore, it seems that offering a gamified learning experience can significantly contribute to a knowledge gain in the context of vaccination. Interventions aimed at increasing parents' empowerment, on the other hand, should cautiously consider a possible information overload as a drawback that can ultimately confuse parents, and also be aware that a call for empowerment might be misread as a call against adhering to official recommendations. Future qualitative research could be relevant to help explain the experimental results, as well as explore parents' experience with the app we developed and their suggestions on possible implementations of this tool. Our study provides further evidence for the suitability of the mHealth context for experimental studies as its versatility allows for different experimental treatments.

Disclosure of potential conflicts of interest

No conflicts of interest were disclosed.

Author Contributions

MF, EG, MF and PJS participated in the research and article preparation. All authors participated in the conception and design of the study, the acquisition of the data, its analysis and interpretation. LR and AZ organized the panel validation. All authors joined the article preparation and have approved the final article.

Acknowledgments

We are extremely grateful to all participants for devolving their time to participate in this study. We would also like to thank Elena Crestanello and all the Lombardy-based pediatricians who helped in the recruitment process. Special thanks go to the panel members who validated the contents of the quiz and to all the people who helped test and improve MorbiQuiz before the start of the trial by providing constant and valuable feedback.

Funding

This work was supported funded by the Swiss National Science Foundation under grant 147333.

Notes

¹ Since the first dose of the MMR vaccination is given between 12–15 months we sampled among parents of young children to avoid cognitive dissonance bias.⁷³

² We decided to recruit our participants in Lombardy because this is the most densely populated region in Italy.⁸³

Appendix 1. Cugelman's principles of gamification and techniques related to them

Table

Principle	Technique used to implement the principle
1. Goal setting: Committing to achieve a goal	The goal to become more informed is highlighted by the use of a daily quiz that allows for active learning.
2. Capacity to overcome challenges: Growth, learning, and development	A personalized trajectory simulating growth is given in the main screen, where users can also display their time management (midnight deadline for each quiz).
3. Providing feedback on performance: Receiving constant feedback through the experience	Users are informed whether they gave a correct or wrong answer. A textual content is unblocked after each answer providing more information on the topic of the quiz.
4. Reinforcement: Gaining rewards, avoiding punishments	Users receive points for correct answers, while no points for ungiven or wrong answers. A monetary voucher is offered as a reward according to the final score obtained in the quiz.
5. Compare progress: Monitoring progress with self and others	Leaderboard where users can compare their score with that of other participants
6. Social connectivity: Interacting with other people	N/A for experimental control purposes
7. Fun and playfulness: Paying out an alternative reality	The quiz simulates parents' information-seeking in the real life but provides at the same time a fun experience made of rewards upon successful learning.