Misinformation on vaccination: A quantitative analysis of YouTube videos

ABSTRACT

In Italy, the phenomenon of vaccine hesitancy has increased with time and represents a complex problem that requires a continuous monitoring. Misinformation on media and social media seems to be one of the determinants of the vaccine hesitancy since, for instance, 42.8 percent of Italian citizens used the internet to obtain vaccine information in 2016.

This article reports a quantitative analysis of 560 YouTube videos related to the link between vaccines and autism or other serious side effects on children.

The analysis revealed that most of the videos were negative in tone and that the annual number of uploaded videos has increased during the considered period, that goes from 27 December 2007 to 31 July 2017, with a peak of 224 videos in the first seven months of 2017.

These findings suggest that the public institutions should be more engaged in establishing a web presence in order to provide reliable information, answers, stories, and videos so to respond to questions of the public about vaccination. These actions could be useful to allow citizens to make informed decisions about vaccines so to comply with vaccination regulations.

KEYWORDS:

• Autism spectrum disorder
• quantitative analysis
• social media
• vaccine hesitancy
• YouTube videos

Introduction

Notwithstanding the evidence demonstrating the benefits of immunization is overwhelming, recently vaccine hesitancy has progressively increased and represents nowadays a complex and rapidly changing global problem that requires continuous monitoring.¹ Vaccine hesitancy refers to the delay in acceptance or refusal of vaccines despite the availability of vaccination services and the presence of vaccination rules.

The Strategic Advisory Group of Experts (SAGE) on Immunization noted that there are many determinants of vaccine hesitancy and grouped these determinants in three categories: contextual, individual and group influences, and vaccine/vaccination specific issues. The SAGE Working Group on Vaccine Hesitancy concluded that a poor or inadequate communication can negatively influence vaccine uptake and contribute to vaccine hesitancy.² Media and social media can create either a negative or a positive attitude towards vaccines and can provide a platform for lobbies and key opinion leaders to influence other citizens. In fact, social media allow users to freely voice opinions and experiences and they can facilitate the organization of social networks in favour of or against vaccines.³

In this context, the Wakefield's paper is well-known as the emblematic case of the vaccine hesitancy, and despite, after the publication of that paper, several studies have shown that there is no link between receiving vaccines and developing Autism Spectrum Disorder ASD,⁴ the public remained negatively influenced toward the safety of vaccination. In fact, measles, mumps, and rubella (MMR) vaccination rates began to drop because parents were concerned about the risk of ASD as a consequence of that kind of vaccination.⁵

In Italy too, starting from 2013, a gradual decrease of childhood vaccination coverage was observed. For the first time, the uptake of hexavalent vaccination dropped under the 95% target and MMR vaccine coverage level decreased below 90% at the national level.⁶ A possible link between vaccination coverage drop and disinformation spread on the web has been suggested in a recent Italian study.⁷

The Internet has rapidly become a widely-used source of information, and its use to obtain vaccine information in Italy has increased up to 42.8%.⁸ YouTube Italian users increased from 38.7% in 2013 to 46.8% in 2016 (up to 73.9% among the young).⁹ Several studies have been conducted in order to analyse YouTube videos in relation to specific health issues^10,11,12 and various studies have examined vaccine-related YouTube videos.^13,14,15,16

The aim of this study was to carry out a quantitative analysis of the Italian videos available on YouTube about the link between vaccines and autism or other serious side effects in children. In particular, we address the following research questions:

•	RQ 1: What are the temporal distribution and the tone of autism-vaccines videos on YouTube?
•	RQ 2: What are the general characteristics of YouTube videos?
•	RQ 3: What are the sources and the categories of the videos and is there any relation between such categories and the tone of the videos?
•	RQ 4: Is there any relationship between the tone of the video and viewers' attitudes (number of views, number of likes, and number of dislikes)?
•	RQ 5: Is there any relationship between the tone of the videos and the number of views for the videos above and below the median value of the number of views?

Results

From Fig. 1, we can see the rapid increase, starting from 2014, in the number of videos with a negative tone, from 27 (6.9%) in 2014 to 147 (37.7%) in the first seven months of 2017, together with a delayed increase of the videos with a positive tone, from 13 (10.2%) in 2014 to 54 (42.2%) in the first seven months of 2017.

In Table 2 the row headings denote the descriptive parameters of the YouTube videos of which the columns provide the mean values, the standard deviations and the values of the significance parameter for the three possible pairwise comparisons.

Table 2. Relationships between the tone of the videos and their general characteristics (one-way ANOVA).

	negative (-) vs positive (+)			negative (-) vs neutral (0)			positive (+) vs neutral (0)
	Mean, SD			Mean, SD			Mean, SD
	−	+	P value	−	0	P value	+	0	P value
n° views	8647	2420	≤ 0.001	8647	5323	≤ 0.05	2420	5323	ns
	32553	8616		32553	13771		8616	13771
video length (minutes)	18.34	8.916	≤ 0.001	18.34	32.33	ns	8.916	32.33	≤ 0.05
	27.78	15.61		27.78	56.29		15.61	56.29
total vision (hours)	623.7	237.5	≤ 0.001	623.7	607.7	ns	237.5	607.7	ns
	2178	828.1		2178	1465		828.1	1465
average vision (minutes)	3.883	2.087	≤ 0.001	3.883	4.405	ns	2.087	4.405	≤ 0.01
	3.027	1.897		3.027	4.910		1.897	4.910
n° shares	214.3	26.38	≤ 0.001	214.3	170.8	≤ 0.001	26.38	170.8	ns
	767.0	57.16		767.0	540.9		57.16	540.9
n° likes	92.19	81.92	≤ 0.001	92.19	61.62	≤ 0.001	81.92	61.62	ns
	290.9	507.1		290.9	203.1		507.1	203.1
n° dislikes	11.44	11.46	ns	11.44	5.238	ns	11.46	5.238	ns
	65.57	26.96		65.57	13.81		26.96	13.81
n° comments	27.27	35.63	ns	27.27	35.61	ns	35.63	35.61	ns
	120.4	171.7		120.4	104.0		171.7	104.0

Note: P values greater than 0.05 are indicated with ns.

As shown in Table 2, there is a significant relationship between the tone and the number of views, the video length, the number of shares and likes. No significant results were found for what concerns the number of dislikes and comments.

All the values contained in Table 2 have been derived directly from the whole set of the YouTube videos so, for instance, average vision does not derive from the total vision divided by the number of the views. Moreover, we could not consider the temporal distribution of the number of visualizations and of the number of shares and likes/dislikes since these data are missing for most of the analysed videos.

From Fig. 3 below, it can easily be deduced how:

Figure 1. Temporal distribution according to the tone of the YouTube videos, absolute values. Note: the number of videos uploaded in 2017 relates only to the months of January, February, March, April, May, June and July).

Figure 2. Video categories associated with the tone of the videos. Note: “People and blogs” or P&B, “No-profit and activism” or NP&A, “News and politics” or N&P, “Science and technology” or S&T, “Education” or Edu, “Entertainment” or Ent. In the analysis, the categories containing few videos (“Films and cartons”, “Music”, “Practical guides and style”, “Humour”, “Sport”, “Animals”) were combined in a category called “Other”.

•	RQ 1: What are the temporal distribution and the tone of autism-vaccines videos on YouTube? We identified and analysed 560 videos of which 392 with a negative tone, 126 with a positive tone and 42 with a neutral tone (see Fig. 1). There was an 11-year span between the first and last video uploads (i.e. from 27/12/2007 to 31/07/2017). As shown in Fig. 1, the highest number of videos was uploaded in 2017 (224 videos that corresponds to 40.0% of the total).
•	RQ 2: What are the general characteristics of YouTube videos? Table 1 presents the descriptive statistics of our sample of 560 YouTube videos. In each row, we present one of the meaningful descriptive parameters whereas the columns provide the associated values according to each column's heading. Such values allow the appreciation of the distribution of each parameter. Table 1. Descriptive statistics: general characteristics of 560 YouTube videos. total range: minimum and maximum 25th, 50th, 75th percentiles % of data availability ^(1) negative positive neutral negative positive neutral negative positive neutral n° views 100.00% 3389436 304978 223570 13 9 13 282.5 67 158.25 417224 76791 75995 1056 267 324 4714 788 1506.25 video length (minutes) 100.00% 7190 1123 1357 0.09 0.15 0.46 3.56 1.39 1.49 212.34 94.44 262 9.21 3.15 6.18 19.70 10.20 36.88 total vision (hours) 78.21% 194590 22322 19446 0.33 0.08 0.12 16 1.70 2.75 26280 6744 6432 72 17 20.5 384 20 132 average vision (minutes) 76.96% 3.88 ^(2) 2.09 ^(2) 4.41 ^(2) 0.16 0.15 0.34 1.49 0.57 1.14 19.52 8.27 22.13 3.2 1.28 2.34 5.26 3.11 6.42 n° shares 78.04% 67074 2427 5464 0 0 0 8 1 2 7910 269 2641 21 4.5 3 99 15.25 14.75 n° likes 98.93% 35861 10076 2588 0 0 0 4 0 0 2960 5275 1158 13 2 2.5 52 7.5 21.5 n° dislikes 98.93% 4450 1410 220 0 0 0 0 0 0 997 182 76 1 1 0 4 8.5 2 n° comments 98.04% 10526 4347 1460 0 0 0 0 0 0 1610 1756 598 1 0.5 0 6 6 6 Notes: (1) the % of data availability specifies the percentage of the videos for which a given datum is available. The number of views and the length of the videos are available for all the videos (so we have a value of 100%) whereas the other data are available for the specified lower percentages of the videos; (2)these values are the average vision values for each tone of the videos. The total amount of views for all videos summed up to 3917984. 461 videos (82.3%) were in the Italian language while the remaining 99 videos (17.7%) were in the English language with Italian subtitles.
•	RQ 3: What are the sources and the categories of the videos and is there any relation between categories and the tone of the videos? Some of the sources have published more than one video, in fact, the total sources were 254 while the uploaded videos were 560. Two sources have been very active in spreading videos with a negative attitude towards vaccines. Radio Autismo (Radio Autism) and Autismo Vaccini (Autism Vaccines) uploaded 74 (13.21%) and 36 (6.43%) videos on YouTube, respectively. The videos have been freely and independently classified by the sources or by those who uploaded the videos. This classification has been done according to the eleven categories provided by YouTube or “People and blogs”, “No-profit and activism, “News and politics”, “Science and technology”, “Education”, “Entertainment”, “Films and cartons”, “Music”, “Practical guides and style”, “Humour”, “Sport”, and “Animals”. From Fig. 2, we can see how the videos with a neutral tone represent a minority for all the categories whereas those with a positive tone are highly represented only for the category “News and politics” (N&P) whereas those with a negative tone represent the majority for all the categories.
•	RQ 4: Is there any relationship between the tone of the video and viewer responses including the number of views, number of likes, and number of dislikes?

	1.	Most of videos with a negative tone are associated with a positive value of the variable balance (defined as the difference between the number of the likes and the number of the dislikes in relation (1(1) $balance=numberoflikes-numberofdislikes$(1) ) of the section Materials and methods), when the number of the likes is higher than the number of dislikes;
	2.	The videos with a positive tone represent the majority when the variable balance assumes a negative value (or when the number of the likes is lower than the number of dislikes);
	3.	In the case balance = 0 (or when the number of the likes and the dislikes are equal) the percentages of the videos with either positive or negative tone are almost coincident;
	4.	The videos with a neutral tone represent a minority in all the cases but where the variable balance assumes a null value they attain their highest percentage value.

•

RQ 5: Is there any relationship between the tone of the videos and the number of views for the videos above and below the median value?

Figure 3. Relationships between the tone of the videos and the sign of the variable balance, percentage values.

In order to verify the possible existence of a difference between the ratio of videos with a negative, positive and neutral tone that are below and above the median value of the number of views we evaluated such value and found that it was equal to 671. By summing up the number of the visualizations of the videos above the median, we found that they reached a number equal to 98.34 percent of the total.

Considering videos below and above the median value respect to positive and negative tone, a chi-square test detected a statistically significant difference (p < 0.0001), with a number of negative videos higher above the median value. Otherwise, considering videos with neutral and negative tone, we found a statistically significant difference (p = 0.0054), with a number of neutral videos higher below the median value.

Discussion

Despite vaccines are among the most effective public health interventions, starting from 2013 in Italy the vaccine coverage decreased for all vaccines.⁶ In a recent study, the decision of the Court of Justice of Rimini in March 2012 was identified as the probable trigger event that led to a spread of vaccine hesitancy in the country.⁷ This sentence granted the vaccine injury compensation based on the findings of Wakefield's study even if subsequently was cancelled by the Court of Appeals in Bologna in 2015. The study showed how the analysis of web search trends and social network data represents a proxy for vaccine hesitancy at the population level.

Parents of children who refuse routine vaccinations usually obtain information about vaccination hazards from the Internet. Flash Eurobarometer report No. 404 “European citizens' digital health literacy”¹⁷ showed that 13% of the Italian citizens look for information on vaccinations on the Internet and 89% are satisfied with the general health-related information they found on the Internet.

Sometimes scaremongering information regarding vaccinations are spread on the Internet also by health professionals.¹⁸ Their messages have a wide resonance and are one of the possible causes of the increase in mistrust about vaccines. The health professionals should spread their opinions with extreme care since they have a greater responsibility compared to other professionals owing to the fact that their role should be to protect the life and health of children and population in general.¹⁹

YouTube was identified as the second largest social network amongst younger internet users.²⁰ Despite this, in Italy, very few studies have analysed how the issue of vaccinations is dealt with on YouTube.²¹ To our best knowledge, this is the first study in Italy that analysed the content of YouTube videos for what concerns the relations between vaccines and autism.

Our data confirm the prevalence on YouTube of the points of view that support the presence of a relation between vaccines and autism.²² The aim of our study was to verify the presence of video with accurate information on the lack of correlation vaccines-autism. This relationship is a long-standing argument put forward by anti-vaccination movements and it is possible that our findings could be biased by the use of keyword "autism. However, our findings show the presence, during the years of observation, of a positive trend of increase for the videos with positive tone.. From the beginning of our research, we found messages with a negative tone and no videos with positive tone up to 2013, with the exception of one video uploaded in 2011. Starting from 2014, we observed an increasing number of positive videos, with a delay of two years respect to the increase of the anti-vaccination attitudes on social networks and Internet searches, as reported in a recent study on autism and vaccination on the web.⁷

An important finding of the study was that the YouTube videos with negative tone were more viewed, shared and with more likes than those with positive or neutral tone, confirming the results obtained by Covolo et al., 2017.²¹ This can be explained by the phenomenon known as confirmation bias, that leads to favour information that confirms our beliefs and rejects facts that contradict them.²³ These behaviours create what is called in the new media an “echo chamber”²⁴ and this feature is confirmed by our findings.

Overall, anti-vaccination videos are over three times more numerous than the pro-vaccination videos, thus the probability of any user to find anti-vaccination videos is higher. Therefore, vaccine-hesitant users have a greater chance of being negatively affected, at least about their opinions on vaccination. Furthermore, when the users visualize a video, YouTube recommended other videos similar in contents on the column on the right side of the screen thus amplifying the effect. Although the algorithm of YouTube is secret, it is known that some video characteristics, such as the number of views and the duration of the visualization, contribute to rank the videos in the search results. This YouTube feature enhance the probability to put in connection the vaccine-hesitant individuals with videos with a negative tone and thus increase the mistrust surrounding vaccination recommendations in Italy.

In conclusion, vaccine hesitancy is closely connected with the increasing importance of the Internet and the new information and communication technologies (ICTs); in fact, they play an important role in parents' decisions on whether or not to vaccinate their children.²⁵

An accurate monitoring of the spread of the misinformation about vaccinations in the social media might be useful in order to explore the main public concerns about the vaccinations and to better clarify the role of the social media in forming and influencing people's attitudes and behaviour towards vaccinations. Furthermore, the social media monitoring could help to measure the impact of vaccination campaigns and programmes.

This information could be used by academic and governmental organizations that should be engaged with the new media in an attempt to fight false beliefs about vaccinations.

This study has some limitations which have to be pointed out. We were not able to account for the temporal distribution of the numbers of the likes and the dislikes of the chosen YouTube videos. This because YouTube does not always provide these data that could be used to reveal the occurrence of peaks of interest associated with particular events such as courts judgements. Moreover, in our study, we did not deal with in which way the link between autism and vaccines is treated by positive, neutral and negative videos. This surely interesting aspect will be dealt with in the forthcoming continuation of our study.

Materials and methods

Samples

YouTube (www.youtube.com) was searched on August 1, 2017, with the aim of finding video clips related to autism and vaccination, in particular to the lack of correlation between vaccines-autism. The following search terms were used: i) “autismo and vaccino” (autism and vaccine), ii) “autismo and vaccini” (autism and vaccines), iii) “autismo and vaccinazione” (autism and vaccination) and iv) “autismo and vaccinazioni” (autism and vaccinations). The search results, sorted according to pertinence, were captured²⁶ and processed with NVivo software (version 11).²⁷ In total, 2014 videos were initially collected, with a final total of 603 videos after the removal of the duplicates. In the final sample, videos that have addressed the relationship between vaccines and other serious side effects were also included, because the search terms were present in the title, in the source, in the description or in the comments. 43 videos were not included in the analysis because they were presented in a language other than Italian (without Italian subtitles) or were not pertinent. The remaining 560 videos constituted the final working sample for the current study.

Coding scheme

The quantitative content analysis consisted of coding a series of typical YouTube predefined video characteristics, including title, date of posting, video length, total and average duration of vision, number of shares, number of views, number of likes, number of dislikes, category, number of comments and source. In addition to these characteristics, a qualitative content analysis was carried out and the tone of each video was categorized into positive (i.e., there is no link between vaccines and autism or other serious health effects), negative (i.e., the vaccines can cause autism or other serious health effects) and neutral (i.e., contains both positive and negative messages, as it occurs during debates). Furthermore, the comments on the videos were categorized as: i) in accordance with the video, ii) in disagreement with the video and iii) irrelevant (e.g., comments inconsistent with the topic).

Coding procedure

In this study, the unit of analysis represents a single video on YouTube. An Excel worksheet was created in order to register the information of all videos. Two researchers (F. A. and G. P.), each of whom blinded to the research questions, coded all of the sample videos. They were trained for two hours before the main coding. Information on how to gather the data from videos on YouTube was provided during the training session. Intercoder reliability was calculated and Cohen's kappa coefficient was 0.854, above the rule-of-thumb value of 0.70.²⁸ The disagreements were resolved by further discussions between the coders.

Data analysis

A descriptive analysis was performed on the data collected from YouTube. Some One-Way ANOVA (ANalysis Of VAriance) tests were conducted to address the relationship between the tone of videos and the total and average duration of vision, video length, number of shares, views, likes, dislikes and comments.

To evaluate the level of video approval in RQ4, we considered the number of both likes and dislikes votes for each video. The difference between like and dislike was considered as a new variable named balance, that can assume a negative, a null or a positive value:(1) $balance=numberoflikes-numberofdislikes$(1)

A Chi-square analysis was conducted with the aim of addressing the relationships between tone and number of views for videos above and below the median value of the number of views. P-values < 0.05 were considered statistically significant. GraphPad Prism 7 was used to perform all statistical analyses.²⁹

Disclosure of potential conflicts of interest

No potential conflicts of interest were disclosed.