Vaccine misinformation on social media – topic-based content and sentiment analysis of Polish vaccine-deniers’ comments on Facebook

ABSTRACT

Introduction: Vaccinations are referred to as one of the greatest achievements of modern medicine. However, their effectiveness is also constantly denied by certain groups in society. This results in an ongoing dispute that has been gradually moving online in the last few years due to the development of technology. Our study aimed to utilize social media to identify and analyze vaccine-deniers’ arguments against child vaccinations.

Method: All public comments posted to a leading Polish vaccination opponents’ Facebook page posted between 01/05/2019 and 31/07/2019 were collected and analyzed quantitatively in terms of their content according to the modified method developed by Kata (Kata, 2010). Sentiment analysis was also performed.

Results: Out of 18,685 comments analyzed, 4,042 contained content covered by the adopted criteria: conspiracy theories (28.2%), misinformation and unreliable premises (19.9%), content related to the safety and effectiveness of vaccinations (14.0%), noncompliance with civil rights (13.2%), own experience (10.9%), morality, religion, and belief (8.5%), and alternative medicine (5.4%). There were also 1,223 pro-vaccine comments, of which 15.2% were offensive, mocking, or non-substantive. Sentiment analysis showed that comments without any arguments as well as those containing statements about alternative medicine or misinformation were more positive and less angry than comments in other topic categories.

Conclusions: The large amount of content in the conspiracy theory and misinformation categories may indicate that authors of such comments may be characterized by a lack of trust in the scientific achievements of medicine. These findings should be adequately addressed in vaccination campaigns.

KEYWORDS:

• Anti-vaccination movement
• vaccination refusal
• social media
• sentiment analysis
• misinformation

1. Introduction

Vaccinations are often described as one of the greatest public health achievements in modern medical history¹. This is due to vaccinations’ huge impact on the reduction in incidence of infectious diseases²⁠⁠. Vaccines have not only improved people’s quality of life, but have also significantly reduced the financial burden of infectious disease³⁠. Different countries have various health policies concerning vaccinations. After the Second World War, until 1960, vaccinations in Poland were carried out as ad hoc campaigns responding to particular events or perceived threats.’ In the 1960s, (Preventive Vaccination Schedules) were introduced in Poland, divided into compulsory and recommended vaccinations.⁴ Currently, eleven vaccinations are mandatory, and provided without charge in Poland.^5,6 However, the anti-vaccination movement has been becoming more and more prevalent both worldwide and in Poland.^7, ⁠During the 71st World Health Assembly, vaccine hesitancy was identified as one of the top 10 global health threats⁸.

Vaccines have, since their first introduction, been accompanied by skepticism, as well as many critical attitudes and actions, such as refusals to vaccinate or active campaigning against vaccinations, that could reverse the public health achievements of vaccination⁹⁠. Nowadays, anti-vaccine activity is thriving offline and online, particularly on social media^10–12 ⁠. Ogólnopolskie Stowarzyszenie Wiedzy o Szczepieniach STOP NOP (The National Association for Knowledge on Vaccines “STOP NOP”), which currently has over 150,000 followers on Facebook^13,14⁠, is the main opinion-leader disseminating anti-vaccine information and providing a platform for vaccine-refusal opinions in social media. The fan page is managed by an eponymous foundation, whose statuses set out goals including promotion and protection of public health based on a holistic concept of man and the world, or prevention of adverse vaccine reactions (NOP). It works promotes optional vaccination and the development of documents facilitating parents’ refusal to vaccinate their children.¹⁵ The Association has also been responsible for public actions such as the demonstration in Auschwitz prisoner uniforms, comparing mandatory vaccinations to Nazism.¹⁶

Nowadays, for an increasing number of people, the Internet has become their only source for information related to health protection and vaccinations¹⁷ . This is a cause for concern, because on social media every statement has the same value, whether it comes from an expert or a person not associated with medical science¹⁸⁠. Therefore, the process of decision-making in regards to childhood vaccination may be increasingly influenced by low-quality information^19,20⁠. Specific features of online communities (that lead to the creation of thought bubbles/silos) also contribute to the problem: people gathered around the ideas of the anti-vaccine movement exist in an “echo chamber,” which prevents substantive criticism from reaching them²¹⁠.

There is an escalating concern about anti-vaccine sentiment in many countries,¹² which has led to vigorous developments in vaccine hesitancy as an area of inquiry. Studies have been carried out using a number of methodologies and with various goals including to study the activity and connections of members in anti-vaccine groups,²² the most prominent Google search results,¹⁰ comments posted on various social media sites²³ and many others.

Sentiment analysis is one of the more promising techniques in the study of online communications as it provides information on what emotions are dominant in vaccination-related statements, and allows us to determine which approach to providing information about vaccines and reassuring hesitant parents will be the most cost-effective.²⁴ This pragmatic approach to vaccine-related communications, which is rooted in commercial application and treats them similarly to consumer products, gives public health workers effective tools for dealing with vaccination hesitancy, and has been used in other studies on the same subject¹¹ .

The value of such studies lies in the information on which content/themes/ideas generate the most hesitancy and negative emotions, and also which might be most amenable to change, allowing for more accurate and thus more effective targeting of online information campaigns. These studies may also equip health-service professionals to interact better with hesitant people who could potentially be persuaded to be vaccinated. Studies show that people do not only have doubts about the effectiveness or safety of vaccinations, but they may also believe in various conspiracy theories or may be convinced that alternative therapies can provide them with better protection against diseases than vaccinations^21,23⁠. Preparing for such arguments and addressing them, even if they are only hinted at, may increase the effectiveness of reassurance and persuasion.

This paper will focus on the most common outlooks in order to map out the extent and nature of the vaccine-related misinformation. The main objective of this study was to understand the causes for vaccine hesitancy and anti-vaccination attitudes among users active on social media. We investigated the arguments and emotions present in comments posted on the most popular Polish anti-vaccine Facebook page.

2. Method

2.1. Data collection

Data collection was conducted between August 1, 2019, and September 1, 2019. Posts and comments were collected from the most popular Facebook page advocating for the refusal of the mandatory vaccinations in Poland (“Ogólnopolskie Stowarzyszenie Wiedzy o Szczepieniach STOP NOP”). All identified comments/responses published on the page between May 1 and July 31 were archived. Given the typical patterns of commenting, with the great majority of comments posted in the first few days after the original post, it was assumed that data collection during this period would not significantly change the results due to comments added later²⁵⁠. The data were assembled in three separate spreadsheets (one for each month). Along with each post, the date of posting, content, and the number of comments was recorded. The content of comments and the number of reactions (without distinguishing between “like,” “love,” “happy” or “sad”) were recorded, as was the gender of the comment’s author based on the users’ first or ‘given’ name. User gender was recorded as male, female, or unknown/undetermined. No other data on commenting users were recorded. Each comment was categorized into one argument category (Table 1). The comments were classified based on Kata¹⁰⁠, who divided anti-vaccination content into categories. Kata’s research concerned website content, so we adapted his classification to analyze comments, with some categories combined to simplify interpretation. Comments that did not match any pre-set category were assigned the code ‘‘0”. The categorization was carried out independently by two researchers (K.B.; K.K.). Comments posted by the page administration were also recorded, but were not taken into account during the content analysis. The database creation process is shown in Figure 1.

Table 1. Frequency of content criteria categories among all analyzed comments (n = 4042)

Comments analysis criteria	n	%
Content categories
Pro-vaccination statement or deliberating with anti-vaccination content	1,223	-
Conspiracy theories/search for truth	(795)	(28.2)
Unusual theories: Unique theories about purposes of vaccination (e.g. sterilization)	178	22.4
Profit: Vaccination policies motivated by profit	166	20.9
Rebel doctors: “Enlightened” doctors break away from the medical establishment	104	13.1
Foolish doctors: Doctors are ignorant, fearful of sanctions	86	10.8
Cover-ups: Vaccine information withheld from the public	73	9.2
Protection: Government protects doctors/manufacturers from liability	45	5.7
Informed choices: Encouragement to make educated decisions for oneself/one’s children	36	4.5
Collusion: Vaccine promoters benefit from illnesses caused by vaccines	33	4.2
Privileged knowledge: Presenting information the medical world is unaware of/rejects	33	4.2
Fear-mongering: Dangers of diseases exaggerated to frighten parents	21	2.6
Anti-science: Biomedicine is wrong; other ways of “knowing” (i.e. intuition, instinct)	20	2.5
Misinformation and falsehoods	(560)	(19.9)
Self-referencing: Links/references to anti-vaccination “experts”	240	42.9
No references: No statistics/citations provided to support claims	137	24.5
Falsehoods: Unsupported statements made	128	22.9
Misrepresentations: Sources not used truthfully, false conclusions drawn	55	9.9
Safety and effectiveness	(395)	(14.0)
Underreporting: Vaccine reactions are underreported	91	23.0
Poisons: Vaccines contain poisons/toxins/contaminants	82	20.8
Idiopathic illnesses: Vaccines cause illnesses of unknown origin (e.g. autism, SIDS)	78	19.7
Immunity: Vaccines erode immunity, create only temporary/ineffective immunity	57	14.4
Trivial diseases: Vaccine-preventable diseases are uncommon/not contagious/relatively mild	39	9.9
Simultaneous vaccinations: Multiple vaccines at once increase adverse events	25	6.3
Disease decreases: Disease incidences declined without vaccines (i.e. from improved hygiene)	23	5.8
Civil liberties	(371)	(13.2)
Parental rights: Civil liberties violated by taking away parental choice	148	39.9
Totalitarianism: Vaccine mandates are excessive government control	119	32.1
Monitoring: Vaccine programs harass parents who do not vaccinate	104	28.0
Personal testimonies: Stories about harmed children/personal experiences	(306)	(10.9)
Morality, religion, and ideology	(241)	(8.5)
Immoral acts: Vaccination involves immoral acts (e.g. child experimentation)	206	85.5
Anti-utilitarianism: Universal vaccination sacrifices a few to benefit many	33	13.7
Religious tenets: Vaccination is against God’s will	2	0.8
Alternative medicine	(151)	(5.4)
Alternative treatments: Promoting treatments superior to vaccination (e.g. homeopathy)	71	47.0
“Back to nature”: Promoting “natural” approaches (e.g. children should get diseases naturally)	50	33.1
Products for sale: Promoting alternative products (e.g. vitamins, essential oils)	13	8,6
Critiquing biomedicine: Established medical knowledge is wrong (e.g. germ theory is untrue)	13	8.6
Implied debate: Suggesting debates over if vaccination is effective/necessary	4	2.6

Figure 1. Procedures in building the database of gathered comments

2.2. Sentiments analysis

All the comments collected in the study were analyzed with Sentimenti software tools for text sentiment analysis. This software was recently developed by W3A.PL Ltd. with the help of European Regional Development Fund. The Sentimenti emotion measurement tools comprise two complementary models: the Plutchik’s model of eight basic emotions²⁶⁠ and a simple sentiment model in which each text is evaluated on two scales: positive-negative polarity and general arousal. Each text is automatically evaluated on all the 11 variables (8 emotions, arousal, positive, and negative sentiment). Our analysis considered two emotions only: happiness and anger since only those were significant for the study.

The Sentimenti database holds responses from 20,000 unique responders who evaluated emotions associated with different meanings of Polish lexemes and 7000 unique responders who evaluated short phrases and texts written in Polish (for example, opinions about hotels or medical services)^27,28⁠. Each participant annotated each word or utterance on 10 scales, describing the 8 basic emotions (from 0 – lack of the emotion, to 4), as well as a positive-negative scale and an arousal scale.

This Sentimenti database was used to create a tool for automatic text annotation. The solution is based on the BiLSTM neural network architecture and described in more details in Kocoń article²⁷⁠. As reported (ibidem), the accuracy of automatic annotation, with each emotion annotated as present or absent in the text, was 70%. Sentimeni analysis uses numerical values to determine the ‘‘emotional loading” of each piece of text. In the measurement model, the 11 variables (eight emotion loadings, two sentiment values, and one arousal value) are expressed as a percentage of the emotion saturation or intensity. This means that each emotion can be expressed in a text with a maximum of 100% saturation, be absent (0%) or expressed in the text with intensity between 0 and 100%. According to the Sentimenti developers, the accuracy of the measurement on the training data set is high: the mean error does not exceed 8% per text and 10% per emotion.

The happiness and anger scores expressed in the comments collected for this study were automatically calculated by the Sentimenti tool. The 0–100% emotion intensity annotations are thus a reasonably accurate simulation of text interpretation by an average Polish native speaker.

2.3. Statistical analysis

Descriptive statistics were used to analyze arguments in posts and comments. The statistical tools included with the Sentimenti algorithm were used to test statistical significance of gender differences in terms of the arguments and sentiment used in the comments (using the chi-square test). The same statistical tool was used to calculate scores to determine the differences in emotion intensity for individual argument categories.

3. Results

3.1. General data structure

In the study period, 18,685 comments were collected, 4,042 of which were manually determined and classified to the categories described in Table 1. Other comments, mostly consisting of brief expressions of emotional reactions including single emoticons/emojis and some off-topic material, were not included in the analysis of the anti-vaccine content. Comments made by women amounted to 13,569 (72.6%) and by men, 4,650 (24.9%). Of the remainder, 342 (1.8%) comments were posted by the studied Facebook page, and 124 (0.7%) were made by users of undetermined gender.

3.2. Content analysis of Facebook post and comments

3.2.1. Conspiracy theories/search for truth (28.2%)

The “Conspiracy theories/search for truth” theme was the most prevalent comment category. The majority of comments in this category were assigned to the subset of “Unique theories about purposes of vaccination” such as depopulation/genocide (54.5% of the subcategory), but also preparing the population to be organ donors (6.7% of the subcategory) or claiming that national administration is controlled by a global government (6.7% of the subcategory).

The second most frequent subcategory in this category included comments claiming that vaccination programs were motivated solely by profit (20.9%), even whilst being ineffective or harmful. Politicians, pharmaceutical companies, and health-care professionals were said to be the beneficiaries of such businesses. Other commenters stated that to maximize profits, some individuals responsible for distributing vaccinations have caused patients to develop additional diseases which would require extra treatment in the future (4.2%). There were also statements claiming that the aware of the dangers of vaccination, but is still protecting pharmaceutical companies and doctors who deal with vaccine distribution (5.7%).

A proportion of comments classified in this category stated that there have been doctors and other health professionals who have endorsed the various negative claims about vaccinations and the consequences of their application (13.1%). Contrary to these beliefs, there were also assertions that doctors could not see the truth because of their ignorance, lack of attention, fear, or lack of knowledge from sources other than official (10.8%), as well as mentions of parents who were intimidated by specialists and ‘‘bullied’’ into vaccinating their children (2.6%). Numerous comments claimed that the governments and pharmaceutical companies are aware of the many adverse effects of vaccinations but hide them from the public (9.2%). In contrast, there was also a belief expressed that the medical world is unaware of that information (4.2%). Other commenters claimed that biomedicine is (generally) “wrong,” with the source of this belief being given as different “ways of knowing” such as intuition or instinct (2.5%). Moreover, some commenters encouraged the undecided and pro-vaccine users to make “educated” decisions for their children (4.5%), encouraging the use of sources of information presenting misinformation or conspiracy theories.

3.2.2. Misinformation and falsehoods (19.9%)

Misinformation and falsehoods were the second most common theme. The most frequent comments in this category were “Links/references to anti-vaccination” experts” (42.9%) including links to YouTube videos in 44.2% of the subcategory. “Misrepresentation” appeared in 9.9% of the statements analyzed in this category. Claims were made that the pharmaceutical companies clearly stated in the SPC that vaccinations caused autism or SIDS. All over-interpretations of scientific articles were also included in this category.

Comments classified as “Falsehood” (22.9%) included claims of vaccinated people spreading diseases that they have gained immunity to, the lack of an effective influenza vaccine, stating that any viral vaccine destroys the intestines, etc. The “No references” subcategory covered the 24.5% of the comments in this category that did not contain references to any statistics, scientific articles, or reliable research.

3.2.3. Safety and effectiveness (14.0%)

The themes of the comments concerned the alleged contamination of vaccinations, their potential impact on health, the meaningfulness of the vaccination program and adverse vaccination reactions. The most common sub-category related to this category was: “Vaccine reactions are underreported” (23.0%). Examples of such comments included stating that most parents are not aware of vaccine reactions and do not report them, that reports sent by parents are not included in the statistics, or that doctors do not need to report unwanted reactions for various reasons.

A large proportion of comments also concerned potential adverse reactions to vaccination (19.7%) directly. The most commonly mentioned specific adverse reactions included autism or autism spectrum disorders (24.4% of the subcategory), epilepsy (15.4% of the subcategory), and cancer (11.5% of the subcategory). Other complaints attributed to vaccines included: autoimmune and genetic diseases, allergies, infertility, diabetes, Sudden Infant Death Syndrome, and many more. Questioning whether vaccines actually conferred immunity was also fairly common (14.4%), while 6.3% of comments in this subcategory focused on increased adverse events with multiple vaccines administered at once.

Numerous statements claimed that vaccines were either contaminated or toxic (20.3%), with emphasis on vaccines containing aluminum (19.5% of subcategory), mercury (8.5% of subcategory), as well as naglaze, neurotoxins, glyphosate, thiomersal, and many other substances.

Some comments in this category pointed out decreases in infectious disease occurring before mass immunizations (5.8%); credit for those was given to improvements in sanitation, nutrition, and decreasing poverty. Others suggested that some diseases should not be a target for immunization because of their mild course or low risk of complications; such as measles (46.2% of the subcategory), chickenpox (15.4% of the subcategory), mumps (12.8% of the subcategory), rubella (12.8% of the subcategory), whooping cough (10.3% of the subcategory) and others.

3.2.4. Civil liberties (13.2%)

The most common notion expressed in the “Civil liberties” category was the idea that the parents should always have the final say on medical procedures their children are subjected to (39.9%). Quoting examples of countries where vaccination programs are non-mandatory occurred in this context. “Monitoring” (28.0%) included content in which parents of unvaccinated children expressed how they opposed repression/persecution, the most frequent being financial penalties (39.4% of the Monitoring-related comments), refusal of admission to nursery/kindergarten (30.8% of the Monitoring-related comments) or withdrawal of parental rights (9.6% of the Monitoring-related comments).

Mandatory vaccination was often associated with totalitarianism (32.1%). Mentions of human rights violations, the constitution, or the government being described as “totalitarian” were found repeatedly in the comments assigned to this category.

3.2.5. Stories about harmed children/personal experiences (10.9%)

Stories about harmed children and personal experiences appeared in 10.9% of the analyzed comments. Content classified in this theme included reports of problems that had occurred as a result of vaccinations in the commenter’s personal environment. The most commonly mentioned symptoms were related to the nervous system (27.8% of the category), including autism, epilepsy, convulsions, increased or decreased muscle tone and many others. Autism appeared in 6.2% of comments in this category.

There was a significant expression of concern regarding subjectively perceived behavioral changes (20.9% of the category), which manifested, inter alia, as mental retardation, increased, and decreased tearfulness or apathy. Other reports related to respiratory symptoms (12.4% of category), skin (12.1% of category) and digestive symptoms (8.8% of the category).

3.2.6. Morality, religion, and ideology (8.5%)

The morality, religion, and ideology category were the least frequently occurring content theme. Only two comments were identified based on the religious approach. Content to do with anti-utilitarianism was also relatively rare (13.7% of this category). “Immoral acts” were the most common theme in this category (85.5%), for example linking vaccines with morally dubious actions, such as using cells of aborted fetuses in their production, experimenting on children, or specific events related to crime in a medical context.

3.2.7. Alternative medicine (5.4%)

This section recorded all of the anti-vaccination comments that promoted alternative medicine. The majority of these statements (47.0%) recommended treatments such as “detoxification from heavy metals and vaccines” (36.6% of subcategory), homeopathy (32.4% of subcategory) but also herbalism, the use of high doses of vitamins, energy healing, and others. The latter included promoting alternative products (8.6%) such as a set of diets for the treatment of autism or testing for the amount of heavy metals in the body.

Other comments focused on a “back to nature” (33.1%) approach, with natural methods of disease prevention including long breastfeeding, avoiding the use of medications, and promoting the acquisition of immunity by children through illness.

Criticisms of biomedicine were also present in 8.6% of the comments in this category and suggested alternative methods for strengthening immunity. Some of the comments involved debating about alternative solutions to vaccination (2.6%).

3.2.8. Pro-vaccination content

Comments containing pro-vaccine content were also recorded, either antagonistic to vaccine deniers or positively encouraging vaccination (n = 1,223). Mocking/non-substantive/offensive content made up 15.3% of this comment category.

3.3. Gender analysis of the content

As mentioned above, there were gender differences in frequency of arguments by theme category. Table 2 presents gender distribution for each argument category.

Table 2. Quantitative distribution of comments by theme and gender. The difference between male and female authors in the frequency of comment themes was statistically significant (Chi = 215.633, p < .001). The “other” category, including page admins and the ‘‘undetermined gender’’ was not taken into account because of the low numbers

Arguments	Female author	Male author	Other
No argument	10430	3347	84
Pro-vaccination statement	727 (59,4%)	476 (38,9%)	20 (1,6%)
Conspiracy theories/search for truth	548 (68,9%)	240 (30,2%)	7 (0,9%)
Misinformation and falsehoods	374 (66,8%)	182 (32,5%)	4 (0,7%)
Safety and effectiveness	298 (75,5%)	97 (24,6%)
Civil liberties	282 (76,0%)	89 (24,0%)
Personal testimonies: Stories about harmed children/personal experiences	279 (91,2%)	25 (8,2%)	2 (0,7%)
Morality, religion, and ideology	173 (71,8%)	68 (28,2%)
Alternative medicine	125 (82,8%)	26 (17.2%)

Moreover, their distribution varied between the three categories of gender involved in the discussion. The “other” used only the following four types of arguments out of eight distinguished in the analysis: Conspiracy theories (9), Misinformation and falsehoods (16), Personal experiences (2) and Pro-vaccination statement (20). However, the majority of statements published by this category of authors were classified as comments lacking arguments (67).

Female authors were more likely than male authors to post comments with no argument, while when posting comments containing arguments, they used arguments classified as Alternative medicine, Personal testimonies or Safety proportionally more often than the male authors.

These differences are significant also when comparing the distribution of comments containing an argument (N = 4042; N(male) = 1203, N(female) = 2806) or without any argument (N = 13861, N(male) = 3347, N(female) = 10430) is taken into account (Chi = 53.246, p < .001). The “other” category was not taken into account because of the low number of comments.

Female authors were more active in the discussions collected for the study. However, in this group the proportion of comments containing no argument was higher than in the comments made by male authors.

We also compared the number of pro-vaccination and anti-vaccination arguments. The difference is statistically significant (Chi = 74.795, p < .001) with men making 39% of pro-vaccination arguments (N(female) = 727, N(male) = 476), and 26% of anti-vaccination arguments (N(female) = 2019, N(male) = 727).

3.4. Sentiment analysis – happiness and anger in the comments

Each emotion may be expressed in a text with the maximum intensity of 100%. When it is not present in the text, its intensity equals 0%. In the Table 3 a summary of the automatic emotion measurements is presented: the mean, maximum and standard deviation (SD) values for anger and happiness expressed in comments containing no argument or comments assigned to one of the categories described in the previous chapter.

Table 3. Share of individual components depending on arguments in sentiment analysis

argument	number of comments	anger (mean)	anger (maximum)	anger (SD)	happiness (mean)	happiness (maximum)	happiness (SD)
No argument	13861	31%	100%	17%	30%	100%	21%
Alternative medicine	151	33%	80%	16%	29%	82%	16%
Civil liberties	371	43%	97%	17%	19%	69%	11%
Conspiracy theories/search for truth	795	43%	98%	16%	19%	80%	11%
Misinformation and falsehoods	560	32%	100%	17%	25%	100%	15%
Morality, religion, and ideology	241	47%	99%	18%	18%	71%	10%
Personal testimonies: Stories about harmed children/personal experiences	306	46%	85%	16%	20%	64%	11%
Pro-vaccination statement	1223	38%	100%	16%	20%	100%	11%
Safety and effectiveness	395	42%	98%	16%	19%	100%	11%

3.4.1. Happiness and anger by comment types

In Figure 2, mean happiness and anger values for different categories of comments are presented. In the analyzed sample the comments without any arguments, as well as those containing statements about alternative medicine or misinformation and falsehoods were more positive and less angry in their emotional content, while personal testimonies and statements about morality and religion expressed the most anger.

Figure 2. Mean values for anger and happiness in sentiment analysis by comment category

When the difference between all the argument-containing and no-argument comments was analyzed, ANOVA showed significant difference for both anger (F = 807.161, p < .001) and happiness (F = 737.458, p < .001). The comments containing an argument tend to be more angry (mean(argument) = 40% vs mean(no-argument) = 31%) and less happy (mean(argument) = 21% vs mean(no-argument) = 30%).

When all the nine categories were taken into account, a significant effect was observed for both emotions: anger (F = 136.2, p < .001) and happiness (F = 101.049, p < .001). As is shown in the chart above, a cluster of more positive comment categories (with no argument, pointing to alternative medicine and to misinformation) and less positive ones could be observed, varying in their anger “loadings.” Morality and Personal Testimonies were the most angry comments in the sample.

In the analysis of the difference between pro-vaccination (N = 1223) and anti-vaccination (N = 2819) arguments, ANOVA proved significant for anger (F = 20.515, p < .001) but not for happiness (F = 0.338, p = .56). The pro-vaccination arguments tended to be less angry than the anti-vaccination ones.

3.4.2. Happiness and anger by gender

As could be expected from the results of comment category frequency by gender analysis, male and female authors differed in the intensity of both anger (F = 8.923, p = .0028) and happiness (F = 80.761, p < .001) expressed in their comments. Female authors expressed more happiness (mean = 29%) than male authors (mean = 26%) and slightly less anger (mean = 33%) compared to male authors (mean = 34%).

4. Discussion

Our study is, to our knowledge, the first analysis of anti-vaccination comments published on Polish social media. We have shown that conspiracy-theories and misinformation were the most frequent themes in the published comments, and that anger was the most frequently expressed emotion. Our results also show that a large majority of comments made on the anti-vaccination Facebook page were made by women overall, and that there were proportionately more female-authored comments among anti-vaccination arguments than among pro-vaccination arguments posted to the page we collected our data from.

The most commonly mentioned post-vaccination adverse reactions were autism spectrum disorders and other disorders affecting the nervous system, which confirms the results obtained in other studies²⁹⁠. This focus on adverse effects affecting the nervous system could be explained by low awareness of the actual frequency and nature of vaccine adverse events not only in the general public, but even in the population of medical students.³⁰ Statements about adverse reactions, often supported by anecdotal “evidence,” lack references to evidence-based medicine. They are, however, highly emotionally charged and widely available, which makes them appear more credible and increases the strength of association between vaccinations and adverse reactions in the society²¹⁠. A large number of comments concerning adverse reactions contradict the statistics^31,32⁠ on vaccine injury in Poland and probably results from the fact that the group of parents reporting vaccine injury is remarkably active, as confirmed in another study, where the approximately 20% of postings which were duplicates created an illusion of a greater number, producing the false notion that the injuries depicted or described occurred frequently²¹⁠. Vaccine deniers active on the page we analyzed do not talk about the dangers of vaccine-preventable diseases, and they ignore the evidence from scientific studies negating the link between vaccines and autism¹²⁠. This suggests that one of the approaches to combating the anti-vaccination misinformation could consist of storytelling with narratives about unvaccinated children who had suffered from vaccine-preventable diseases, countering anecdotes shared on social media by anti-vaccination posters²³⁠ and initiating discussions on whether the given child’s illness was causally related to vaccination or its lack. However, raising questions about the personal motivations of the loudest vaccine deniers is not recommended.

Some individuals seem to benefit from the widespread misinformation about vaccination and the distrust of biomedicine. This may be explained by a more general world-view since the opponents of vaccination are often associated with pro-natural groups and organizations, and support alternative and complementary medicine³³⁠. Comments in the Alternative medicine category referred directly to the “detoxification of vaccines,” the removal of heavy metals from the body after vaccination, or other therapies not supported in evidence-based medicine. We also, disturbingly, found communications offering paid therapies and diets that purportedly treat autism.

Anti-vaccine groups often encourage parents to make an “informed choice” while providing unsubstantiated and misleading information¹⁰⁠. Moreover, the notions of “medical liberty” remain popular, with the rhetoric of anti-vaccination activists unchanged for a few decades since the time when they opposed vaccination legislation³⁴⁠. In Poland, a mandatory vaccination policy is in operation^6,35⁠ to maintain a high vaccination rate. Although mandatory vaccination policies could be considered controversial from the individual liberty point of view,³⁶ herd immunity typically requires 85–95% vaccination uptake within a population³⁷⁠, from which the implication can be derived that vaccinating a child is not up to the “individual decision of the parent”²¹⁠. The analysis of the comments we collected suggests that financial penalties are effective in forcing opponents of vaccination to undergo at least some vaccinations. The commenters are aware that avoiding vaccination in a society with a mandatory vaccination policy bears the mark of “free-riding,” i.e. obtaining benefits without bearing the risk¹²⁠. Zarobkiewicz et al. have shown that there is a belief that non-vaccination of children would be penalized³⁰⁠. These concerns could be strengthened further by clear labeling of the behavior of anti-vaccine movement as the aforementioned “free-riding.”

Although vaccine hesitancy may have many causes, a high level of mistrust in conventional medicine is surely a leading one³⁸⁠. Regrettably, it is frequently a mistrust that is directed toward governments and pharmaceutical companies being allegedly implicated in world-scale conspiracies and is based on unclear premises²⁹⁠. Our study shows how much of the discourse with vaccine-hesitant individuals is occupied by such presumptions. Predominance of conspiracy arguments and decidedly non-mainstream sources can lead to increasingly extremist beliefs, and consequently to ideological isolation, where the penetration of evidence-based medicine becomes impossible and the anti-vaccine communities function in impenetrable echo chambers³⁹⁠. It also happens that conspiracy-focused anti-vaccination attitudes are sometimes supported by people with academic degrees^30,40⁠. Worryingly, such opinions and disinformation are also sometimes repeated and spread by prominent politicians²³⁠. The high frequency of conspiracy theories concerning genocide/depopulation suggests that a large number of people among the vaccine-hesitant perceive vaccines as a threat unequivocally detrimental to human health and seek explanation for the situation.

Studies show that religion-related content is of relatively lesser prominence in anti-vaccination communications.²⁹ Our analysis shows an even lower proportion of religion-related comments, which might be explained by the fact that, unlike in Poland, in the US it is possible to avoid vaccination in some states by citing philosophical/religious exception⁴¹⁠. Polish society is predominantly Catholic, and this denomination has no religious objections to the use of vaccines³⁸⁠.

The observed gender bias, with female-authored comments dominating the collected data may indicate that women have more concerns to do with children’s vaccination. It is reasonable to assume that a significant proportion of these comments come from mothers, or prospective mothers. This gender bias was also demonstrated in other research focused on groups and sites related to the anti-vaccination movement^22,23⁠.

Anti-vaccine arguments are in many cases emotionally loaded. As indicated by our results, the comments that convey no argument are positive in terms of emotional load and communicate the most happiness of all the comment types analyzed, which is similar to results obtained by Faasse et al.¹¹ as are our results concerning positive emotions (happiness) in anti-vaccine arguments.¹¹ However, in contrast to the results reported by Faasse et al., our analysis shows that pro-vaccine comments were less “angry” than anti-vaccine comments. The arguments against vaccination which proposed alternative medicine solutions and pointed out alleged misinformation, namely the ones that promoted “informed choice,” were also rather positive in their emotional tone, which may also encourage parents to join the anti-vaccination movement. Researchers in another study also found that the “Food as medicine” topic was characterized by a more positive sentiment than other topics,²² similarly to the comparable “alternative medicine” category in our analysis. On the other hand, the arguments which typically expressed the highest amount of anger were the Personal testimonies and the ones concerning Morality and religion. As a result, they may also be very persuasive.

Users of the Internet and social media include not only supporters and opponents of vaccinations, but also numerous parents who are vaccine-hesitant. Despite the necessity to develop an adequate strategy to promote vaccination to vaccine-hesitant parents, the most effective model of persuasion has still not been determined⁴²⁠. It was shown that previous educational strategies were not successful in encouraging parents to vaccinate, and some may even have had an inverse effect⁴³⁠. However, research conducted on a considerably larger sample of utterances collected from Twitter shows that pro-vaccine content is much more widespread than anti-vaccine posts⁴⁰⁠. We also know that the discussion about vaccination suggests the existence of two deeply polarized groups who only process information either in favor of or against vaccines⁴⁴⁠. Therefore, it is crucial for pro-vaccine users’ statements to be correctly structured. They should not be offensive or repeat anti-vaccine rhetoric, but they ought to improve general knowledge about vaccinations³³. In our study, many of the “pro-vaccine” comments were shown to be mocking and abusive.

Our study has several strengths. The categorization of the arguments in the comments was performed by human evaluators, which allowed us to obtain results unavailable to a computer algorithm. This allowed for an accurate interpretation of the content of the utterances. Language features, such as sarcasm or metaphor were less likely to cause a misunderstanding of the statement in comparison to categorization depending on algorithms⁴⁰⁠. Furthermore, the manual coding of users’ gender was also potentially more accurate than when coding gender automatically using a name database²²⁠. Despite the process of interpretation being time-consuming, a large number of comments were analyzed. However, the efficiency was not comparable with what can be achieved with the help of specialized analysis software. Nonetheless, this is the first study in Poland attempting to quantify the arguments in comments in the online anti-vaccination environment. What also can be considered a success is the fact that the study was conducted on a Facebook page that gathers vaccine deniers, as there is a tendency in those groups to transition into less public arenas outside of the social media altogether in the face of mainstream criticism⁴⁵⁠.

5. Limitations

Despite the advantages of the manual evaluation of the collected data described above, it is worth mentioning the disadvantages of this approach. Some statements were difficult to unambiguously categorize, and the boundaries between categories were not always clear. Such evaluation is necessarily subjective, even if conducted based on well-defined criteria.

A bigger limitation was the methodology’s focus on arguments without any validity, so that comments which potentially could have some merit, such as comments criticizing the qualification of children for vaccination or current medical procedures were not taken into account. Manual categorization of comments is also more time-consuming than similar tasks performed by a computer algorithm and leads to a smaller sample than in similar studies which used machine categorization.

The coding frame used to classify the comments was not comprehensive, as demonstrated by a large proportion of comments that were not assigned to any category. Moreover, pictures, which are responsible for a significant part of the emotional impact of personal narratives were not taken into account²¹ ⁠. Our study aimed to present a quantitative evaluation of the content vaccine-hesitant parents may encounter online, focusing on the relative proportion of comments devoted to various topics. However, the relative impact of different comments may vary and so could individual interpretation by each user²⁹ ⁠. It is also possible that, for example, some users only read short comments, avoiding more extensive statements.

One of the most significant limitations of this study is that it was limited to a single Facebook page, without a wider analysis of Facebook or other online content. We might have also missed some of the most extreme comments or those that questioned the page’s motivations or operations, which may have been removed either by Facebook or the page administrators.

A further distortion of the data might have occurred due to the fact that one person could post the same comment multiple times, which is a known limitation of using social media data⁴⁶. What’s more, no user data have been collected, so it is not known how many comments came from each user. There may be a group of extremely active users responsible for a large proportion or even the majority of the content posted to the page and it may have distorted our analysis.

For the most, sentiment analysis is based on the identification of keywords and phrases. Such an approach is never completely accurate, as it would require a full knowledge of the language habits of the authors of the statements, taking into account each specific individual word usage as well as the contextual variability of meanings. Sentiment analysis of statements formulated in informal language is particularly difficult^46,47⁠. A large number of very short statements, as is typical of Facebook comments, also hinders the operation of the algorithm.

6. Conclusions

Although we expected to encounter a particular type of content researching the comments on a vaccine deniers’ Facebook page, the number of comments as well as specific content are troubling, especially the high proportion of conspiracy theories and misinformation-spreading comments. The highly emotional character of anti-vaccine comments strongly suggests that vaccine deniers, at least the ones active in social media antivaccine spaces, hold views that would be very difficult to change, and pro-vaccination messages that do not address emotional connotations may prove ineffective.

However, anti-vaccine activists make, as of now, a relatively small if very vocal group. Being aware of their arguments, public health institutions can prepare the rest of society for confrontation with “anti-vaccine indoctrination.” The results of our study suggest that the key to achieving this goal is focusing on combating distrust by making sources of unbiased information on vaccinations available, ensuring transparency of procedures, and support of credible and trustworthy experts in creating pro-vaccine content.

Article highlights

• The largest category of comments concerned conspiracy theories and misinformation.

• Autism was the most-mentioned post-vaccination reaction.

• The pro-vaccination arguments tended to be less angry than the anti-vaccine ones.

Author contributions

K.K. collected the data, drafted and drafted the manuscript, analyzed the data, and conducted critical revisions. A.C. drafted the manuscript and analyzed the data. K.B. collected and analyzed the data. Ł.B. conceived the manuscript and conducted critical revisions. All authors have read and agreed to the published version of the manuscript.

Disclosure of potential conflicts of interest

There are no potential conflicts of interest to disclose.

Acknowledgments

We want to thank Sentimenti team (https://sentimenti.com) for help in emotional analysis of the data.

Additional information

Funding

Part of the research (sentiment analysis) was co-financed by the National Centre for Research and Development, Poland, grant no [POIR.01.01.01-00-0472/16] – Sentimenti (https://sentimenti.com/eu-grant/).