Readability of online information about HPV Immunization

ABSTRACT

The human papillomavirus (HPV) vaccine is highly effective in preventing anogenital cancers, but vaccination rates in the United States remain low. In deciding whether to vaccinate their children, parents are increasingly using the internet to gather information. This study aimed to examine the level of readability of information on HPV vaccines written on 100 websites that were found via an internet search. To create the website sample, the first 50 websites from the search “HPV immunization” were recorded, followed by the first 50 non-duplicated websites from the search “Gardasil.” The content of the sites was analyzed using established readability scales. Websites were compared based on keyword search and on whether they had a commercial or non-commercial URL extension. The majority of websites were found to have information of a difficult reading level. Websites with commercial URL extensions and websites found using the search “Gardasil” had higher mean readability scores. These results suggest that many parents may not understand the information currently presented on the internet and that interventions aimed at improving the readability of online information could help to increase parental approval of the vaccine.

Keywords:

• human papilloma virus
• cervical cancer
• readability
• on-line information

Introduction

The human papillomavirus (HPV) vaccine, the two forms of which are Gardasil® and Cervarix®, is effective in preventing anogenital cancer and genital warts; the vaccine is therefore recommended for both females and males beginning at age 11–12.¹ Parents ultimately decide whether to vaccinate their children and current vaccination rates in the United States remain low. In 2015, among adolescents aged 13–17, only 41.9% of females and 28.1% of males had received the recommended three doses of the vaccine.²

The internet is increasingly used as a tool for parents to learn about the HPV vaccine. Surveys of parents of teenagers show that those who access information on the HPV vaccine from the internet have greater knowledge about HPV and more positive attitudes towards HPV vaccination.³ Nevertheless, previous studies of the readability of specific websites related to HPV immunization have found them to be of difficult readability. A study of information posted on the internet by the Canadian Provincial Department/Ministry of Health found the readability level to be inadequate for the general population to understand.⁴ Furthermore, a study of online messages about HPV vaccination in Japan found that most messages had “neutral” (i.e. not easy nor difficult) readability.⁵ To our knowledge, there have been no United States-based studies on the readability of information posted on the internet regarding HPV immunization. We therefore sought to analyze the readability of information on HPV immunization presented on popular websites.

Methods

The methods were adapted from a previous study on the readability of online information.⁶ We searched a cleared internet browser with the term “HPV immunization” and the URLs of the first 50 websites written in English were recorded. The keyword, “Gardasil” was then searched and the first 50 non-duplicated URLs were recorded, for a total of 100 websites in the sample. After confirming that each URL had relevant content to be analyzed, we analyzed all text from the direct URL. We did not search beyond this page of the website as that would be the likely tactic used by a person searching for information. Gardasil was chosen because it was introduced prior to Cervarix and thus more information referencing Gardasil is available on the internet.

We generated readability scores using Readable.io, a service recommended by Medline.⁷ Five commonly-recommended readability tests were used: Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook and (SMOG) Grade Level. The scores from these tests were then grouped into their established readability categories based on US grade level: “easy” (grade <6, age <11–12), “average” (grade 6–10, age 12–15), and difficult (grade >10, age 15–16+). A fifth test, the Flesch-Kincaid Reading Ease (FRE), reports readability on a 1–100 scale, where a higher score indicates more difficult readability.

We compared website content based on URL type, with reputable extensions (.org, .gov, .edu) designated as “Group 1” and those with other extensions (.com, .net, or other) designated as “Group 2.” Websites were also compared based on whether they were found by searching for “HPV immunization” or “Gardasil.” We performed statistical tests using SPSS (v23). For categorical variables, we used chi-square tests of association except where an expected cell count was less than 5, in which case we used Fisher’s exact tests. For continuous variables, we used independent sample t-tests. We considered results to be significant if p < 0.05.

The IRB at William Paterson University does not review studies that do not involve human subjects and considers them to be exempt.

Results

Four of the five readability tests found the majority of websites to be of difficult readability (grade level >10, Table 1). Three of the tests determined that no websites were graded as “easy” (grade <6), which is the suggested level for information aimed at the general public. Among the four tests that determine readability based on grade level, three found the average grade to be above 10th grade, which indicates difficult readability.

Table 1. Comparison of websites based on URL type (N = 100).

				Group 1 (n = 46)			Group 2 (n = 54)
Readability Test	Group 1^a (n = 46)	Group 2^b (n = 54)	p*	Easy	Average	Difficult	Easy	Average	Difficult	p**
	Score (Grade level) ^c
FKGL	9.1	9.9	0.095	2	30	14	1	30	23	0.436 ^X
GFI	9.8	10.9	0.026	2	27	17	0	18	36	0.004 ^X
CLI	11.0	11.4	0.225	0	15	31	0	8	46	0.035
SMOG	11.3	12.0	0.033	0	11	35	0	5	49	0.046
	Score
FRE ^c	53.1	50.6	0.253	0	16	30	0	9	45	0.037

^a Group 1: .org, .gov, .edu

^b Group 2: .com, .net, other

^c Corresponding ages: grade <6: age <11–12; grades 6–10: ages 12–15; grade >10: age 15–16 +

^d Unlike the other tests, FRE does not report readability by grade level but rather on a 1–100 scale, where a higher score indicates more difficult readability

* Independent sample T test

** Chi-square test

^X Fisher’s exact test

Two tests determined that websites with reputable extensions (Group 1) had lower mean readability scores than websites with other extensions (Group 2). These tests included the GFI (p = 0.026) and SMOG (p = 0.033). Similarly, four tests determined that Group 1 websites were more likely to be graded as “average” rather than “difficult.”

When compared by keyword, three tests found that websites found via the search term “Gardasil” had a significantly higher difficulty rating (Table 2). Another test found that websites varied significantly in how many were designated as easy, average, or difficult, with websites found via “Gardasil” having more websites considered difficult versus websites found using “HPV immunization” (p = 0.035).

Table 2. Comparison of websites based on search keyword (N = 100).

				HPV Immunization (n = 50)			Gardasil® (n = 50)
Readability Test	HPV Immunization (n = 50)	Gardasil® (n = 50)	p*	Easy	Average	Difficult	Easy	Average	Difficult	p**
	Score (Grade level) ^a
FKGL	9.1	10.1	0.038	2	33	15	1	27	22	0.402 ^X
GFI	10.0	10.9	0.094	2	27	21	0	18	32	0.035 ^X
CLI	10.9	11.6	0.038	0	14	36	0	9	41	0.235
SMOG	11.2	12.1	0.009	0	11	39	0	5	45	0.102
	Score
FRE ^b	54.4	49.1	0.017	0	16	34	0	9	41	0.106

^a Corresponding ages: grade <6: age <11–12; grades 6–10: ages 12–15; grade >10: age 15–16 +

^b Unlike the other tests, FRE does not report readability by grade level but rather on a 1–100 scale, where a higher score indicates more difficult readability

* Independent sample T test

** Chi-square test

^X Fisher’s exact test

Discussion

The findings of this study suggest that information on HPV immunization may be difficult for the general public to read. Given that there is substantial controversy regarding the vaccine, the information presented on the internet may play a substantial role in parent decision-making. Parent surveys suggest that there is uncertainty regarding the safety and effectiveness of the vaccine. Furthermore, some parents are concerned that vaccinating their children against a sexually-transmitted infection may promote sexual activity.³ In Japan, a study of online messages found that anti-vaccination messages had considerably higher readability than pro-vaccination messages, which suggests that the general public may be more influenced by messages aimed at discouraging vaccination.⁵ Our results suggest that information from government or university websites is easier to read. Parents who seek information from blogs or other “.com” websites are therefore more likely to find information that is difficult to read, which could cause confusion and discourage vaccination.

These results expand on the assertion by Patel et al. that healthcare providers should be educated on what information is available on the internet: healthcare providers should also understand what information would be accessible to the average reader.⁸ Previous studies of HPV information on the internet have focused on whether the text encouraged or discouraged parents from vaccinating their children.⁵ Our results suggest that further research is warranted to compare popular pro-vaccination and anti-vaccination websites based on readability. Furthermore, future studies could examine the ways that information is presented on websites outside of pure text, such as through graphics or images,

Our results suggest that interventions are needed to improve the readability of online information regarding HPV vaccination. Specifically, web content must be edited such that it can be read by the general public. The readability tools used in this study are readily available on the internet. Content writers should use these tools when developing content to ensure that their information can be read by a wide audience. Given the technical nature of information regarding vaccine safety and efficacy, readability guidelines could be established and adopted by universities, government institutions, or pharmaceutical companies to standardize the information that is presented.

Our study is limited in its cross-sectional design and in its reliance on materials from the internet, where information is continuously evolving. Furthermore, our materials were limited to websites written in English and we only analyzed the direct URL found, which means that the text analyzed may not be representative of the entire website. We also did not use Cervarix as a search term because of its shorter history than that of Gardasil.

Conclusion

Despite the limitations, our results suggest that interventions aimed at increasing readability could make online information more accessible to the general public. Given that HPV vaccination rates are low,² these interventions could increase parental approval of the vaccine if the information is easier to understand.

Disclosure of potential conflicts of interest

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was unfunded.