Assessing the Accuracy, Quality, and Readability of Patient Accessible Online Resources Regarding Ocular Gene Therapy and Voretigene Neparvovec

Figures & data

Table 1 Questionnaire to Assess Online Resources Describing Ocular Gene Therapy with Mean and Total Scores

	Mean Points^a
Questions	AAO^b	AOA^c	EyeWiki	FDA^d	MedicineNet	NEI^e	Novartis	Spark Therapeutics	WebMD	Wikipedia
What is a gene?	0.33	0.00	1.67	1.33	0.33	1.33	0.67	3.33	0.67	1.33
What is gene therapy?	2.33	1.33	4.00	2.67	0.67	1.67	1.67	3.67	0.67	1.33
How does gene therapy work?	2.33	2.00	4.00	3.00	0.67	3.00	2.67	3.33	0.67	1.33
Are there any FDA approved gene therapy treatments?	3.33	2.67	4.00	4.00	3.33	0.00	1.33	4.00	4.00	4.00
What conditions can be treated with gene therapy?	3.33	3.00	4.00	4.00	2.00	3.00	3.67	3.33	3.33	3.67
What is retinal dystrophy?	1.33	0.67	1.33	3.33	1.00	1.00	3.00	1.67	1.67	1.00
What is Leber’s Congenital Amaurosis?	2.67	1.33	2.00	1.00	0.67	3.00	3.00	3.00	1.33	3.33
Are all patients with Leber’s Congenital Amaurosis eligible for gene therapy?	3.00	2.67	3.00	1.00	1.33	2.00	3.00	3.33	2.00	2.33
What is RPE65?	1.67	1.33	4.00	3.33	1.33	4.00	3.33	3.67	2.33	2.67
Does gene therapy cure the conditions it is used to treat?	2.00	0.67	0.67	1.00	0.67	3.33	1.67	1.00	1.00	3.67
At what age may gene therapy be administered?	0.00	0.00	0.00	1.00	1.67	0.00	0.00	3.67	2.67	0.67
What is a vector?	0.00	0.00	4.00	1.33	0.00	2.67	0.00	1.00	0.00	2.67
What vectors can be used for gene therapy?	0.00	0.00	4.00	0.67	0.00	0.67	0.00	1.00	0.00	2.33
What are the potential risks associated with gene therapy?	0.00	0.00	3.33	3.67	1.00	0.00	0.00	4.00	2.33	1.00
What is the price of gene therapy?	1.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1.00	2.67
Does insurance approve gene therapy procedures?	0.33	0.00	0.00	0.00	0.00	0.00	0.33	0.33	0.00	0.00
What is Luxturna?	3.33	0.00	3.67	4.00	2.00	1.33	3.33	4.00	3.67	4.00
What is an intravitreal injection?	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Are there FDA approved gene therapies delivered through intravitreal injection?	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
What is a pars plana vitrectomy with a subretinal injection?	0.00	1.00	2.67	1.67	0.00	0.00	0.00	1.33	0.00	1.00
Are there FDA approved gene therapies delivered through pars plana vitrectomy with a subretinal injection?	0.00	1.33	4.00	2.67	0.00	0.00	0.33	2.67	1.33	0.67
How long can results of treatment last?	2.00	0.00	0.00	0.33	0.00	2.33	0.67	0.00	0.67	0.00
Are repeat treatments of gene therapy necessary?	0.00	0.00	0.00	0.00	0.00	0.00	0.67	0.67	1.00	0.00
Is there gene therapy available for retinitis pigmentosa?	2.33	0.00	0.67	0.33	0.00	0.00	0.67	0.67	0.00	1.33
Is there gene therapy available for Stargardt’s disease?	0.67	0.00	0.33	0.33	0.00	0.00	0.00	0.00	0.00	0.33
Total Points, Max 100	32.00	18.00	51.33	40.67	16.67	29.33	30.00	49.67	30.33	41.33
Points, Mean	1.28	0.72	2.05	1.63	0.67	1.17	1.20	1.99	1.21	1.65
SD	1.25	0.96	1.71	1.44	0.85	1.33	1.31	1.53	1.21	1.34
95% CI	0.49	0.38	0.67	0.56	0.33	0.52	0.51	0.60	0.48	0.53

Notes: ^aScores of individual questions scaled from 0–4. ^bAmerican Academy of Ophthalmology. ^cAmerican Optometric Association. ^dU.S. Food and Drug Administration. ^eNational Eye Institute. Table 1 displays the 25 questions designed to assess accuracy and quality of each article along with the mean number of points each article received for each question. A grading scale from 0–4, with 4 as a maximum, was used to evaluate each question. Each score in the table is the average of three scores given by three independent graders.

Table 2 Mean and Total Scores for Grading Online Resources Describing Ocular Gene Therapy

	Total Points^a	Points, Mean	SD	CI 95%
EyeWiki	51.33	2.05	1.71	0.67
Spark Therapeutics	49.67	1.99	1.53	0.60
Wikipedia	41.33	1.65	1.34	0.53
FDA^b	40.67	1.63	1.44	0.56
AAO^c	32.00	1.28	1.25	0.49
WebMD	30.33	1.21	1.21	0.48
Novartis	30.00	1.20	1.31	0.51
NEI^d	29.33	1.17	1.33	0.52
AOA^e	18.00	0.72	0.96	0.38
MedicineNet	16.67	0.67	0.85	0.33

Notes: ^aOut of a possible 100 points. ^bU.S. Food and Drug Administration. ^cAmerican Academy of Ophthalmology. ^dNational Eye Institute. ^eAmerican Optometric Association. Table 2 displays the total scores each of the 10 articles received on the 25-question assessment in order from highest to lowest score. These scores were obtained using the average score between three independent graders.

Table 3 Mean and Total Scores for Online Sources Describing Ocular Gene Therapy Graded Using the DISCERN Instrument

	Total Points^a	Points, Mean	SD	CI 95%
EyeWiki	60.00	3.75	1.27	0.62
Spark Therapeutics	54.50	3.41	1.34	0.66
FDA^b	53.00	3.31	1.21	0.59
Wikipedia	47.50	2.97	1.18	0.58
AAO^c	46.00	2.88	1.02	0.50
NEI^d	41.50	2.59	1.06	0.52
Novartis	41.00	2.56	1.12	0.55
AOA^e	39.50	2.47	0.89	0.44
WebMD	39.00	2.44	0.97	0.47
MedicineNet	30.00	1.88	0.89	0.44

Notes: ^aOut of a possible 80 points. ^bU.S. Food and Drug Administration. ^cAmerican Academy of Ophthalmology. ^dNational Eye Institute. ^eAmerican Optometric Association. Table 3 displays the total scores each article received on the 16-question DISCERN instrument in order from highest to lowest score. These scores were obtained using the average score between two independent graders.

Table 4 JAMA Benchmarks Achieved by Online Sources Regarding Ocular Gene Therapy

JAMA Benchmarks	N (%)
4 Benchmarks	0 (0%)
3 Benchmarks^a	1 (10%)
2 Benchmarks	3 (30%)
1 Benchmark	5 (50%)
0 Benchmarks^b	1 (10%)
Attribution	4 (40%)
Authorship	4 (40%)
Currency	6 (60%)
Disclosure	0 (0%)

Notes: ^aEyeWiki achieved 3 of the 4 benchmarks. ^bSpark Therapeutics achieved 0 of the 4 benchmarks. Table 4 displays the number of articles that achieved JAMA benchmarks and the most common benchmarks displayed by the articles. Each article was assessed for the 4 standards as determined by JAMA: Authorship, attributions, disclosure, and currency.

Table 5 Readability Analysis of Online Sources Describing Ocular Gene Therapy

Readability	Wikipedia	Spark Therapeutics	AAO^a	EyeWiki	MedicineNet	NEI^b	AOA^c	WebMD	Novartis	FDA^d
Flesch Reading Ease	37.6	53.5	50.4	21.8	42.5	41.2	46.2	45.2	32	24.7
Mean Reading Grade	9.63	10.40	11.45	12.85	13.03	13.03	13.63	13.75	14.35	16.73
Mean Reading Grade SD	1.22	1.05	0.90	1.03	0.93	0.92	1.91	1.39	0.96	1.76
Mean Reading Grade CI (95%)	1.20	1.03	0.88	1.01	0.91	0.90	1.87	1.36	0.94	1.72

Notes: ^aAmerican Academy of Ophthalmology. ^bNational Eye Institute. ^cAmerican Optometric Association. ^dU.S. Food and Drug Administration. Table 5 displays the mean readability scores as determined for each article by the online tool, Readable, and reported in increasing mean reading grade order. The Flesch Kincaid Grade Level, Gunning Fox Index, Coleman Liau Index, and SMOG Index state a US reading grade level. The mean score of these indices was used to determine the numerical grade level of the article. A mean reading grade of 12 indicates a 12th grade reading level while a score above 12 corresponds to a college reading level.

Table 6 Summary Table of Grading Indices Used to Assess Online Sources Describing Ocular Gene Therapy

Article Source	25-Question Assessment Total Points^a	DISCERN Instrument Total Points^b	Number of JAMA Benchmarks Achieved^c	Mean Reading Grade^d
AAO^e	32.00	46.00	1	11.45
AOA^f	18.00	39.50	1	13.63
EyeWiki	51.33	60.00	3	12.85
FDA^g	40.67	53.00	1	16.73
MedicineNet	16.67	30.00	2	13.03
NEI^h	29.33	41.50	2	13.03
Novartis	30.00	41.00	2	14.35
Spark Therapeutics	49.67	54.50	0	10.40
WebMD	30.33	39.00	1	13.75
Wikipedia	41.33	47.50	1	9.63

Notes: ^aOut of a possible 100 points. ^bOut of a possible 80 points. ^cOut of a possible 4 benchmarks. ^dAverage of 4 indices reporting US reading grade levels. ^eAmerican Academy of Ophthalmology. ^fAmerican Optometric Association. ^gU.S. Food and Drug Administration. ^hNational Eye Institute. Table 6 provides a summary of all the grading indices used in assessing the articles including the 25-question assessment, the DISCERN instrument, the number of JAMA benchmarks achieved, and the mean reading grade. The total points on the 25-question assessment is the average score between three independent graders. The total points on the DISCERN instrument is the average score between two independent graders. The number of JAMA benchmarks achieved quantifies the number each article met of the 4 JAMA benchmarks: Authorship, attributions, disclosure, and currency. The mean reading grade states a US reading grade level with a mean score of 12 reflecting a 12th grade reading level while a score above 12 corresponds to a college reading level.