Development and Validation of the Perceived Deepfake Trustworthiness Questionnaire (PDTQ) in Three Languages

Figures & data

Figure 1. Questionnaire development procedure.

Table 1. Content validity of the retained items.

Item no.	CVR	CVI	Modification
Content of the video
The presented information seemed convincing. (I2)	.85 (12/13 ess.)	.92	No
The presented information seemed credible. (I30)	.69 (11/13 ess.)	.92	No
The message was not something I would expect from the person in the video. (I16)	.38 (9/13 ess.)	.92	Yes
The message was consistent with my previous knowledge and information. (I21)	.54 (10/13 ess.)	.92	Yes
The presented information seemed plausible. (I6)	.54 (10/13 ess.)	1.00	No
The message conveyed something that I already know to be true. (I11)	.38 (9/13 ess.)	1.00	Yes
The presented information seemed questionable. (I24)	.38 (9/13 ess.)	.92	No
The message contained errors. (I27)	.38 (9/13 ess.)	.92	Yes
Person in the video
The face of the person in the video (or parts of it) was distorted. (I20)	.50 (9/12 ess.)	.92	No
The facial features of the person in the video changed during the video. (I15)	.50 (9/12 ess.)	.83	No
I found the voice of the person in the video to be different from their usual voice. (I9)	.67 (10/12 ess.)	.92	No
The mouth movements of the person in the video did not completely match the sound. (I4)	.67 (10/12 ess.)	1.00	No
The face of the person in the video (or parts of it) was blurry. (I25)	.50 (9/12 ess.)	.92	No
I found the voice of the person in the video unnatural. (I7)	.50 (9/12 ess.)	.92	No
The person in the video behaved authentically. (I8)	.83 (11/12 ess.)	.83	No
The mouth of the person in the video was moving strangely. (I23)	.45 (8/11 ess.)	1.00	No
The gestures displayed by the person in the video did not seem natural. (I17)	.45 (8/11 ess.)	.82	Yes
Source of the video
I am not familiar with the source that published the video. (I26)	.50 (9/12 ess.)	.92	No
It is unclear to me who is the original source of the video. (I31)	.50 (9/12 ess.)	.92	No
The source of the video is verified in some way. (I14)	.50 (9/12 ess.)	.92	No
The source of the video is well-known. (I19)	.67 (10/12 ess.)	1.00	No
The content of the video is consistent with what this source has published previously. (I28)	.50 (9/12 ess.)	.92	No
The source of the video seems credible. (I22)	.67 (10/12 ess.)	.83	No
The video was posted by a reputable source. (I29)	.67 (10/12 ess.)	.92	No
Technical aspects
The background in the video contained irrelevant or out-of-place objects. (I5)	.50 (9/12 ess.)	.83	No
The background in the video seemed authentic. (I13)	.83 (11/12 ess.)	1.00	No
I noticed a lot of flickering and lighting anomalies throughout the video. (I1)	.67 (10/12 ess.)	.83	No
Something about this video seemed off. (I12)	.50 (9/12 ess.)	.92	No
The video was visibly edited. (I3)	.50 (9/12 ess.)	.83	No
The audio was low quality. (I10)	.50 (9/12 ess.)	.92	No
The video quality was inconsistent. (I18)	.50 (9/12 ess.)	1.00	No

Notes. Item number in the 31-item version of the questionnaire is shown in brackets.

Table 2. Demographic characteristics of subsamples.

Sample	N	Gender	Age	Education
English	313	59.1% female, 40.3% male, 0.6% other	16-87 years (M = 47.75, SD = 15.74)	55.3% secondary, 30.0% Bachelor’s, 11.5% Master’s
Italian	202	57.4% female, 42.1% male, 0.5% other	16-75 years (M = 48.50, SD = 12.92)	56.9% secondary, 20.8% Master’s, 18.8% Bachelor’s
Slovene	218	65.1% female, 33.9% male, 0.9% other	18-65 years (M = 28.49, SD = 10.82)	57.8% secondary, 29.8% Bachelor’s, 10.6% Master’s

Figure 2. Study procedure.

Figure 3. Scree plot.

Table 3. Eigenvalues and total variance explained by the extracted factors before and after rotation.

	Initial Eigenvalues			Extraction sums of squared loadings
Factor	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %
1	10.57	34.09	34.09	10.00	32.26	32.26
2	3.37	10.86	44.95	2.83	9.13	41.39

Table 4. Exploratory factor analysis: loadings and communalities.

	Factor 1	Factor 2	Communalities		Factor 1	Factor 2	Communalities
Item 1	−.05	.44	.17 (.25)	Item 17	.01	.68	.47 (.48)
Item 2	.68	−.02	.45 (.57)	Item 18	.06	.69	.52 (.54)
Item 3	.17	.43	.29 (.38)	Item 19	.66	−.21	.33 (.39)
Item 4	.04	.62	.42 (.49)	Item 20	−.11	.80	.55 (.58)
Item 5	−.03	.57	.30 (.43)	Item 21	.76	−.12	.48 (.54)
Item 6	.74	−.04	.52 (.63)	Item 22	.78	.11	.71 (.70)
Item 7	.06	.63	.44 (.56)	Item 23	.01	.76	.59 (.63)
Item 8	.46	.22	.37 (.45)	Item 24	.53	.29	.53 (.59)
Item 9	.02	.65	.43 (.56)	Item 25	−.21	.71	.39 (.46)
Item 10	−.16	.65	.34 (.37)	Item 26	.26	.18	.15 (.30)
Item 11	.73	−.22	.41 (.46)	Item 27	.35	.39	.43 (.46)
Item 12	.37	.48	.57 (.60)	Item 28	.59	−.16	.27 (.34)
Item 13	.19	.38	.26 (.44)	Item 29	.68	.02	.48 (.53)
Item 14	.75	−.04	.52 (.54)	Item 30	.83	.05	.74 (.71)
Item 15	−.20	.67	.34 (.35)	Item 31	.25	.15	.13 (.28)
Item 16	.36	.21	.25 (.38)	/	/	/	/

Note. Initial communalities are in brackets.

Figure 4. Standardized loadings for the modified 2-factor solution.

Table 5. Fit indices pertaining to the three investigated models.

	χ^2	CFI	RMSEA [90% CI]	SRMR	AIC	BIC	SABIC
1-factor solution	3054.43***	.64	.14 [.13; .14]	.12	55268.05	55571.46	55361.89
2-factor solution	1287.55***	.86	.08 [.08; .09]	.07	53503.17	53811.18	53598.43
Modified 2-factor solution	802.63***	.92	.06 [.06; .07]	.06	53026.45	53352.84	53127.40

Table 6. Measurement invariance of the English, Italian, and Slovene version.

	χ^2	CFI	RMSEA [90% CI]	SRMR	Comparison	△CFI	△RMSEA	△SRMR
Configural invariance	1423.28***	.89	.074 [.069; .079]	.074	–	–	–	–
Metric invariance	1534.59***	.88	.074 [.070; .079]	.088	Conf vs. Metric	.009	.000	.014

Table 7. Descriptive statistics and bivariate associations.

	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1. Trust. content	3.27	1.11	(.91)
2. Trust. presentation	3.75	1.11	.43***	(.89)
3. Age	42.23	16.32	.12***	.19***	–
4. Education	2.58	0.80	−.06	−.01	−.03	–
5. Conservatism	3.63	1.26	.14***	.01	.16***	.02	(.80)
6. Social media use	2.79	0.97	.15***	.03	−.28***	−.01	−.01	–
7. Bullshit receptivity	2.77	0.86	.27***	.05	.01	−.10**	.13***	.11**	(.79)
8. Reflectiveness	0.39	0.36	−.17***	−.05	−.18***	.05	−.07*	−.07	−.25***	(.66)
9. Trust in media	2.31	0.74	.14***	.06	−.13***	.08*	.01	.15***	.22***	−.01	(.72)
10. Media literacy	3.73	0.64	−.09*	−.06	−.11**	.10**	−.11**	.14***	.08*	.02	.02	(.77)
11. Deepfake knowledge	2.85	1.37	.08*	−.09*	−.20***	.05	.03	.16***	.14***	−.02	.16***	.24***	(.90)
12. Message believability	3.39	1.36	.67***	.52***	.05	−.02	.11**	.13***	.19***	−.06	.16***	−.04	.05	(.82)
13. Video fake	5.37	1.59	−.63***	−.50***	−.20***	.07*	−.09*	−.04	−.16***	.11**	−.10*	.12***	.01	−.52***	–
14. Behavioral int.	2.12	1.33	.52***	.28***	.12***	−.04	.11**	.14***	.29***	−.19***	.25***	−.05	.17***	.44***	−.40***	(.92)

Table 8. Comparison of perceived trustworthiness between the manipulated and the authentic video.

	Manipulated video		Authentic video		Comparison
	M	SD	M	SD	t	df	p	d
Trust. content	3.27	1.11	4.48	0.97	−24.60	732	< .001***	−0.91
Trust. presentation	3.75	1.11	5.37	1.06	−31.79	732	< .001***	−1.17

Table 9. Incremental validity of the perceived deepfake trustworthiness questionnaire.

	Perceived video fakeness				Viral behavioral intentions
	Step 1	Step 2	Step 3	Step 4	Step 1	Step 2	Step 3	Step 4
Age	−.22***	−.22***	−.18***	−.09**	.16***	.17***	.14***	.11**
Education	.07	.05	.05	.04	−.03	−.02	−.02	−.01
Conservatism	−.06	−.02	.01	.01	.09*	.04	.01	.01
Social media use	−.10**	−.08*	−.02	.02	.18***	.12**	.07*	.05
Bullshit receptivity		−.12**	−.05	−.01		.19***	.14***	.11**
Reflectiveness		.03	.03	−.01		−.10**	−.10**	−.08*
Trust in media		−.09*	−.03	−.02		.19***	.14***	.14***
Media literacy		.12**	.09**	.05		−.09*	−.07*	−.04
Deepfake knowledge		−.02	−.01	.00		.15***	.14***	.13***
Message believability			−.49***	−.08*			.36***	.15***
Trust. content				−.46***				.30***
Trust. presentation				−.24***				.04
R^2	.06***	.10***	.32***	.48***	.06***	.19***	.31***	.36***
ΔR^2		.04***	.22***	.16***		.14***	.12***	.05***

Data availability statement

Data will be made available on request.