All mouth and trousers? Use of the Devil’s Advocate questioning protocol to determine authenticity of opinions about protester actions

Figures & data

Table 1. The six levels of activism and an example of each level.

Order level	Example
1. Public protest/passive support for cause	Display a poster or car sticker with a political message
2. Civil disruption/slightly active (lawful) support for cause	Attend an informational meeting of a relevant political group
3. Social disruption/persuade or try to convince others of view and to support cause	Invite a friend to attend a meeting of a relevant political organisation or event
4. Civil disobedience	Use tactics designed to overwhelm police resources to make their point
5. Criminal damage	Graffiti an opinion on public property to make their point
6. More extreme criminal actions/risk personal safety	Use physical violence to make their point

Table 2. Level of action allocation as a function of veracity.

Level	Truth tellers		Lie tellers
	N	%	N	%
1	27	49.1	25	45.5
2	3	5.5	3	5.5
3	23	41.8	23	41.8
4	1	1.8	1	1.8
5	1	1.8	0	0
6	0	0	0	0

Table 3. The self-reported strategies used by lie tellers and truth tellers.

Strategy	Example	Lie tellers	Truth tellers
Improvise	Go with the flow/thought quickly	8	0
Speech-related	Repeating questions, pretend not to be knowledgeable, used a conversational style	7	0
Nonverbal demeanour	Controlling behaviour such as hands, eye contact or appearing confident to interviewer	5	1
Truth	Based answer on their own opinions etc	4	6
Reverse	Thought of own opinion and said the opposite	4	0
Consistency	Consistent answers	4	0
Example	Based answer on an example and kept building on that	2	3
Solution	Came up with a specific solution to problem e.g. ‘I said they should write to the university’	1	2
Other person perspective	Thought of someone with opposing views and imagined what they would say	1	1
Facts	Based answer on facts they know/other people’s anecdotes etc.	0	3

Table 4. Statistical results for the expressing actions question, eliciting-opinion questions and devil’s advocate question.

	Truth			Lie			NHST			Equivalence testing
	M	SD	95% CI	M	SD	95% CI	F	p	d [95% CI]	t	p
Expressing actions question (Q1)
Pro-action arguments	2.73	1.87	2.36, 3.10	0.29	0.57	−0.08, 0.66	85.48	<.001	1.77 [1.30, 2.18]	7.35	1.00
Anti-action arguments	0.45	0.77	0.16, 0.75	0.87	1.36	0.58, 1.17	3.94	.050	0.38 [0.00, 0.75]	0.39	.349
Plausibility	4.71	0.94	4.49, 4.93	3.20	0.68	2.98, 3.42	93.75	<.001	1.84 [1.37, 2.26]	6.47	1.00
Immediacy	4.33	0.90	4.12, 4.54	3.29	0.66	3.08, 3.50	47.29	<.001	1.32 [0.89, 1.71]	3.56	1.00
Clarity	4.38	0.99	4.14, 4.63	3.64	0.82	3.39, 3.88	18.40	<.001	0.81 [0.41, 1.19]	1.41	.920
Being scripted	2.18	0.67	1.98, 2.38	1.98	0.80	1.78, 2.18	2.01	.159	0.27 [–0.11, 0.64]	−2.13	.018
Three eliciting-opinionquestions (averaged)
Pro-action arguments	3.02	2.63	2.45, 3.59	1.02	1.44	0.45, 1.59	24.35	<.001	0.94 [0.54, 1.32]	3.70	1.00
Anti-action arguments	3.16	1.90	2.54, 3.79	3.64	2.72	3.01, 4.26	1.11	.294	0.20 [–0.17, 0.58]	0.06	.476
Plausibility	4.39	0.88	4.19, 4.58	3.19	0.54	2.99, 3.38	74.44	<.001	1.64 [1.19, 2.05]	5.03	1.00
Immediacy	3.98	0.85	3.78, 4.18	3.15	0.63	2.95, 3.35	33.18	<.001	1.11 [0.69, 1.49]	2.27	.987
Clarity	4.17	0.92	3.95, 4.39	3.18	0.68	2.96, 3.39	41.30	<.001	1.22 [0.80, 1.61]	3.19	.999
Being scripted	1.95	0.61	1.78, 2.12	1.82	0.65	1.65, 1.99	1.22	.272	0.21 [–0.17, 0.58]	−3.03	.002
Devil’s advocate question
Pro-reason arguments	0.64	0.78	0.47, 0.80	0.16	0.37	0.00, 0.33	16.49	<.001	0.79 [0.39, 1.16]	−0.23	.408
Anti-reason arguments	0.89	0.99	0.61, 1.17	0.76	1.05	0.49, 1.04	0.43	.516	0.13 [–0.25, 0.50]	−1.91	.029
Plausibility	4.45	0.98	4.22, 4.69	3.15	0.73	2.92, 3.38	63.25	<.001	1.50 [1.06, 1.90]	4.92	1.00
Immediacy	4.05	0.95	3.83, 4.28	3.05	0.73	2.83, 3.28	38.24	<.001	1.18 [0.76, 1.57]	3.09	.999
Clarity	4.22	1.10	3.95, 4.49	3.22	0.90	2.95, 3.49	27.31	<.001	1.00 [0.58, 1.38]	2.61	.995
Being scripted	1.96	0.86	1.74, 2.18	1.60	0.78	1.38, 1.82	5.37	.022	0.44 [0.05, 0.81]	−0.87	.193

Note: NHST = Null-hypothesis significance testing; CI = confidence interval.

Table 5. Statistical results for the three individual eliciting-opinion questions.

	Truth			Lie			NHST			Equivalence testing
	M	SD	95% CI	M	SD	95% CI	F	p	d [95% CI]	t	p
Q2. Multivariate F(6, 103) = 11.14, p< .001, ${\eta }_{\mathrm{p}}{}^2$ = .39
Pro-action arguments	1.64	1.79	1.27, 2.01	0.42	0.79	0.05, 0.79	21.38	<.001	0.88 [0.48, 1.26]	2.73	.996
Anti-action arguments	0.55	1.02	0.24, 0.85	0.69	1.25	0.39, 1.00	0.45	.503	0.12 [–0.25, 0.49]	1.64	.052
Plausibility	4.53	0.94	4.30, 4.76	3.18	0.80	2.95, 3.42	65.65	<.001	1.55 [1.10, 1.95]	5.09	1.00
Immediacy	4.09	1.01	3.85, 4.34	3.16	0.81	2.92, 3.41	28.35	<.001	1.02 [0.60, 1.40]	2.45	.992
Clarity	4.31	1.09	4.04, 4.58	3.35	0.91	3.08, 3.61	25.49	<.001	0.96 [0.55, 1.34]	2.43	.992
Being scripted	1.95	0.80	1.72, 2.17	1.76	0.88	1.54, 1.99	1.28	.261	0.23 [–0.15, 0.60]	−1.98	.025
Q4. Multivariate F(6, 103]= 9.43, p< .001, ${\eta }_{\mathrm{p}}{}^2$ = .36
Pro-action arguments	0.95	0.89	0.74, 1.16	0.42	0.66	0.21, 0.63	12.47	<.001	0.68 [0.28, 1.05]	0.18	.572
Anti-action arguments	0.11	0.37	−0.10, 0.32	0.45	1.03	0.25, 0.66	5.45	.021	0.44 [0.05, 0.81]	1.04	.150
Plausibility	4.27	0.99	4.05, 4.49	3.24	0.61	3.02, 3.46	43.80	<.001	1.25 [0.83, 1.64]	3.43	1.00
Immediacy	3.89	0.92	3.67, 4.12	3.11	0.76	2.88, 3.33	23.67	<.001	0.92 [0.52, 1.30]	1.75	.959
Clarity	4.13	1.02	3.88, 4.37	3.09	0.80	2.85, 3.33	35.19	<.001	1.13 [0.72, 1.52]	3.07	.999
Being scripted	1.93	0.72	1.73, 2.12	1.71	0.74	1.52, 1.90	2.48	.118	0.30 [–0.08, 0.67]	−2.03	.022
Q5. Multivariate F(6, 103) = 13.71, p< .001, ${\eta }_{\mathrm{p}}{}^2$ = .44
Pro-action arguments	0.44	0.88	0.24, 0.63	0.18	0.55	−0.01, 0.38	3.35	.071	0.35 [–0.03, 0.73]	−1.76	.041
Ant-action arguments	2.50	1.65	2.05, 2.97	2.49	1.78	2.03, 2.95	0.003	.956	0.01 [–0.37, 0.38]	−1.47	.072
Plausibility	4.36	1.08	4.13, 4.60	3.15	0.62	2.91, 3.38	52.74	<.001	1.37 [0.94, 1.77]	4.28	1.00
Immediacy	3.95	0.99	3.71, 4.18	3.18	0.77	2.95, 3.42	20.37	<.001	0.87 [0.46, 1.25]	1.56	.939
Clarity	4.07	1.14	3.82, 4.33	3.09	0.75	2.83, 3.35	28.56	<.001	1.02 [0.60, 1.40]	2.62	.995
Being scripted	1.98	0.80	1.76, 2.21	1.98	0.87	1.76, 2.21	0.00	1.00	0.00 [–0.37, 0.37]	3.13	.001

Note: NHST = null-hypothesis significance testing.

Table 6. Statistical results for the residue scores.

	Truth			Lie			NHST			Equivalence testing
	M	SD	95% CI	M	SD	95% CI	F	p	d [95% CI]	t	p
Pro-reason arguments	−0.40	2.51	−1.16, 0.36	−3.22	3.15	−3.98, –2.46	26.91	<.001	0.99 [0.58, 1.37]	4.27	1.00
Plausibility	−0.07	0.68	−0.23, 0.10	0.04	0.53	−0.12, 0.21	0.88	.351	0.18 [–0.20, 0.55]	3.35	<.001
Immediacy	−0.08	0.68	−0.26, 0.11	0.10	0.70	−0.09, .028	1.79	.184	0.26 [–0.12, 0.63]	2.47	.008
Clarity	−0.05	0.87	−0.26, 0.16	−0.04	0.66	−0.25, 0.17	0.002	.967	0.01 [–0.36, 0.39]	3.34	<.001
Being scripted	−0.01	0.77	−0.21, 0.19	0.22	0.71	0.02, 0.42	2.65	.106	0.31 [–0.07, 0.68]	1.91	.03

Note: NHST = Null-hypothesis significance testing.

Table

Vignettes used
Cannabis
Jane has developed multiple sclerosis and her health and ability to function or hold down her job was suffering badly, until a friend recommended she try using cannabis to help with her pain and symptoms. She was sceptical as she had tried using CBD oil which hadn’t helped at all, but she tried using cannabis and was amazed at how much it helped her. She started using a small amount each evening and was able to resume her job and other activities as well as before her diagnosis. However, the man she bought the cannabis from was apprehended by police and gave Jane’s details as one of his customers. Jane was arrested and, as a result, lost her job whilst awaiting trial. Jane and husband Matt cannot manage the mortgage on their small home just on Matt’s wage and Jane is struggling to get another job, even if she could manage to work without using the cannabis for her symptoms.
University fees
Jane, aged 30, decided to pursue her dream job as an English teacher, having become very dissatisfied with her work as an administrative assistant which she had done since leaving school with just GCSEs. Jane was easily capable of doing a degree in English and paid to study for A levels to secure her place at university. She studied hard for these whilst still working and received ‘A’ grades in all three subjects. However, as she was about to start her degree the university fees tripled, and to get into so much debt in order to study further was not possible for her, as she and her husband Matt could not financially manage that and their small home and child. Jane is now 40 and still working as an administrative assistant. Life is hard and she and husband Matt struggle financially, with any spare money they can save going towards their child’s university education.
Cancer/COVID
Jane has an aggressive form of cancer. She was told she would qualify for a promising experimental treatment of the disease on the NHS. However, whilst waiting for the treatment the Coronavirus Omicron variant started spreading rapidly and she was informed that she would have to wait for the treatment as it would require a hospital stay and all hospital places were currently full. Jane and her husband, Matt, have heard that the beds are full of unvaccinated patients who have experienced complications with Coronavirus. When Jane was finally called in for her treatment she was informed that her cancer had progressed too far for the experimental treatment to now be a possibility.
Cost of living crisis
Jane was very excited to be buying a house finally. She and husband Matt, in their late 30s, had been living in a small damp one-bedroom rented flat for several years whilst saving for a deposit to buy a small home of their own. Jane had been working an evening bar job as well as her full-time day job as a carer in a nursing home, but had to give up her extra job due to exhaustion and the fact that Jane’s elderly mother’s health is now failing, requiring Jane to visit daily and give her substantially more practical help. In addition, Jane suffers from asthma which is exacerbated by the dampness at home. Matt, whose job is based out of town, needs to use their small old car to get to his job. If Jane’s mother lived in a larger house the couple would temporarily move in with her, but she lives in a one-bed cottage. With the dramatic recent increase in fuel prices, and their knock-on effects on the cost-of-living, combined with their reduced income, the couple now have to use their savings just to keep the flat warm and themselves fed. After their car broke down as Matt tried driving home from work they have just learned it needs a new engine which will cost the majority of their remaining savings. Jane had already sold anything of value that they owned and, as further fuel increases are promised, no longer turns on the heating, though the cold makes her more unwell.