Mapping spontaneous facial expression in people with Parkinson’s disease: A multiple case study design

Figures & data

Table 1. Participant demographics and overall sample central tendency

Women							Men
ID	Age	MoCA	GDS	Expr	UPDRS face	Disease severity	ID	Age	MoCA	GDS	Expr	UPDRS face	Disease severity
A	69	26	1	5.00	Slight	Bilateral (2)	E	62	24	2	4.80	Slight	Bilateral (2)
B	67	29	1	4.25	Slight	Bilateral (2)	F	63	27	3	3.50	Slight	Unilateral (1)
C	72	27	1	3.20	Mild	Bilateral (2)	G	60	26	3	2.60	Mild	Bilateral (2)
D	67	24	1	2.40	Slight	Severe (4)	H	66	26	0	1.50	Moderate	Severe (4)
Sample	63	26	3.39	3.65	Slight	Bilateral (2)	Sample	66	25	1.63	2.62	Mild	Bilateral (2)

Notes: Within genders, participants are ordered by their expressivity score. MoCA = Montreal Cognitive Assessment, GDS = Geriatric Depression Scale, Expr = trained coder rated expressivity, UPDRS face = Unified Parkinson Disease Rating Scale Facial Item Rating, Disease severity = Hoehn and Yahr disease severity rating, corresponding numerical scores are in parentheses; Statisitics in the row Sample are mean or medians for the whole sample of 55 participants from which the 8 participants were purposefully selected.

Table 2. Number and description of observed action unit

Action unit #	Description
1	Inner brow raise
2	Outer brow raise
4	Brow lowerer
5	Upper lid raise
6	Cheek raise
7	Lids tight
10	Upper lip raiser
12	Lip corner puller
14	Dimpler
17	Chin raiser
18	Lip pucker
20	Lip stretch
23	Lip tightener
24	Lip presser
28	Lip suck
37	Lip wipe
43	Eye closure
45	Blink

Figure 1A. Visualization of activation and intensity of AUs across 600 frames and summary statistics for women.

Figure 1B. Visualization of activation and intensity of AUs across 600 frames and summary statistics for men.

Figure 2. Intensity correlations between AUs for women and men.

Table 3. Summary of activation for each participant

Case	# AUs produced	AU with largest % frames active	AU with greatest # of activations	AU with greatest % active frames at max intensity	AU with highest intensity	AU with greatest cumulative intensity	# Blinks
A	12	12 (52.67)	1(8)	18 (84.62, 2)	2, 43 (5)	2 (927)	2
B	9	12 (48.50)	6, 24 (2)	43 (65.00, 5)	12, 43 (5)	12 (691)	5
C	11	12 (67.83)	12, 17, 28, 37 (2)	17 (88.37, 2)	6 (4)	12 (534)	11
D	9	12 (27.83)	18 (3)	2 (73.97, 2)	18 (4)	12 (201)	13
E	8	12 (62.50)	2, 6, 12, 43 (2)	43 (78.93, 5)	6, 12, 43 (5)	6 (1,437)	7
F	6	4 (60.33)	1, 2 (4)	6 (53.57, 3)	4 (4)	4 (818)	5
G	5	12 (23.00)	12 (2)	6 (76.19, 2)	12 (4)	12 (260)	1
H	3	12 (38.00)	12, 17 (1)	–	12, 17 (1)	12 (228)	4

Notes: AU = Action unit. For all columns that present the AU that received the largest value for each category, the number in parentheses is the corresponding value for the AU. AU with greatest % active frames at max intensity presented the AU that was active at its maximum intensity for the greatest percentage of frames only for AUs that were active at an intensity greater than one. Participant H does not have a value for greatest % active frames at max intensity because he only produced AUs at an intensity of one.

Table 4. Correlations of action units across the face for each participant

Case	Upper face		Lower face		Between upper and lower
	Action units	r	Action units	r	Action units	r
A	1 & 2	.88	10 & 12	−.26	1 & 10, 18, 20, 23	.21 − .43
	2 & 43	.24	10 & 20	.65	2 & 10, 18, 20, 23	.25 − .45
	4 & 1, 2, 5, 43	−.56 − −.23	12 & 20	−.29	4 & 12	−.58
	5 & 1, 2	.61			4 & 20	−.25
	5 & 43	−.21			5 & 10	.48
	6 & 1, 2, 4, 5	−.31 − −.23			5 & 20	.76
					6 & 12	.65
B	1 & 2	.96	N/A		12 & 1, 2, 5, 6	.24 – .55
	1 & 5	.77
	2 & 5	.80
	6 & 1, 2, 5, 43	.30 − .50
C	1 & 2, 4, 6, 7	.53 − .97	12 & 14	−.69	1 & 14, 17, 28	−.30 − −.25
	2 & 4, 6, 7	.48 − .69	14 & 28	−.33	2 & 14, 17, 28	−.29 − −.24
	4 & 6	.40	17 & 28	.71	6 & 14	.44
	4 & 7	.76			6 & 17, 28, 37	−.52 − −.22
	6 & 7	.53			7 & 17, 28	−.20
					12 & 1, 2, 4, 7	.43 − .59
D	1 & 2	.99	12 & 17	−.24	1 & 17	.89
			14 & 18	.21	2 & 17	.90
					6 & 12	.73
					12 & 1, 2	−.23
E	1 & 2	.61	N/A		6 & 12	.94
	2 & 4	.56
F	1 & 2	.83	N/A		4 & 17	−.34
	1 & 4	−.37
	1 & 6	−.26
	1 & 45	.30
	2 & 4	−.43
	2 & 6	−.20
	2 & 45	.36
	4 & 6	.66
	4 & 45	−.20
G	1 & 2	.87	N/A		6 & 12	.72
H	N/A		12 & 17	−.23	N/A

Notes: In the action unit columns numbers reported refer to the specific AUs: 1 = inner brow raise, 2 = outer brow raise, 4 = brow lowerer, 5 = upper lid raiser, 6 = cheek raise, 7 = lids tight, 10 = upper lip raiser, 12 = lip corner puller, 14 = dimpler, 17 = chin raiser, 20 = lip stretch, 28 = lip suck, 45 = blink. N/A indicates there were no correlations in that region of the face. Ranges are given when action units are correlated with three or more other action units in that region of the face in the same direction. All correlations have an n = 600 and p < .00001.

Table 5. Summary statistics averaged across all eight participants

AU	N	% Frames active	# Active	% Active at max int	Max int	Cml int
1. Inner brow raise	7	17.10	2.43	53.60	2.57	239.00
2. Outer brow raise	7	16.55	2.43	44.06	2.71	224.43
4. brow lowerer	4	23.54	1.50	49.09	3.25	358.00
5. Upper lid raise	2	13.08	1.00	86.97	2.50	250.00
6. Cheek raise	6	25.12	1.57	35.12	3.29	380.43
7. Lids tight	1	9.17	1.00	29.09	3.00	108.00
10. Upper lip raiser	1	6.83	1.00	78.05	2.00	73.00
12. Lip corner puller	7	45.76	2.14	24.90	3.57	552.29
14. Dimpler	3	9.28	1.33	62.19	1.67	82.00
17. Chin raiser	5	12.07	1.40	49.49	2.20	108.20
18. Lip pucker	2	4.92	2.00	51.00	3.00	57.00
20. Lip stretch	1	15.33	2.00	58.69	2.00	146.00
23. Lip tightener	1	2.17	1.00	100.00	1.00	13.00
24. Lip presser	1	16.17	2.00	100.00	1.00	97.00
28. Lip suck	1	29.17	2.00	71.43	2.00	300.00
37. Lip wipe	2	5.25	1.50	21.62	2.00	39.50
43. Eye closure	3	8.44	1.33	73.90	5.00	227.33
45. Blink	8	8.19	6.00	–	–	–

Notes: N = The number of participants that produced the AU, % Frames active = Percent of the total 600 frames where the AU was active, # Active = The number of times a participants activated the AU during the 600 frames, % Active at max int = the percentage of active frames where the AU was activated at its maximum intensity, Max int = The maximum intensity at which the AU was activated, Cml int = The cumulative intensity or total of all intensity codes across the 600 frames.

Table 6. Correlations among summary statistics averaged across all 8 participants

	1.	2.	3.	4.
1. % Frames active
2. Number of activations	.29
3. % Of active Frames at max intensity	−.63	−.48
4. Maximum intensity	.76	.50	−.65
5. Cumulative intensity	.98	.53	−.51	.79

Notes: % Frames active = Percent of the total 600 frames where the AU was active, % of Active at max intensity = the percentage of active frames where the AU was activated at its maximum intensity.