The role of pointing gestures and eye gaze in second language vocabulary learning

Figures & data

Figure 1. Examples of stimuli illustrating the four main conditions: gaze, pointing only (eyes closed), gaze + pointing, and no cue (eyes closed).

Table 1. Psycholinguistic properties of the selected stimuli.

	Mean	SD	Min	Max
Frequency	4.334	0.470	3.318	5.210
Age of acquisition	2.553	0.542	1.529	3.382
Familiarity	6.199	0.412	5.286	6.783
Imageability	6.553	0.237	5.971	6.933
Concreteness	6.542	0.231	5.912	6.903

Figure 2. Trial procedure that introduced an instructor with a straight gaze (0.5 s), followed by a target appearing in one of two positions on the lateral plane along with the instructor’s cue (depending on condition) and audio naming the item and its translation to L1.

Table 2. Mean percentage correct scores, SDs, and SEs on each of the four conditions for both production and recognition tasks in the first experiment.

	Production task			Recognition task
Condition	Mean	SD	SE	Mean	SD	SE
No cue	36.9%	0.483	0.015	75.1%	0.433	0.013
Gaze only	33.2%	0.471	0.015	74%	0.439	0.013
Pointing only	35.7%	0.479	0.015	74.3%	0.437	0.013
Gaze + pointing	36.4%	0.481	0.015	74.5%	0.436	0.013

Figure 3. Images in four modified conditions for Experiment 2: gaze only, pointing only, gaze + pointing, and no cue.

Table 3. Mean percentage scores, SDs, and SEs on each of the four conditions for both production and recognition tasks in the second experiment.

	Production task			Recognition task
Condition	Mean	SD	SE	Mean	SD	SE
No cue	30.8%	0.013	0.014	73.4%	0.442	0.013
Gaze only	34%	0.474	0.014	72%	0.449	0.013
Pointing only	34.4%	0.475	0.014	75.5%	0.431	0.013
Gaze + pointing	35.9%	0.48	0.014	71.4%	0.452	0.013

Table 4. Summary of the generalized linear mixed-effects model for word learning outcomes measured by the recognition test (Experiment 2).

	Estimate	SE	Z value	p value	Lower CI (2.5%)	Upper CI (97.5%)
Fixed effects
Intercept	1.570	0.190	8.268	<.001	1.199	1.957
Gaze	− 0.330	0.106	− 3.106	.002	− 0.542	− 0.119
Pointing	− 0.081	0.090	− 0.886	.376	− 0.259	0.099
Gaze × pointing	0.01	0.181	0.055	.956	− 0.351	0.372
	Variance	SD
Random effects
ID	2.633	1.623
Item	0.483	0.695
	R^2 marginal	R^2 conditional
Model fit
	0.005	0.489

Model equation: model_r <- glmer(score ~ gaze × pointing + (1|id) + (1|item), data = total_recognition2, family = binomial()).

Figure 4. New (distal) position of the items combined with the incongruent and absent cues.

Table 5. Mean percentage scores, SDs, and SEs on each of the six conditions for both production and recognition tasks in the third experiment.

Condition		Production task			Recognition task
Cue type	Position	Mean	SD	SE	Mean	SD	SE
Absent	Within	32.8%	0.470	0.016	74.4%	0.437	0.015
Congruent	Outside	30.2%	0.460	0.016	75.6%	0.430	0.015
Incongruent	Within	33.4%	0.472	0.016	74.9%	0.434	0.015
Absent	Outside	30.1%	0.459	0.016	71.5%	0.452	0.015
Congruent	Within	34.6%	0.476	0.016	75.9%	0.428	0.015
Incongruent	Outside	28.9%	0.453	0.015	62.1%	0.485	0.017

Table 6. Summary of the generalized linear mixed-effects model for word learning outcomes measured by the recognition test (Experiment 3).

	Estimate	SE	Z value	p value	Lower CI (2.5%)	Upper CI (97.5%)
Fixed effects
Intercept	1.441	0.234	6.165	<.001	0.983	1.899
Cue Type (congruent)	0.184	0.115	1.610	.107	− 0.040	0.409
Cue Type (incongruent)	0.143	0.122	1.171	.242	− 0.096	0.382
Position (outside)	0.330	0.127	2.602	.009	0.081	0.579
Cue_type×Position (Congruent×Outside)	0.209	0.269	0.777	.437	− 0.318	0.735
Cue_type×Position (Incongruent×Outside)	0.156	0.237	0.667	.511	− 0.308	0.619
	Variance	SD
Random effects
ID	2.44	1.56
Item	1.27	1.13
	R^2 marginal	R^2 conditional
Model fit
	0.005	0.532

Model equation: model_r <- glmer(score ~ cue_type × position + (1|id) + (1|item), data = total_recognition3, family = binomial()).

Table

Wordlist A (4 letters, 2 syllables)	Wordlist B (5 letters, 2 syllables)	Wordlist C (6 letters, 3 syllables)	Wordlist D (7 letters, 3 syllables
saro	bueda	neveto	bafioro
bava	tromo	dekido	tavilco
abes	firpa	isalto	nerbona
ibre	yarza	conveo	deroxte
laga	conzo	tovasa	fesiano
izmo	milbo	seanos	almieso
poad	sigre	temiga	parensa
bipa	resel	pamatu	acrivar
luvo	gimar	civeno	sovosta
azme	zindo	revato	inzagio

Table

Word	Category
Egg	Food
Bread	Food
Muffin	Food
Mushroom	Food
Orange	Food
Apple	Food
Lemon	Food
Banana	Food
Pear	Food
Potato	Food
Pepper	Food
Onion	Food
Tomato	Food
Carrot	Food
Corn	Food
Lettuce	Food
Book	Household_items
Tape	Household_items
Ruler	Household_items
Notebook	Household_items
Box	Household_items
Clock	Household_items
Vase	Household_items
Battery	Household_items
Candle	Household_items
Cup	Household_items
Glass	Household_items
Plate	Household_items
Bowl	Household_items
Bottle	Household_items
Tray	Household_items
Envelope	Household_items

Table

Summary of the generalized linear mixed-effects model for word learning outcomes measured by the production test (Experiment 1)
	Estimate	SE	Z value	p value	Lower CI (2.5%)	Upper CI (97.5%)
Fixed effects
Intercept	− 0.979	0.21	− 4.658	<.001	− 1.392	− 0.567
Gaze	− 0.079	0.077	− 1.022	.307	− 0.233	0.072
Pointing	0.077	0.077	0.999	.318	− 0.074	0.228
Gaze × pointing	0.257	0.154	1.663	.096	− 0.046	0.559
	Variance	SD
Random effects
ID	3.524	1.877
Item	0.563	0.750
Fixed effects
Intercept	1.532	0.174	8.822	<.001	1.191	1.872
Gaze	− 0.039	0.077	− 0.500	.617	− 0.190	0.113
Pointing	− 0.036	0.077	− 0.470	.638	− 0.188	0.115
Gaze × pointing	0.159	0.154	1.028	.304	− 0.144	0.461
Random effects
ID	1.835	1.355
Item	0.48	0.693

Model equation: model_p <- glmer(score ~ gaze × pointing + (1|id) + (1|item), data = total_production1, family = binomial()).

Model equation: model_r <- glmer(score ~ gaze × pointing + (1|id) + (1|item), data = total_recognition1, family = binomial()).

Table

Summary of the generalized linear mixed-effects model for word learning outcomes measured by the production test (Experiment 2)
	Estimate	SE	Z value	p value	Lower CI (2.5%)	Upper CI (97.5%)
Fixed effects
Intercept	−1.106	0.200	−5.534	<.001	−1.505	−0.713
Gaze	0.142	0.107	1.325	.185	−0.071	0.355
Pointing	0.086	0.085	1.006	.314	−0.084	0.256
Gaze × pointing	−0.142	0.17	−0.834	.405	−0.482	0.197
	Variance	SD
Random effects
ID	2.869	1.694
Item	0.577	0.759

Model equation: model_p <- glmer(score ~ gaze × pointing + (1|id) + (1|item), data = total_production2, family = binomial()).