Expanding the Interpretive Potential of Eye Tracking for the Rorschach: Replication and Extension of the Findings of Ales et al. (2020)

Abstract

The present study was undertaken to determine if the findings of Ales et al. (2020) could be replicated and extended, especially in light of the replication crisis in psychology and the social sciences. Ales et al. (2020) found that measures from the engagement and cognitive processing domain of the Rorschach performance assessment system (R-PAS) were associated with Eye Tracking variables that reflect cognitive engagement and effort. Notably, Complexity was associated with the number of fixations participants made while scanning the blots and Vg% was inversely associated with a participant’s average fixation durations. The present study utilized a non-clinical sample of 60 adult participants. The basic findings of Ales et al. (2020) were replicated. In addition, we found that Complexity and Vg% are associated with additional Eye Tracking variables not utilized in the original study. The current findings bolster and extend the interpretation of Ales et al. (2020), indicating that higher levels of Complexity are also associated with scanning more regions of the blot overall, albeit at a slower rate. Similar effect size values were observed in the two different cultural contexts. Higher levels of Vg% are associated with measures indicating shallower and more superficial search strategies, consistent with the interpretation of Vg% as indicative of a vague, impressionistic, and unsophisticated cognitive style.

Given the centrality of visual scanning to the Rorschach response process, Eye-Tracking technology holds promise as a tool for improving our understanding of processing strategies involved in producing responses. To date, Eye-Tracking methods have been used to study various aspects of the Rorschach, including typical scanning patterns across the inkblots (Baughman, 1954; Blake, 1948; Dauphin & Greene, 2012; Thomas, 1963), the scanning patterns of individuals diagnosed with schizophrenia (Hori et al., 2002; Minassian et al., 2005), vertical visual field asymmetry (Greene et al., 2014) and personality characteristics (Ryan et al., 1971).

Recently, (Ales et al., 2020) discovered an association between the R-PAS (Meyer et al., 2011) measure of Complexity (from the Engagement and Cognitive Processing Domain) and eye movement measures of engagement and effort, namely, the number of fixations utilized by participants as they searched the Rorschach blots to produce responses. Ales et al. (2020) noted that the average number of fixations utilized throughout the Rorschach relate to the level of engagement a person has with a visual stimulus on average, with more fixations indicating greater engagement and possibly effort in processing the stimulus. Fixations reflect the allocation of attention to some aspect of the visual field (Rayner, 2009), and scanning strategies can be affected by type of stimulus and task (Castelhano et al., 2009; Castelhano & Henderson, 2008). Not only did Ales et al. (2020) find that Complexity was associated with the number of fixations utilized, but Complexity explained variance associated with fixations above and beyond the other variables from the Engagement and Cognitive Processing Domain. Their results provided psychophysiological support for the interpretation of Complexity as a measure of engagement and possibly effort and showed the value of utilizing Eye-Tracking in the study of the Rorschach. Interestingly, additional psychophysiological evidence for Complexity as an indicator of attention and effort has recently emerged from Vitolo et al. (2021), who found that the level of Complexity on the Rorschach was associated with increased activity in the brain’s dorsal attentional network.

Second, they showed an association between Vg% (percent of responses with a coding of Vague) and fixation duration (namely, the length of time a person’s eye fixates to a given location on a visual stimulus), with higher levels of Vg% being associated with shorter fixation durations. The shorter the fixation duration, the less time a person is using to allocate attention to areas of visual space to process the stimulus. Studies in the Eye-Tracking literature indicate that longer fixation durations tend to occur when individuals are engaged in processing more complex stimuli or when the area is providing a great deal of information relevant to the task (Rayner, 2009). Shorter fixation durations tend to occur when a task or stimulus is not complex or when an individual displays less engagement or superficial engagement with the stimulus. As Ales et al. (2020) noted, in the R-PAS system Vg% is interpreted to be related to vague, impressionistic, and unsophisticated processing (Meyer et al., 2011). In the context of the Rorschach, as a visual processing task, this suggests more superficial processing of the blots which corresponds to the R-PAS psychological interpretation of Vg% as indicative of a superficial processing and possibly associated with an avoidant style.

The present study seeks to determine if the results of Ales et al. (2020) can be replicated and extended. Because the replication crisis in psychology and the social sciences (Open Science Collaboration, 2015) has raised questions about the stability of many research findings, it is incumbent upon researchers to undertake replication studies despite the traditional aversion that such studies do not contribute anything new to the field. As such we hypothesize Complexity to be associated with the number of fixations (Ales et al., 2020). However, the number of fixations does not, by itself, indicate whether the individual’s attention was spread across many regions of the blot or concentrated on fewer spatial areas. We include indices of the degree to which visual attention is spread over more or less of the visual field. Furthermore, to clarify the extent of the engagement, we hypothesize an association of Complexity with Unique Regions Visited and Unique Regions Visited per Fixations, as these indices reflect relative levels of stimulus engagement as measures of the dispersal of visual attention across the stimulus. We hypothesize inverse relationships between Complexity and Fixations per second and Unique Regions Visited per second, as more engagement would most likely be reflected in more time consuming processing. In other words, covering different areas of the blot more slowly would be consistent with attempts to integrate information from different spatial regions, whereas moving rapidly across the blot regions would be more consistent with a superficial scanning process in a task like responding to the Rorschach. Second, we expect that Vg% would be inversely correlated with Fixation Duration (as in Ales et al., 2020) and hypothesize a relationship with Unique Regions Visited and Unique Regions Visited per second. That would reflect less processing time over a wider area of the stimulus field. For a task such as the Rorschach, the combination would indicate shallower or more superficial processing of the blots in general. Assuming Vg% to indicate shallow or superficial processing, the combination of shorter fixations durations with more Unique Regions Visited and/or Unique Regions Visited per second would expand the interpretation from Ales et al. (2020).

Method

Participants

Sixty individuals (N = 42 females) from the undergraduate participant pool of a Midwestern University and from the surrounding community participated in this study (age range 18 to 60 years, mean = 25.73, S.D. = 8.72). All individuals had normal or corrected-to-normal visual acuity. Color blindness was an exclusion criterion for this study. Participants were paid $30 for their time in the laboratory (approximately 2-2 1/2 h). The study was approved by the Institutional Review Board (Protocol #1314-41) of a midwestern university and was conducted in accordance with the Belmont Principles, United States (1978).

Stimuli & apparatus

The standard 10 Rorschach inkblots (Rorschach, 1921) were digitized for presentation on a 17-inch color monitor. Eye movement characteristics were recorded using an Eyelink-II Eye-Tracking system. The apparatus uses head-mounted video cameras angled to sample pupil position at 500 Hz, as a participant engages in visual tasks. An eye movement was detected when eye velocity exceeded 30° s − 1, or when eye acceleration exceeded 8000° s − 2. After a 9-point calibration of the tracker, the accuracy of reported gaze position on the Rorschach images was 0.5° −1.0° of visual angle. Accuracy was not hindered by minor head movements less than ±15° of visual angle (e.g., from uttering verbal responses during the administration of the Rorschach test). The eye tracker was controlled by EYETRACK software (see http://www.umass.edu/psychology/div2/eyelab/).

Procedure

After completing a demographic questionnaire, participants completed a set of self-report measures (not in the scope of the present study). Next participants completed a performance-based task, the subtest Symbol Search of the WAIS-IV, as a measure of Processing Speed. Ales et al. (2020) noted the potential usefulness of including some measure of cognitive processing speed. Following this, participants were taken to the Eye-Tracking station in the laboratory. The head-mounted Eyelink II eye-tracking cameras were fitted such that each research participant was comfortable during extended use. Participants sat 55 cm from the monitor such that Rorschach images subtended visual angles of approximately 6° by 6°. Viewing was binocular, but for convenience, recording was monocular, measuring only right eye movements¹. The Eyelink-II software reported direct measurements including fixation coordinates, fixation duration, and saccadic amplitude.

R-PAS administration of the Rorschach (Meyer et al., 2011) was utilized with minor modifications to account for viewing the images on a computer screen. Modified R-PAS instructions were: Okay, now we are ready to start. I will show the inkblots to you one at a time. Your task is to look at each inkblot to answer the question ‘What might this be?’ Does that make sense? YES: Good, we can get started then. Try to give two responses…or maybe three, to each inkblot. That is, for each inkblot try to see two different things; possibly three. Let me know when you have finished responding, so we can go to the next inkblot. (DISPLAY CARD 1). What might this be?

The images were presented in sequential order (Cards I-X), and a trial was terminated when the participant indicated that he/she was done. The Clarification phase was completed with the physical cards, according to the standardized R-PAS procedure, after removal of the eye tracker. Rorschach verbal responses were coded and scored according to R-PAS criteria (Meyer et al., 2011). Each participant’s responses were coded by the first and third authors to consensus. A subset of 14 protocols was coded by the 6^th author to establish inter-rater reliability with the consensus coded protocols. Intraclass correlation coefficients (ICCs) ranged from .5 for V and FD to 1.0 for R and R8910% with a mean ICC of .84 (SD=.14). One variable (C) had a score of 0 for both sets of protocols, so there was no variance resulting in no ICC calculated (though 100% agreement between protocols). Relevant for the emphasis on replication and extension of main Ales et al. (2020) findings are that Complexity had an ICC of .95 and Vg% had an ICC of .92. All Page 1 variables showed ICCs in the excellent range, >.75 (Cicchetti, 1994; Shrout & Fleiss, 1979). Page 2 variables showed good-to-excellent ICCs for 8 of the 11 variables, fair for two variables (FD and V) and not able to be calculated for one variable.

Measures

In order to understand how the Eye-Tracking indices relate to visual processing strategies, we provide basic definitions for indices not utilized in Ales et al. (2020). For more detailed information about Number of Fixations, Fixation Duration, and Saccade Amplitude, see Ales et al. (2020, pr. 541-542).

Unique regions visited

The Number of Unique Regions Visited is an index that reflects the degree to which visual attention is relatively more or less dispersed across the stimulus field. Unique Regions Visited is a method of dividing the stimulus field into non-overlapping regions of equal size. The present study divided the stimulus field (the inked regions plus white area around the inked regions, the equivalent of a Rorschach card) into 12 × 16 regions of equal size, we tracked the number of regions in which an individual produced at least one fixation. The more regions visited during the processing of the blots, the more dispersed visual attention is, in general.

Unique regions visited per number of fixations

In a similar vein, the number of Unique Regions Visited per the number of fixations reflects the degree of attention dispersal per number of fixations (expressed as a ratio URV/NF). The higher the ratio the more dispersed visual attention is across the visual stimulus taking the Number of fixations into consideration. The lower the number, the more focused visual attention is to fewer regions of the stimulus field per the number of fixations. For example, even if individuals use a high number of fixations, their visual attention might be highly focused into very few regions of the stimulus, suggestive of more engagement and possibly effort onto a specific Rorschach location (for example, a D area) as opposed to more extensive processing engagement across the whole blot. It is possible, for example, for a person to invest much attention and effort to a D area and then provide a response using that area. Using such data in the context of analyzing a response, we could wonder if the response displayed ordinary form quality or might have been a minus form quality response despite investing so much attention to the area. Either direction provides some potential interpretive utility. Likewise, it is possible for a person to invest a great deal of attention in an area, but then not use the area in the response. Ryan et al. (1971) showed the individuals who were rated as having more hysterical personality features, spent more time viewing the red areas on Cards II and III than individuals rated with lower levels of hysterical features. Yet, they utilized the red areas less frequently in their verbal responses.

Initial Saccade amplitude

Initial Saccade Amplitude is the length of the 1^st saccade. In general, it is expected to be highly influenced by the stimulus or reflect more stimulus-driven effects as opposed to reflective of top-down (goal-directed) in processing.

Number of fixations per second, unique regions visited per second and Initial saccade latency

Number of fixations per second and Unique regions visited per second reflect the rate of processing the visual stimulus. Higher rates of processing could be observed on tasks that are straightforward, such as finding a target stimulus among unambiguous distractors, and it may be higher if a person processes a more complex stimulus in a relatively superficial manner. Both can contribute to our understanding of the quality of the individual’s engagement with the stimulus while producing a response. Initial Saccade Latency is the time it takes for an individual to make the first saccade. Like Initial saccade amplitude, this is usually a stimulus driven variable.

Although there are general interpretations one can make about each of the Eye-Tracking indices, interpretation is mostly context dependent. That is, interpretation of Eye-Tracking indices is related to the kind of task the individual is undertaking, the level of familiarity with the visual stimuli, viewing conditions (e.g., clarity of visual images, lighting, etc.), the timeframe for viewing, etc. If two or more indices are relevant for interpretation of understanding processing strategies relevant to participant characteristics (e.g., Complexity scores, level of Vg%, etc.), then it is likely that we can develop a more specific or sophisticated understanding of the psychophysiological processes involved.

Results

In line with the procedure of Ales et al. (2020), we reviewed the descriptive statistics of the variables to determine if any departed substantially from normality (i. e., skewness > 2 and kurtosis > 7; West et al., 1995). Two R-PAS variables and one Eye-Tracking variable violated normality. Thus, we followed the same process as Ales et al. (2020) for variables that violated normality and applied a square root transformation to them. The Eye-Tracking variable Initial Saccade Latency showed a skewness of 2.04 and a kurtosis of 7.78. After square root transformation, Initial Saccade Latency had a skewness of 1.09 and a skewness of 3.27. The R-PAS variable V had a pre-transformation skewness of 2.57 and a kurtosis of 10.30, with a post square root transformation skewness of .506 and kurtosis of −0.571. C displayed a skewness of 2.74 and a kurtosis of 5.67. Skewness and kurtosis remained unchanged following transformation, as inspection of data showed all participants had a C of either 0 or 1, which leaves the variable unchanged with a square root transformation.

Table 1 shows the descriptive statistics for the R-PAS variables in the Engagement and Cognitive Processing domain. Table 2 shows descriptive statistics for the Eye-Tracking variables and the correlation matrix for the Eye-Tracking variables. Fifteen of the 21 R-PAS variables display standard scores between 95-105, suggesting that the adjusted administration procedures may have had a small impact on the Rorschach task in the present study (e.g., compared with 19 of 21 variables in the Ales et al. (2020) study) or the sample is somewhat different than a normative sample.

Table 1. Descriptive statistics of engagement and cognitive processing domain variables.

Raw Scores					Standard Scores
	M	SD	Skew	Kurtosis	M	SD	Skew	Kurtosis	ICCs
Page 1
Complexity	78.9	21.1	.33	−.51	103.2	13.4	−.13	−.49	.95
R	23.7	3.1	.81	.47	100.3	9.1	.25	.14	1.0
F%	36.0	16.6	.48	.28	94.8	15.0	−.06	.43	.92
Blend	5.7	3.2	.83	1.5	107.6	13.5	−.17	.66	.82
Sy	7.4	4.0	.37	−.62	102.8	14.7	.06	−.83	.90
MC	7.1	3.0	.55	.32	99.8	12.4	−.12	−.13	.93
MC - PPD	−3.8	4.3	−.76	1.0	95.2	13.1	−.28	−.11	.85
M	3.9	2.4	.68	.07	101.1	13.2	.03	−.26	.95
M - WSumC	.77	2.7	.27	−.15	2.32	16.36	.06	−.04	.88
(CF + C) – FC	1.7	1.8	.51	.18	112.87	17.15	.31	−.19	.87
Page 2
W%	37.77	15.78	.47	−.08	98.43	11.25	.245	−.33	.96
Dd%	15.60	9.10	.63	.49	100.92	12.27	−.082	−.21	.67
SI	2.15	1.81	1.29	2.49	94.98	14.40	.338	−.32	.87
IntCont	1.75	1.63	1.31	2.76	96.33	11.62	.228	−.38	.75
Vg%	1.09	2.16	1.69	1.36	89.37	6.61	1.578	.77	.92
V	1.00	1.38	.506	−.571	104.85	13.65	.608	−.56	.50
FD	3.88	1.86	.14	−.49	126.85	13.18	−.737	.81	.50
R8910	7.45	1.42	.66	−.63	98.55	9.92	.521	−.82	1.0
WSumC	3.15	1.70	.58	.67	98.77	11.43	−.104	.23	.83
C	.10	.30	2.74	5.67	96.90	5.75	2.74	5.67	*
Mp - Ma	−.50	2.31	.40	.78	.67	18.6	−.37	−.10	.72

Note: Values in bold refer to variables after square root transformation. Intraclass correlation coefficients (ICCs) were calculated based on raw scores. Variables that are underlined are proportion variables computed as subtraction. *All protocols for ICC contained C = 0.

Table 2. Descriptive statistics and correlation matrix for eye-tracking variables.

	M	SD	Skew	Kurtosis	1	2	3	4	5	6	7	8	9
1. NF	1300.79	610.21	.93	.33	1.00
2. FD (msec)	334.55	54.01	−.19	.44	−.04	1.00
3. URV	367.90	101.17	.92	1.64	.79**	−.15	1.00
4. SA (o visual angle)	2.58	.53	.03	−.80	−.10	−.03	.05	1.00
5. ISA (o visual angle)	1.91	.64	.49	−.43	−.13	−.02	−.06	.50**	1.00
6. ISL (msec)	279.29	73.94	2.04	7.78	.16	.00	.08	−.25	−.30*	1.00
7. URV/NF	.31	.08	.28	−.14	−.82**	.02	−.44**	.15	.11	−.20	1.00
8. NF/sec	2.54	.41	.25	.79	−.05	−.82**	−.13	−.11	−.05	−.06	−.07	1.00
9. URV/sec	.79	.24	.45	−.66	−.73**	−.47**	−.44**	.07	.06	−.22	.81**	.52**	1.00

Note: NF (Number of Fixations); FD (Fixation Duration); URV (Unique Regions Visited); SA (Saccade Amplitude); ISA (Initial Saccade Amplitude); URRV/NF (Unique Regions Visited per number of fixations); NF/sec (Number of fixations per second); URV/sec (Unique Regions Visited per second).

*p<.05; **p<.01.

Table 3 shows the correlations between Engagement and Cognitive Processing variables and Symbol Search with the nine Eye-Tracking measures used in the study. Complexity, Blend, Sy, MC and M show significant relationships to Number of Fixations of moderate effect size (Cohen, 1988), while IntCont shows a significant correlation but with a small effect size. The relationship between Complexity and Number of Fixations is the same as the central finding of Ales et al. (2020). The significant relationships between Number of Fixations and Blend, Sy, MC, M, and IntCont replicates Ales et al. (2020). In contrast to Ales et al. (2020), in our study F% and Si did not correlate significantly with the number of fixations. However, Ales et al. (2020) had N = 71 participants, whereas our study had N = 60. In using Fisher’s method to compare two correlations drawn from two independent samples, there is no significant difference between Ales et al. (2020) and the current study: for F%, z = .318, p = .375 and for Si, z = .903, p = .183.

Table 3. Correlations of Rorschach performance assessment system engagement and cognitive processing variables, symbol search, and eye tracking variables.

	NF	FD	URV	SA	ISA	ISL	URV/NF	NF/sec	URV/sec
Page 1
Complexity	.475**	−.060	.435**	.013	.132	−.156	−.463**	.084	−.368**
R	.211	−.269*	.350**	−.053	−.072	−.180	−.090	.190	.029
F%	−.207	.027	−.129	−.102	−.117	.112	.290*	−.085	.208
Blend	.355**	−.003	.291*	.096	.138	−.154	−.327*	−.020	−.300*
Sy	.405**	.156	.329*	−.019	.195	−.106	−.444**	−.036	−.423**
MC	.305*	−.092	.231	.010	.127	−.132	−.373**	.105	−.284*
MC – PPD	−.244	.079	−.337**	−.159	−.127	.143	.031	−.022	−.008
M	.340**	−.010	.219	−.105	.068	−.071	−.384**	.116	−.287*
M – WsumC	.243	.087	.119	−.193	−.027	.026	−.242	.081	−.175
(CF + C) – FC	.026	−.217	.142	.227	.073	−.085	.061	.002	.044
Page 2
W%	−.020	.071	−.089	−.093	.014	−.001	−.147	−.091	−.200
Dd%	.238	.209	.171	.164	.025	−.194	−.190	−.102	−.241
SI	.094	.187	.006	−.142	−.110	−.162	−.230	−.020	−.209
IntCont	.260*	−.105	.174	−.084	−.012	.242	−.299*	.160	−.174
Vg%	−.151	−.259*	−.052	.110	.143	−.136	.219	.171	.299*
V	−.021	−.146	−.065	.047	.285*	−.107	−.037	.145	.064
FD	.059	−.020	−.014	.017	−.033	.043	−.160	−.029	−.167
R8910%	−.176	.112	−.142	.171	−.119	−.085	.109	−.043	.096
WsumC	.082	−.153	.114	.164	.137	−.140	−.144	.031	−.117
C	−.109	−.112	−.066	−.001	.093	−.184	.123	−.111	.017
Mp – Ma	−.056	−.018	−.141	.022	−.233	.230	.010	.088	.075
Symbol Search from WAIS-IV
SS	−.069	−.165	−.036	.028	.055	−.129	.034	.072	.096

*p < .05, **p < .01.

None of the Engagement and Cognitive Processing variables that showed statistically significant relationships with Number of Fixations had statistically significant relationships with Fixation Duration or Saccade Amplitude in either Ales et al. (2020) or the present study. In the present study, additional relationships of moderate effect size are revealed between Complexity and Unique Regions Visited, Unique Regions Visited per Number of Fixations and Unique Regions Visited per second, variables not studied in Ales et al. (2020). Ales et al. (2020) indicated the potential value of including a measure of cognitive processing speed. We utilized Symbol Search, which was not significantly related to any of the Eye-Tracking variables, nor related to Complexity (r=.046, ns) or Vg% (r=.167, ns). For a direct comparison of the correlations between R-PAS variables and Eye-Tracking variables from Ales et al. (2020) and the present study, see the Online Supplemental Material.

As the central question in Ales et al. (2020) involved the role of Complexity, we followed their data analytic strategy of determining whether Complexity explained the variance of the Number of Fixations beyond the effects of the other significant variables in the Engagement and Cognitive Processing Domain via a series of hierarchical regressions. We extended this strategy to the other Eye-Tracking variables that had significant relationships with Complexity (Unique Regions Visited, Unique Regions Visited per Number of Fixations, and Unique Regions Visited per second). These analyses can be seen in Tables 4. For Number of Fixations, when Complexity is entered in the first step, none of the other variables entered in Step 2 significantly improve the model. Furthermore, when Complexity is entered in Step 2, it improves the model in all cases, the same finding as Ales et al. (2020).

Table 4. Hierarchical multiple regressions predicting eye-tracking variables.

NF
Model		β		Adj R^2
Step1	Step 2	M1	M2^a	M1	M2	ΔR^2
Complexity	Blend	.475**	.544**/−.085	.212	.201	.003
Complexity	Sy	.475**	.525**/−.057	.212	.199	.001
Complexity	MC	.475**	.680**/−.251	.212	.220	.021
Complexity	M	.475**	.473**/.003	.212	.199	.000
Complexity	InCont	.475**	.440**/.093	.212	.206	.007
Blend	Complexity	.355**	.085/.544**	.111	.201	.102**
Sy	Complexity	.405**	−.057/.525**	.150	.199	.062*
MC	Complexity	.305*	−.251/680**	.078	.220	.153**
M	Complexity	.340**	.003/.473**	.100	.199	.110**
InCont	Complexity	.260*	.093/.440**	.052	.206	.165**
URV
Complexity	R	.435**	.350**/−.202	.175	.195	.033
Complexity	Blend	.435**	.578**/−.177	.175	.172	.011
Complexity	Sy	.435**	.641*/−.235	.175	.174	.012
Complexity	MC−PPD	.435**	359**/−203	.175	.197	.035
R	Complexity	.350**	.202/.350**	.107	.195	.100**
Blend	Complexity	.291*	−.177/.578**	.069	.172	.115**
Sy	Complexity	.329*	−.235/.641*	.093	.174	.093*
MC−PPD	Complexity	−.337**	−.203/.359**	.098	.197	.111**
URV/NF
Complexity	Blend	−.463**	−.576**/.139	.201	.194	.007
Complexity	F%	−.463**	−.549**/−.116	.201	.193	.006
Complexity	Sy	−.463**	−.320/−.163	.201	.193	.006
Complexity	MC	−.463**	−.478*/.018	.201	.187	.000
Complexity	M	−.463**	−.385*/−.109	.201	.193	.006
Complexity	InCont	−.463**	−.408**/−.144	.201	.205	.018
Blend	Complexity	−.327*	.139/−.576**	.092	.194	.114**
F%	Complexity	.250*	−.116/−.549	.069	.193	.136**
Sy	Complexity	−.444**	−.163/−.320	.183	.193	.023
MC	Complexity	−.373**	.018/−.478**	.124	.215	.076*
M	Complexity	−.384**	−.109/−.385*	.132	.193	.073*
InCont	Complexity	−.299*	−.144/−.408**	.074	.205	.143**
URV/sec
Complexity	Blend	−.368**	−.363/−.007	.121	.105	.000
Complexity	Sy	−.368**	.019/-.440	.121	.151	.044
Complexity	MC	−.368**	−410/.051	.121	.106	.001
Complexity	M	−.368**	−.332/−.050	.121	.106	.001
Complexity	Vg%	−.368**	−358**/.286*	.121	.190	.082*
Blend	Complexity	−.300*	−.007/−.363	.074	.105	.045
Sy	Complexity	−.423**	−.440/.019	.165	.151	.000
MC	Complexity	−.284*	.051/−.410	.065	.106	.056
M	Complexity	−.287*	−.050/−.332	.066	.106	.055
Vg%	Complexity	.299*	.286*/−.358**	.074	.190	.128**

Note: M 1= Model 1; M2 = Model 2.

^aValues to the left of the slash refer to variables entered at Step 1; values on the right of the slash refer to the variable entered at Step 2.

*p<.05; **p<.01.

R-PAS variables significant for Unique Regions Visited include R, Blend, Sy, and MC-PPD. When Complexity is entered in the first step, none of these variables entered in Step 2 significantly improve the model. When Complexity is entered in Step 2, it improves the model in all cases. R-PAS variables significant for Unique Regions Visited per Number of Fixations include Blend, F%, Sy, MC, M, and IntCont. When Complexity is entered in the first step, none of the variables entered in Step 2 significantly improve the model. When Complexity is entered in Step 2, it improves the model for all cases except Sy. R-PAS variables significant for Unique Regions Visited per second include Blend, Sy, MC, M, and Vg%. When Complexity is entered in the first step, only Vg% improves the model in Step 2. When Complexity is entered in Step 2, it only improves the model for Vg%.

The only variables that showed significant relationships with Fixation Duration were R and Vg% with small effect sizes. Vg% was also significantly correlated with Fixation Duration in Ales et al. (2020), so their finding on this variable was replicated in the present study of approximately the same magnitude (r = −0.266 in Ales et al., 2020 and r = −0.259 in the present study). Vg% is also significant with Unique Regions Visited per second with a small effect size, while R is not. None of the Cognitive and Engagement variables exhibited a significant relationship with Saccade Amplitude, which is the same as Ales et al. (2020).

Discussion

The present study provides strong support for the major findings of Ales et al. (2020) of higher levels of Complexity showing clear evidence of greater scanning/exploratory activity in the form of higher Number of Fixations while scanning the blots. The present study expands the interpretation and helps provide greater specificity to the association of Complexity to processing strategies. Higher levels of Complexity are also associated with scanning more regions of the blot, generally concentrating more of their fixations into fewer areas, and scanning blot regions at a slower rate. The higher the Complexity score the more psychological activity and effort a person is presumed to put into coping with the demands of the Rorschach (Meyer et al., 2011). The higher the Complexity score, the more methodical and effortful is the search strategy by participants in the present study. The conjunction of several Eye-Tracking indices helps us understand how the processing strategies vary according to levels of Complexity, with participants’ Complexity levels associated to variations in search/scanning, which reflects variations in the attention paid to the blots. The findings provide further evidence to interpreting Complexity as a measure of cognitive engagement and effort (Meyer et al., 2011). In addition, a measure of processing speed (Symbol Search) was not associated with any Eye-Tracking variables or Complexity, so that the relationship between Complexity and Eye-Tracking does not appear to be an artifact of levels of processing speed abilities.

The present study extends the findings of Ales et al. (2020) by noting that not only is higher Complexity associated with more fixations but also with spreading attention of more areas of the blot. As information can only be processed during fixations (Rayner, 2009), we see that higher Complexity indicates attempts to process information from a wider span of locations than lower Complexity. The fact that Complexity is also associated with covering those regions more slowly and concentrating relatively more of their fixations on fewer regions suggests greater attempts to incorporate more aspects of the blot into responses. It is a style reflecting both a wider span of attention to the blot overall, while engaging in more concentrated attention to fewer of the areas. In other words, individuals with higher Complexity scores allocate attention to larger regions of the blot during the course of a response process, but do not appear to do so in a casual or superficial manner. Instead, they devote the bulk of their attention to fewer regions at some point during the process. This appears to indicate some degree of sophistication in approach, in that the greater number of fixations do not appear to reflect a haphazard processing style, as might be the case if many fixations were spread across many regions of the blot with little-to-no focus of attention allocation.

The present study also replicated and extended the finding of Ales et al. (2020) of a relationship between Vg% and Fixation Duration. Both studies found that the higher the Vg%, the shorter the average Fixation Durations during the scanning process. Shorter Fixation Durations indicate less time spent picking up information during fixations. This process is congruent with the interpretation of Vg% as indicative of a vague, impressionistic, and unsophisticated cognitive style (Meyer et al., 2011). In addition, the present study showed Vg% to be positively associated with Unique Regions Visited per second, which indicates scanning over more of the blot more rapidly. This indicates that the general search strategy associated with higher Vg% is one consistent with a shallower form of processing in which processing proceeds covering different regions of the blot more rapidly but spending less time during fixations picking up information for processing. The finding of more Unique Regions Visited per second the higher Vg% helps strengthen the interpretation of Vg% as indicative of an impressionistic cognitive style. It would be more challenging to regularly create responses with clear form if one is regularly scanning the blots in this manner.

The replication and extension of the findings of Ales et al. (2020) is noteworthy because the current study was conducted in a different country (US vs. Italy), the Rorschach was administered in a different language (English vs. Italian), and that almost the same effect size values were observed in the two different cultural contexts. Thus, the current study provides cross-cultural validation of the original findings. Furthermore, the current study suggests that the central findings of Ales et al. (2020) are robust and not dependent upon idiosyncrasies of their procedure or local contextual factors.

Like all empirical studies the present study has limitations. In terms of ecological validity, the blots were displayed on a computer screen, while participants wore a head mounted eye tracker. This is different from a typical administration. Also, the method did not enable turning of the blots as occurs sometimes during administration.

The sample was a student and community sample and not a clinical sample, so that one cannot automatically generalize the findings of the present study to more pathological samples. Over time, it would be valuable to undertake Eye-Tracking studies with clinical populations in addition to expanding its use with non-clinical populations in order to maximize the interpretive potential of this technology for clinical use. Fifteen of the standard scores from the Engagement and Cognitive Processing Domain were within average limits in the present study while 6 were outside of average limits, suggesting a potential influence that current procedures might have had on performance. Ales et al. (2020) recommended the use of measure of Cognitive Processing Speed such as a computerized tool, such as the Vienna Testing Systems or other measures. We utilized a highly reliable and well validated measure from the WAIS-IV, Symbol Search, showing that processing speed was not associated with Complexity or any of the Eye-Tracking variables.

Overall, the present study contributes to the ongoing effort of establishing validity for the R-PAS through illustrating the association of extra-test measures with pertinent R-PAS variables. Replication is essential considering the difficulty shown in replicating many mainstream psychological findings, as it can serve as a guard against false positive results (Open Science Collaboration, 2015). The present study concentrated on Complexity and Vg%, although correlations with Eye-Tracking variables were found for other R-PAS measures in the Engagement and Cognitive Processing Domain, which could provide some interpretive value to them.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data from the study will be made available upon reasonable request.

Notes

1 Binocular recording of eye movements is only of concern during some reading studies where very fine spatial resolution is of importance (e.g., Liversedge et al., 2006), and do not outweigh the set-up convenience and cost-savings to researchers or the comfort of participants offered by monocular recording of eye movements.