Evaluating untimed and timed abridged versions of Raven’s Advanced Progressive Matrices

ABSTRACT

Introduction

Raven’s Advanced Progressive Matrices (APM) are frequently utilized in clinical and experimental settings to index intellectual capacity. As the APM is a relatively long assessment, abridged versions of the test have been proposed. The psychometric properties of an untimed 12-item APM have received some consideration in the literature, but validity explorations have been limited. Moreover, both reliability and validity of a timed 12-item APM have not previously been examined.

Method

We considered the psychometric properties of untimed (Study 1; N = 608; M_age = 27.89, SD = 11.68) and timed (Study 2; N = 479; M_age = 20.93, SD = 3.12) versions of a brief online 12-item form of the APM.

Results

Confirmatory factor analyses established both versions of the tests are unidimensional. Item response theory analyses revealed that, in each case, the 12 items are characterized by distinct differences in difficulty, discrimination, and guessing. Differential item functioning showed few male/female or native English/non-native English performance differences. Test–retest reliability was .65 (Study 1) to .69 (Study 2). Both tests had medium-to-large correlations with the Wechsler Abbreviated Scale of Intelligence (2nd ed.) Perceptual Reasoning Index (r = .50, Study 1; r = .56, Study 2) and Full-Scale IQ (r = .34, Study 1; r = .41, Study 2).

Conclusion

In sum, results suggest both untimed and timed online versions of the brief APM are psychometrically sound. As test duration was found to be highly variable for the untimed version, the timed form might be a more suitable choice when it is likely to form part of a longer battery of tests. Nonetheless, classical test and item response theory analyses, plus validity considerations, suggest the untimed version might be the superior abridged form.

KEYWORDS:

• Raven’s advanced progressive matrices
• online administration
• Wechsler intelligence scales
• validity
• classical test theory
• item response theory

Acknowledgments

The authors wish to thank Cameron Patrick from the Melbourne Statistical Consulting Platform for advice conducting analyses.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data and Materials Availability

The datasets analyzed for the current study are available in the Open Science Framework repository (https://osf.io/ry23m/). The study was not pre-registered.

Permissions

A Pearson Research License Agreement is in place to enable the use of the Raven’s Progressive Matrices for research purposes.

Ethics approval

This study was performed in line with the principles of the Declaration of Helsinki. The University of Melbourne Human Ethics Committee approved this study (ID: 1,544,791).

Consent to participate/ Consent for publication

All participants provided informed consent. In doing so, they acknowledged reading a Plain Language Statement that explained aggregated group level data from this study may be published or presented at conferences.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/13803395.2022.2080185

Additional information

Funding

This research was supported by an Australian National Health and Medical Research Council grant (1050766) and an Australian Research Council fellowship (FT110100088).