Neural Activity Associated with Visual Search for Line Drawings on AAC Displays: An Exploration of the Use of fMRI

ABSTRACT

Visual aided augmentative and alternative communication (AAC) consists of books or technologies that contain visual symbols to supplement spoken language. A common observation concerning some forms of aided AAC is that message preparation can be frustratingly slow. We explored the uses of fMRI to examine the neural correlates of visual search for line drawings on AAC displays in 18 college students under two experimental conditions. Under one condition, the location of the icons remained stable and participants were able to learn the spatial layout of the display. Under the other condition, constant shuffling of the locations of the icons prevented participants from learning the layout, impeding rapid search. Brain activation was contrasted under these conditions. Rapid search in the stable display was associated with greater activation of cortical and subcortical regions associated with memory, motor learning, and dorsal visual pathways compared to the search in the unpredictable display. Rapid search for line drawings on stable AAC displays involves not just the conceptual knowledge of the symbol meaning but also the integration of motor, memory, and visual-spatial knowledge about the display layout. Further research must study individuals who use AAC, as well as the functional effect of interventions that promote knowledge about array layout.

Keywords:

• Brain imaging
• AAC system design
• neural activity

Acknowledgements

Various subcomponents of this study served as a doctoral dissertation for the fourth author, as Master’s projects for the final four authors, a Master’s project for Miranda Purdy, and an Honor’s Project for Molly Lichtenwalner, under the supervision of the first author.

This research was supported by a Level 2 Award from the Social Science Research Institute (SSRI) at the Pennsylvania State University. The authors wish to thank the Penn State Social, Life & Engineering Sciences Imaging Center (SLEIC) 3T MRI facility for their support. The first author is co-funded by the SSRI at the Pennsylvania State University, and two of the authors (Farmer, Leach) were supported by USDE grant H325K110315 (Light, PI).

We thank the willing volunteers for their time, Carolyn Bell for her contributions, and Rick Gilmore for his aid in conceptualizing the design.

Declaration of interest

The authors report no conflicts of interests. The authors alone are responsible for the content and writing of this paper.

Notes

1. Contraindications to fMRI include, but are not limited to, implanted metal, cochlear implants, pace makers, heart stents, ferromagnetic ink in tattoos, pregnancy, aneurysm clips in the brain, claustrophobia, and non-removable body piercings.

2. In event-related fMRI designs, each trial must be modeled and the corresponding neural activation accounted for in order to accurately model the variance in the model. Regressors of no interest account for variance that is not associated with events of interest in the primary analysis.

3. Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States. https://www.google.com/imghp?hl = en&tab = wi&ei = CTbfVYuQFoHA-AGZ56PAAw&ved = 0CBUQqi4oAQ

4. The Picture Communication Symbols ©1981–2011 by Mayer-Johnson LLC. All rights reserved. Used with Permission. Boardmaker is a trademark of Mayer-Johnson LLC. http://www.mayer-johnson.com/boardmaker-software/

5. The Siemens 3 T Magnetom Trio MRI scanner is a product of Siemens Medical Solutions USA, Inc., 40 Liberty Boulevard, Malvern, PA 19355, USA; http://www.usa.siemens.com

6. In accord with standard realignment and normalization procedures in SPM8, we used both rigid body and affine registrations. During realignment, rigid body transformations (translations and rotations in the X, Y, and Z directions) were computed to find the resulting image that minimized differences between slices, within a subject. Then in normalization, affine transformations (zooms and shears) were computed to maximize the fit between the EPI template brain and the anatomical scans, as well as to correct for anatomical differences between subjects.

7. The temporal derivative was included to account for small latency differences in hemodynamic delays due to the self-paced nature of the task (Calhoun, Stevens, Pearlson, & Kiehl, 2004).