Understanding Speech amid the Jingle and Jangle: Recommendations for Improving Measurement Practices in Listening Effort Research

References

• Alhanbali, S., Dawes, P., Millman, R. E., & Munro, K. J. (2019). Measures of listening effort are multidimensional. Ear and Hearing. doi:10.1097/AUD.0000000000000697
   Web of Science ®Google Scholar
• Alsius, A., Navarra, J., & Soto-Faraco, S. (2007). Attention to touch weakens audiovisual speech integration. Experimental Brain Research, 183(3), 399–404.
   Web of Science ®Google Scholar
• Barry, A. E., Chaney, B., Piazza-Gardner, A. K., & Chavarria, E. A. (2014). Validity and reliability reporting practices in the field of health education and behavior: A review of seven journals. Health Education & Behavior: The Official Publication of the Society for Public Health Education, 41(1), 12–18.
   PubMed Web of Science ®Google Scholar
• Bourland-Hicks, C., & Tharpe, A. M. (2002). Listening effort and fatigue in school-age children with and without hearing loss. Journal of Speech, Language, and Hearing Research: JSLHR, 45(3), 573–584.
   Web of Science ®Google Scholar
• Broadbent, D. E. (1958). The effects of noise on behavior. In D. E. Broadbent (Ed.), Perception and communication (pp. 81–107). Elmsford, NY, US: Pergamon Press.
   Google Scholar
• Brown, V. A., McLaughlin, D. J., Strand, J. F., & Van Engen, K. J. (2020). Rapid adaptation to fully intelligible nonnative-accented speech reduces listening effort. The Quarterly Journal of Experimental Psychology. doi:10.1177/1747021820916726
   Web of Science ®Google Scholar
• Brown, V. A., & Strand, J. F. (2018). Noise increases listening effort in normal-hearing young adults, regardless of working memory capacity. Language, Cognition and Neuroscience, 34, 628–640.
   Web of Science ®Google Scholar
• Brown, V. A., & Strand, J. F. (2019). About face: Seeing the talker improves spoken word recognition but increases listening effort. Journal of Cognition, 2(1). doi:10.5334/joc.89
   Google Scholar
• Brysbaert, M. (2019). How many participants do we have to include in properly powered experiments? A tutorial of power analysis with reference tables. Journal of Cognition, 2(1), 187.
   Google Scholar
• Cohen, R. J., & Swerdlik, M. (2010). Psychological testing and assessment: An introduction to tests and measurement. New York: McGraw Hill.
   Google Scholar
• Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302.
   PubMed Web of Science ®Google Scholar
• Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450–466.
   Google Scholar
• Dang, J., King, K. M., & Inzlicht, M. (2020). Why are self-report and behavioral measures weakly correlated? Trends in Cognitive Sciences, 24(4), 267–269.
   PubMed Web of Science ®Google Scholar
• Desjardins, J. L., & Doherty, K. A. (2013). Age-related changes in listening effort for various types of masker noises. Ear and Hearing, 34(3), 261–272.
   PubMed Web of Science ®Google Scholar
• Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science. doi:10.31234/osf.io/hs7wm
   Google Scholar
• Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378.
   Web of Science ®Google Scholar
• Francis, A. L., & Love, J. (2019). Listening effort: Are we measuring cognition or affect, or both? Wiley Interdisciplinary Reviews. Cognitive Science, 11(1), e1514.
   Google Scholar
• Francis, A. L., & Oliver, J. (2018). Psychophysiological measurement of affective responses during speech perception. Hearing Research, 369, 103–119.
   Web of Science ®Google Scholar
• Fraser, S., Gagné, J.-P., Alepins, M., & Dubois, P. (2010). Evaluating the effort expended to understand speech in noise using a dual-task paradigm: The effects of providing visual speech cues. Journal of Speech, Language, and Hearing Research: JSLHR, 53(1), 18–33.
   PubMed Web of Science ®Google Scholar
• Gagné, J.-P., Besser, J., & Lemke, U. (2017a). Behavioral assessment of listening effort using a dual-task paradigm: A review. Trends in Hearing, 21, 2331216516687287.
   Web of Science ®Google Scholar
• Gagné, J.-P., Besser, J., & Lemke, U. (2017b). Behavioral assessment of listening effort using a dual-task paradigm: A review. Trends in Hearing, 21, 2331216516687287.
   Web of Science ®Google Scholar
• García-Pérez, M. A. (2012). Statistical conclusion validity: Some common threats and simple remedies. Frontiers in Psychology, 3, 325.
   Web of Science ®Google Scholar
• Gosselin, P. A., & Gagné, J.-P. (2011). Older adults expend more listening effort than young adults recognizing audiovisual speech in noise. International Journal of Audiology, 50(11), 786–792.
   PubMed Web of Science ®Google Scholar
• Hales, A. H., Wesselmann, E. D., & Hilgard, J. (2018). Improving psychological science through transparency and openness: An overview. Perspectives on Behavior Science. doi:10.1007/s40614-018-00186-8
   PubMed Web of Science ®Google Scholar
• Herrmann, B., & Johnsrude, I. S. (2020). A model of listening engagement (MoLE). Hearing Research, 397, 108016.
   PubMed Web of Science ®Google Scholar
• Johnson, J., Xu, J., Cox, R., & Pendergraft, P. (2015). A comparison of two methods for measuring listening effort as part of an audiologic test battery. American Journal of Audiology, 24(3), 419–431.
   PubMed Web of Science ®Google Scholar
• Kelley, T. L. (1927). Interpretation of educational measurements. 353. https://psycnet.apa.org/fulltext/1928-00533-000.pdf
   Google Scholar
• Kramer, S. E., Lorens, A., Coninx, F., Zekveld, A. A., Piotrowska, A., & Skarzynski, H. (2012). Processing load during listening: The influence of task characteristics on the pupil response. Language and Cognitive Processes, 28(4), 426–442.
   Google Scholar
• Larsby, B., Hällgren, M., Lyxell, B., & Arlinger, S. (2005). Cognitive performance and perceived effort in speech processing tasks: Effects of different noise backgrounds in normal-hearing and hearing-impaired subjects. International Journal of Audiology, 44(3), 131–143.
   PubMed Web of Science ®Google Scholar
• Liben-Nowell, D., Strand, J., Sharp, A., Wexler, T., & Woods, K. (2019). The danger of testing by selecting controlled subsets, with applications to spoken-word recognition. Journal of Cognition, 2, 1.
   Google Scholar
• Mackersie, C. L., & Cones, H. (2011). Subjective and psychophysiological indexes of listening effort in a competing-talker task. Journal of the American Academy of Audiology, 22(2), 113–122.
   PubMed Web of Science ®Google Scholar
• McClelland, G. H., Lynch, J. G., Irwin, J. R., Spiller, S. A., & Fitzsimons, G. J. (2015). Median splits, Type II errors, and false–positive consumer psychology: Don’t fight the power. Journal of Consumer Psychology: The Official Journal of the Society for Consumer Psychology, 25(4), 679–689.
   Web of Science ®Google Scholar
• McCoy, S. L., Tun, P. A., Cox, L. C., Colangelo, M., Stewart, R. A., & Wingfield, A. (2005). Hearing loss and perceptual effort: Downstream effects on older adults’ memory for speech. The Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology, 58(1), 22–33.
   PubMed Web of Science ®Google Scholar
• McGarrigle, R., Dawes, P., Stewart, A. J., Kuchinsky, S. E., & Munro, K. J. (2016). Pupillometry reveals changes in physiological arousal during a sustained listening task. Psychophysiology, 54(2), 193–203.
   Web of Science ®Google Scholar
• McGarrigle, R., Munro, K. J., Dawes, P., Stewart, A. J., Moore, D. R., Barry, J. G., & Amitay, S. (2014). Listening effort and fatigue: What exactly are we measuring? A British society of audiology cognition in hearing special interest group “white paper.”. International Journal of Audiology, 53(7), 433–445.
   PubMed Web of Science ®Google Scholar
• Mishra, S., Lunner, T., Stenfelt, S., Rönnberg, J., & Rudner, M. (2013a). Visual information can hinder working memory processing of speech. Journal of Speech, Language, and Hearing Research, 56, 1120–1132.
   PubMed Web of Science ®Google Scholar
• Mishra, S., Lunner, T., Stenfelt, S., Rönnberg, J., & Rudner, M. (2013b). Seeing the talker’s face supports executive processing of speech in steady state noise. Frontiers in Systems Neuroscience, 7, 96.
   PubMedGoogle Scholar
• Murphy, D. R., Craik, F. I. M., Li, K. Z. H., & Schneider, B. A. (2000). Comparing the effects of aging and background noise on short-term memory performance. Psychology and Aging, 15(2), 323–334.
   PubMed Web of Science ®Google Scholar
• Ng, E. H. N., Rudner, M., Lunner, T., Pedersen, M. S., & Rönnberg, J. (2013). Effects of noise and working memory capacity on memory processing of speech for hearing-aid users. International Journal of Audiology, 52(7), 433–441.
   PubMed Web of Science ®Google Scholar
• Obleser, J., Wöstmann, M., Hellbernd, N., Wilsch, A., & Maess, B. (2012). Adverse listening conditions and memory load drive a common α oscillatory network. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 32(36), 12376–12383.
   PubMed Web of Science ®Google Scholar
• Pals, C., Sarampalis, A., & Baskent, D. (2013). Listening effort with cochlear implant simulations. Journal of Speech, Language, and Hearing Research: JSLHR, 56(4), 1075–1084.
   PubMed Web of Science ®Google Scholar
• Panico, J., & Healey, E. C. (2009). Influence of text type, topic familiarity, and stuttering frequency on listener recall, comprehension, and mental effort. Journal of Speech, Language, and Hearing Research: JSLHR, 52(2), 534–546.
   Web of Science ®Google Scholar
• Parsons, S., Kruijt, A.-W., & Fox, E. (2019). Psychological science needs a standard practice of reporting the reliability of cognitive-behavioral measurements. Advances in Methods and Practices in Psychological Science, 2(4), 378–395.
   Google Scholar
• Pichora-Fuller, M. K. (2016). How social psychological factors may modulate auditory and cognitive functioning during listening. Ear and Hearing, 37(Suppl 1), 92S– 100S.
   Google Scholar
• Pichora-Fuller, M. K., Kramer, S. E., Eckert, M. A., Edwards, B., Hornsby, B. W. Y., Humes, L. E., … Wingfield, A. (2016). Hearing impairment and cognitive energy: The framework for understanding effortful listening (FUEL). Ear and Hearing, 37(Suppl 1), 5S– 27S.
   PubMed Web of Science ®Google Scholar
• Pichora-Fuller, M. K., Schneider, B. A., & Daneman, M. (1995). How young and old adults listen to and remember speech in noise. The Journal of the Acoustical Society of America, 97(1), 593–608.
   PubMed Web of Science ®Google Scholar
• Picou, E. M., & Ricketts, T. A. (2014). The effect of changing the secondary task in dual-task paradigms for measuring listening effort. Ear and Hearing, 35(6), 611–622.
   PubMed Web of Science ®Google Scholar
• Picou, E. M., Ricketts, T. A., & Hornsby, B. W. Y. (2011). Visual cues and listening effort: Individual variability. Journal of Speech, Language, and Hearing Research: JSLHR, 54(5), 1416–1430.
   PubMed Web of Science ®Google Scholar
• Piquado, T., Isaacowitz, D., & Wingfield, A. (2010). Pupillometry as a measure of cognitive effort in younger and older adults. Psychophysiology, 47(3), 560–569.
   PubMed Web of Science ®Google Scholar
• Porretta, V., & Tucker, B. V. (2019). Eyes wide open: Pupillary response to a foreign accent varying in intelligibility. Frontiers in Communication,4(8).
   Google Scholar
• Rabbitt, P. M. (1968). Channel-capacity, intelligibility and immediate memory. The Quarterly Journal of Experimental Psychology, 20(3), 241–248.
   PubMed Web of Science ®Google Scholar
• Rönnberg, J., Lunner, T., Zekveld, A., Sörqvist, P., Danielsson, H., Lyxell, B., … Rudner, M. (2013). The ease of language understanding (ELU) model: Theoretical, empirical, and clinical advances. Frontiers in Systems Neuroscience, 7, 31.
   PubMedGoogle Scholar
• Rönnberg, J., Rudner, M., Foo, C., & Lunner, T. (2008). Cognition counts: A working memory system for ease of language understanding (ELU). International Journal of Audiology, 47(sup2), S99–S105.
   PubMed Web of Science ®Google Scholar
• Rudner, M., Lunner, T., Behrens, T., Thorén, E. S., & Rönnberg, J. (2012). Working memory capacity may influence perceived effort during aided speech recognition in noise. Journal of the American Academy of Audiology, 23(8), 577–589.
   PubMed Web of Science ®Google Scholar
• Rudner, M., Mishra, S., Stenfelt, S., Lunner, T., & Rönnberg, J. (2016). Seeing the talker’s face improves free recall of speech for young adults with normal hearing but not older adults with hearing loss. Journal of Speech, Language, and Hearing Research: JSLHR, 59(3), 590–599.
   Web of Science ®Google Scholar
• Sarampalis, A., Kalluri, S., Edwards, B., & Hafter, E. (2009). Objective measures of listening effort: Effects of background noise and noise reduction. Journal of Speech, Language, and Hearing Research: JSLHR, 52(5), 1230–1240.
   PubMed Web of Science ®Google Scholar
• Seeman, S., & Sims, R. (2015). Comparison of psychophysiological and dual-task measures of listening effort. Journal of Speech, Language, and Hearing Research: JSLHR, 58(6), 1781–1792.
   Web of Science ®Google Scholar
• Shadish, W. R., Jr., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
   Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.
   PubMed Web of Science ®Google Scholar
• Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017). Constraints on generality (COG): A proposed addition to all empirical papers. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 12(6), 1123–1128.
   PubMedGoogle Scholar
• Sommers, M. S., & Phelps, D. (2016). Listening effort in younger and older adults: A comparison of auditory-only and auditory-visual presentations. Ear and Hearing, 37(Suppl 1), 62S– 8S.
   Google Scholar
• Sommers, M. S., Tye-Murray, N., Barcroft, J., & Spehar, B. P. (2015). The effects of meaning-based auditory training on behavioral measures of perceptual effort in individuals with impaired hearing. Seminars in Hearing, 36(4), 263–272.
   Google Scholar
• Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74(11), 1.
   Google Scholar
• Strand, J. F., Brown, V. A., & Barbour, D. L. (2020). Talking points: A modulating circle increases listening effort without improving speech recognition in young adults. Psychonomic Bulletin & Review. doi:10.3758/s13423-020-01713-y
   Web of Science ®Google Scholar
• Strand, J. F., Brown, V. A., Merchant, M. B., Brown, H. E., & Smith, J. (2018). Measuring listening effort: Convergent validity, sensitivity, and links with cognitive and personality measures. Journal of Speech, Language, and Hearing Research: JSLHR, 61, 1463–1486.
   PubMed Web of Science ®Google Scholar
• Thorndike, E. L. (1904). An introduction to the theory of mental and social measurements. Teacher’s College, Columbia University, Science Press, New York.
   Google Scholar
• Tun, P. A., McCoy, S., & Wingfield, A. (2009). Aging, hearing acuity, and the attentional costs of effortful listening. Psychology and Aging, 24(3), 761–766.
   PubMed Web of Science ®Google Scholar
• Zekveld, A. A., Koelewijn, T., & Kramer, S. E. (2018). The Pupil Dilation Response to Auditory Stimuli: Current State of Knowledge. Trends in Hearing, 22, 2331216518777174.
   Web of Science ®Google Scholar
• Zekveld, A. A., & Kramer, S. E. (2014). Cognitive processing load across a wide range of listening conditions: Insights from pupillometry. Psychophysiology, 51(3), 277–284.
   PubMed Web of Science ®Google Scholar
• Zekveld, A. A., Kramer, S. E., & Festen, J. M. (2010). Pupil response as an indication of effortful listening: The influence of sentence intelligibility. Ear and Hearing, 31(4), 480–490.
   PubMed Web of Science ®Google Scholar
• Zekveld, A. A., Kramer, S. E., & Festen, J. M. (2011). Cognitive load during speech perception in noise: The influence of age, hearing loss, and cognition on the pupil response. Ear and Hearing, 32(4), 498–510.
   PubMed Web of Science ®Google Scholar