Monitoring speech recognizer feedback during data entry from short‐term memory: A working memory analysis

Subjects recalled items from short‐term memory by speaking into a speech recognizer. Two experiments examined effects of the type of feedback provided by the device during this data entry task. Three types of feedback were compared, varying in: modality, either auditory or visual; timing, either concurrent or terminal, and specificity, either verbal or nonverbal. Recognizer performance was better with concurrent feedback than with terminal feedback and better with nonverbal feedback than with verbal feedback. In terms of the efficiency of memory (the number of errors and the rate of data throughput), performance was more impaired by concurrent verbal feedback than by nonverbal feedback. Two main functional features of feedback in automatic speech recognition were identified: (1) the degree of similarity between the feedback and the phonologically‐coded information held in short‐term memory, which pointed to the dangers of spoken feedback and to a lesser extent the use of verbal visual feedback, and (2) the extent to which prompting is required to establish the timeliness of data input, a feature which is especially important with isolated‐word speech recognition.