Process scores on measures of learning and memory: Issue 1

As most clinical neuropsychologists can attest, interpretation of summary scores from multi-trial learning and memory measures have long been a staple of clinical evaluations. These scores include the total information recalled immediately while learning presented stimuli, and the information recalled after a delay. Relatively recently, increased consideration has been paid to the process used by a patient to acquire the material. In essence, understanding “how” a patient learns can also shed light onto clinical condition and – importantly – potential response to behavioral intervention. A host of learning and memory “process scores” – methods to understand patterns of performance – have been developed, including those examining the impact of order of stimuli presentation (e.g., serial position effect), the proportions of stimuli recalled or forgotten, the frequencies of repetition or intrusions, and the degree of benefit from clustering stimuli. When considering such research, it is important to highlight that advocating for the use of process scores does not imply that their use should take the place of traditional summary metrics; instead, they can be used to complement learning and memory summary scores to aid in clinical decision making.

As a result of these trends, the Journal of Clinical and Experimental Neuropsychology sought to gather up-to-date findings on process scores from influential investigators in the field. The response to our call for a Special Issue titled “Process Scores on Measures of Learning and Memory,” was great enough to fill two separate issues, separated into those considering patterns of performance in either learning or aspects of delayed memory. In the present issue, highlighted aspects of learning pertain to both strategies for enhancement of and obstacles to successful information processing, statistical considerations for understanding uncertainty in verbal learning, and methods of characterizing rates of learning.

Articles in introduction 1 of the special issue

In our first manuscript, “Semantic clustering on common list-learning tasks: A systematic review of the state of the literature and recommendations for future directions,” Bair et al. (2023) examined semantic clustering (SC) on the California Verbal Learning Test (CVLT) and the Hopkins Verbal Learning Test – Revised (HVLT-R). By grouping stimuli by semantically-related categories, SC is commonly considered one of the most efficient organizational strategies when completing list-learning tasks. However, to date, comprehensive review of this processing strategy has been lacking. As a result, the authors conducted a systematic review of 104 studies on SC on the CVLT and HVLT-R (common measures that utilize a clustered stimulus-presentation approach), which included characterizing the relationship of SC with cognitive performance and focusing on contrasting this approach with other memory strategies. The authors observed that there is general consistency across the literature regarding the strong association between higher tendencies toward SC and better learning and memory performance on independent measures (rs = .30 to .55). These results appear to be stronger than the associations between SC and tasks of executive functioning (most findings had rs = −.18 to .24), which is surprising given that the encoding strategy is often considered an organizational – or executive – tool. The authors additionally identified that strategy usage tended to be mutually exclusive; for example, use of SC usually came at the expense of serial clustering (memory processing based on the order of items), and that SC was a more beneficial memory aid between the two. Finally, the authors found that clinical groups tended to utilize SC less frequently than cognitively intact populations. Overall, the authors concluded that while SC seems to be a beneficial memory tool, future research is warranted to better understand its association with biomarkers of Alzheimer’s disease (AD; e.g., β-amyloid), as well as to broaden the investigation of the strategy beyond the CVLT (and the HVLT-R to a limited extent). Additionally, the authors proposed that future work should consider the impact of anxiety on implementing such organizational strategies for patients both in the clinic and in daily life.

Also examining learning styles, Egeland and Raudeberg (2023) – in the article “Patterns of Proactive Interference in CVLT-II: Evidence of a low-organized, disorganized, and highly organized learning” – focused their investigation not into clustering strategies, but on the ways that learning can be impacted by proactive interference (PI). PI, or the reduced capacity to learn new information as a result of information that has been previously presented, can be operationalized as better performance on the first trial of a word list (list A) in comparison to a subsequent distractor word list (list B). PI has been described as a normal phenomenon, one that is often explained as reduced allocation of mental effort as testing progresses. In their study, the authors sought to identify patterns of cognitive performance as a function of PI, false positive recognition errors, and semantic organization on the CVLT in 731 patients with neurological or psychiatric disorders from Norway. They observed six clusters of performance, with three clusters displaying increased PI and one displaying notably low PI. When combining all predictors, the authors identified three overall styles of learning organization, including a Low Organized – Dysexecutive style (many false positive recognition errors, low semantic organization), a Disorganized – PI dominant style (low semantic organization and a high degree of PI), and a Highly Organized – PI style (high semantic organization, presence of PI, and few false positive errors). Further characterization of each style revealed that the Low Organized – Dysexecutive style resulted in impaired acquisition, free recall, and speeded executive functioning, and had a higher prevalence toward intellectual disability. The Disorganized – PI style displayed impaired consolidation, free recall and recognition, and speeded executive skills, and tended toward a higher prevalence of Attention Deficit Hyperactivity Disorder. Finally, the Highly Organized – PI style displayed effective acquisition, recall, recognition, and speed, with a tendency toward normal neuropsychological status despite the presence of PI. The authors concluded that when PI is paired with high semantic organization, its presence likely reflects a learning style characterized by a high degree of effort. When PI occurs with low semantic organization and disorganization, it likely reflects executive dysfunction. Finally, if executive dysfunction is paired with low organization, PI is likely not present. Egeland and Raudeberg noted a caveat, however, that their designations for low-organized, disorganized, and highly-organized have yet to be meaningfully operationalized in the literature, which will be necessary for future studies to apply these cluster results to be used with consistency in clinical samples.

The next manuscript, “Process models of verbal memory in cancer survivors: Bayesian process modeling approach to variation in test scores”, applied Bayesian statistics to verbal memory performances to better understand uncertainty in neuropsychological evaluations. Specifically, Potthoff et al. (2024) posited that measurement of cognition using summary scores “(over)simplifies the underlying verbal memory processes and assumes that each word in the task contributes equally to the overall score.” Instead, they proposed that process scores permit the derivation of uncertainty, which represents the variability of score measurement due to chance within an individual. Of particular clinical relevance, the authors suggested that such techniques permit a quantification of error/variance/uncertainty at a single time point, instead of needing to rely on serial assessment using test-retest reliability statistics. In their sample of 184 cancer survivors and 204 controls from the Netherlands, the authors examined memory performance from the Amsterdam Cognitive Scan – a 15-item word list across five trials – by separating performance into three states (according to Alexander et al., 2016): unlearned, partially learned, and learned. Upon multiple presentation of stimuli, words can progress from being unlearned to partially learned or learned through various learning processes, and words progress from being partially learned to learned using consolidation processes. The authors’ findings revealed that cancer survivors possessed impaired verbal memory compared to controls, likely due to a deficient retrieval process in immediate word recall from the partially learned and learned states. The study highlighted (1) the considerable uncertainty evident in using summary scores, though these summary scores still possessed greater certainty than single trials, and (2) the impact of individual-level uncertainty on group-level estimations. Taken together, these results demonstrated the potential of Bayesian estimation of memory processes to predict outcome scores and estimate uncertainty. These factors can subsequently be used to identify specific cognitive processes underlying the mechanism of verbal memory impairment for a patient, and provide more accurate diagnosis and treatment strategies.

Additionally, there has been increased interest over the past half-decade on the consideration of learning slopes, including alternative methods of calculation. Learning slopes reflect acquisition beyond the initial trial of a multi-trial learning task, and provide insight into a patient’s benefit from cuing in daily life. The traditional learning slope metric, which has been termed the Raw Learning Score (RLS), is calculated as the difference in performance between the first trial of a multi-trial learning task and the final trial; a slight modification to this method is to replace the final trial performance with the highest score on a single trial after the first trial, as it is common in memory measures with 4–5 trials for the penultimate trial to result in the highest score. Shallow learning slope curves are a hallmark of amnestic conditions, including AD (Gifford et al., 2015). A weakness with this RLS method is that it can become confounded by performance on the first trial. As in, given that there is a fixed number of items to learn on a learning task, the more information learned at the first trial, the less information is left to learn at subsequent trials – which results in a more shallow learning slope curve. In response to this problem, Spencer et al. (2020) developed an alternative to the RLS that accounted for performance at the first trial. Specifically, the Learning Ratio (LR) takes the original RLS and then divides by the quantity of information that an individual could have learned after the first trial. As such, it reflects the proportion of information learned after the first trial of a multi-trial learning task, relative to the amount of information left-to-learn.

Multiple manuscripts in this issue have focused on examining the validity of this LR metric. First, Hall et al. (2023) considered the clinical utility of LR when derived from the Neuropsychological Assessment Battery (NAB), in the study “Novel learning ratio from the NAB list learning test distinguishes between clinical groups: clinical validation and sex-related differences.” In this article, the authors used a real-world sample of 115 patients seen in a memory-disorders clinic to compare LR performance among patients categorized as cognitively normal, those with amnestic MCI, and those with dementia. Like with the NAB summary scores for learning and memory, patients with dementia performed worse on LR than those with MCI, who performed worse than the cognitively normal participants. The authors also observed an interaction between diagnostic group and sex, such that women tended to perform better than men on LR across the cognitively normal and MCI diagnostic groups; however, performance between sexes was equal in the dementia group. This finding is notable in that it supports previous suggestions that sex effects observed in cognitively normal samples tend to get washed out in impaired samples (Brunet et al., 2020), likely as a result of the flattened range of performance in diagnostic groups. The authors additionally observed that the NAB LR displayed convergent validity with summary scores for other memory measures (rs = .53 to .55), as well as some tasks of executive functioning (Trail Making Test-B and Reynolds Interference Test; rs = −.22 to .38), but not with tasks of visuospatial construction or basic speeded processing (rs = −.11 to −.08). Hall and colleagues concluded that despite previously-studied list-learning tests, the values observed for LR derived from the NAB – and their convergent validity – were consistent with LR values derived from other memory measures in the literature (e.g., HVLT-R, Rey Auditory Verbal Learning Test [RAVLT], and those from the Repeatable Battery for the Assessment of Neuropsychological Status) despite differences in active/passive encoding, the number of words, and the number of trials.

Similarly, in the article, “The relationship between learning slopes and Alzheimer’s Disease biomarkers in cognitively unimpaired participants with and without subjective memory concerns (SMC)”, Hammers et al. (2023) extended learning slope research earlier in the AD continuum. By examining LR derived from the RAVLT and the ADAS-Cog Word Recall subtest in 950 cognitively unimpaired older participants from the Alzheimer’s Disease Neuroimaging Initiative study, the authors observed that learning slope predicted SMC status for most learning slope metrics, including Word Recall LR, RAVLT LR, RAVLT RLS, and RAVLT Learning Over Trials. Additionally, they identified that lower LR scores from both measures were related to greater hippocampal atrophy in the SMC group, whereas the cognitively unimpaired group displayed either no relationship or a negative relationship between slope and hippocampal volume. A similar trend was observed for worse RAVLT LR performance being associated with higher β-amyloid deposition for the SMC group, but not the controls. The authors’ findings suggest not only that learning slopes appear to be sensitive to SMC, but that the presence of SMC in cognitively unimpaired participants influences the relationship between learning slopes and AD biomarkers.

Lastly, Spencer et al. (2024) aimed to curate recent learning slope research in their study, “A quantitative review of competing learning slope metrics: Effects of age, sex, and clinical diagnosis.” They undertook a systematic literature review of 82 studies with data on individual learning trials of memory measures to understand the effects of demographic variables and clinical diagnosis on slope performance. Owing to the majority of the aforementioned studies using the RAVLT, and because few studies have specifically examined RLS or LR, the authors retrospectively calculated learning slope metrics on the 56 RAVLT studies in their review – excluding the other 26 studies that used various other memory measures from subsequent analyses due to limited power. When considering demographic models, the authors observed that older age and male sex was associated with worse slope performance (RLS and LR), which became more pronounced later in life. They also identified that an AD diagnosis was related to learning 55–59% less material after the first trial than healthy individuals of a similar age or sex, with effects also being observed for other clinical samples (mixed clinical samples and seizure disorder = 22–30% less; severe traumatic brain injuries: 28–39% less). While the effects were observed for both learning slopes, LR scores were more strongly sensitive to effects across analyses than RLS scores. Further, the authors used this pooled data to develop age and sex-specific normative comparisons for both learning slopes across a broader range of the lifespan than previous research that was restricted to the elderly. Overall, Spencer and colleagues argued for the greater clinical utility of the LR metric than the traditional RLS one, with future aims to focus on the use of LR in the prediction of clinical prognosis.

Future directions

Research on unique patterns of performance underlying acquisition and consolidation of information is strong, as evidenced by the articles in both the current Special Issue Introduction and its companion. These findings reinforce support for the use of process scores in the clinic to inform patterns of learning and behavioral recommendations. The current landscape also poses an opportunity to raise the importance of process scores in the diagnostic process even higher beyond a complementary role to traditional summary scores. Specifically, the recent FDA approvals of disease modifying treatments for AD have the potential to transform approaches toward the treatment of neurodegenerative disease. A consequence of these approvals is that the pressure to identify patients with memory decline earlier in the process has become more intensified. Focus appears to be shifting away from dementia due to AD or MCI, and onto the earliest stages of cognitive decline – including very mild MCI, subtle cognitive decline, and subjective memory concerns. It is thought that these latter conditions – which represent a perception of cognitive change by the patient that does not reach clinical thresholds using traditional summary scores – may uniquely benefit from consideration of memory process scores in differential diagnosis. While the sensitivity of such measures to identify these pre-clinical states will be paramount, future work will also require close scrutiny of their specificity to ensure clinical usefulness of the metrics. As the “signal” in these conditions is likely to be quite small, the incorporation of multiple patterns of performance may be necessary to develop “meta-process” scores that can accurately and reliably identify those barely starting the AD spectrum. For example, the Hammers lab are developing profiles of performance based on constellations of process-score metrics that are the most predictive of AD pathology – especially at the earliest stages of disease. The high cost and lack of clinical availability of amyloid- and tau-PET scans makes them prohibitive for most clinical patients; therefore, if we as a field can better predict those at greatest risk of pathology in pre-clinical populations – potentially using process scores – then neuropsychology could critically benefit patients in this new era of disease modifying treatment.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.