Process scores on measures of learning and memory: issue 2

This is the second issue of manuscripts submitted for our call for a Special Issue titled “Process Scores on Measures of Learning and Memory.” In the present issue, highlighted aspects of memory include examinations of serial position effects in relation to biomarkers of Alzheimer’s disease (AD), using process scores to find differences among clinical groups that may be obscured when focusing only on traditional metrics, and how changes in process score calculations can affect their ability to identify cognitive decline.

Articles in issue 2 of the special issue

Our first manuscript, “A comparison of diagnostic performance of word-list and story recall tests for biomarker-determined Alzheimer’s disease” by Bruno et al. (2023) compared serial position scores from the Rey Auditory Verbal Learning Test (RAVLT) and Logical Memory subtest as predictors of a biomarker of AD, CSF p-tau/Aβ42 ratio, in 295 participants from the University of Wisconsin – Madison Alzheimer’s Disease Research Center and Wisconsin Registry for Alzheimer’s Prevention. Past studies had not directly compared serial position scores from these two popular types of verbal memory assessment (i.e., list-learning and story recall). Bruno and colleagues measured serial position effects using delayed primacy and recency ratios. These ratios capture a loss of primacy and recency information by comparing recall from those regions from the initial learning trial to the delayed recall trial (e.g., Bruno et al., 2021). For example, the primacy ratio for the RAVLT compares recall of the words of the first four words on the RAVLT list on the first learning trial to recall of those words on the delayed recall trial. Logical Memory metrics were superior to RAVLT metrics in predicting biomarker-determined AD with the delayed primacy ratio being the strongest predictor. These measures were more robust predictors than traditional memory metrics (e.g., total delayed recall) which highlights the potential for process scores to provide incremental validity in predicting important clinically relevant outcomes.

Primacy and recency as predictors of an AD biomarker was also the focus of another manuscript, “Primacy and recency effects in verbal memory are differentially associated with post-mortem frontal cortex p-tau 217 and 202 levels in a mixed sample of community-dwelling older adults.” Gicas et al. (2023) used data from 1091 participants from two studies conducted by the Rush Alzheimer’s Disease Center, the Religious Orders Study and Rush Memory and Aging Project. The majority of participants (70%) were cognitively unimpaired at baseline, but in a later assessment prior to death, a majority (65%) met criteria for MCI or dementia. The CERAD Word List Memory test was used to derive primacy and recency scores for learning trials; as the words are presented in different orders at each learning trial, no delayed recall serial position effect scores could be calculated. Nonetheless, the authors found that primacy at baseline was associated with p-tau 217 at autopsy. Findings for recency were moderated by the progression of neurofibrillary tangles (i.e., Braak stage) such that associations with p-tau only emerged in more advanced Braak stages. No associations were found with p-tau 202 as predicted; this comparison was chosen as it does not track with AD disease progression or cognitive decline. Interestingly, primacy on learning trials had a comparable association with p-tau 217 as a traditional memory measure in AD research, total delayed recall. This finding is important as some longitudinal aging studies only assess learning and immediate recall due to time constraints for data collection. A limitation of the study is that biomarker assessment was done from dorsolateral prefrontal cortex tissue rather than in other brain regions that may have been affected earlier in the course of AD. Nonetheless, like the Bruno et al. (2023) study, these results add to a growing literature on the potential for process scores to assist in screening those who may be positive for AD biomarkers (e.g., as part of clinical trials).

The next two manuscripts both include a comparison of traditional and process score memory measures for clinical groups. The first, “Recognition Subtests of the Repeatable Battery for the Assessment of Neuropsychological Status: Evidence for a Cortical vs. Subcortical Distinction,” includes groups of those with AD (n = 36), Parkinson’s disease (PD) (n = 55), and essential tremor (ET) (n = 105). Vehar et al. (2023) examined scores derived from recently developed recognition trials for story and figure subtests of the RBANS (Duff et al., 2021). As hypothesized, the AD group had worse recognition, as measured by a recognition discriminability index assessing the ability to distinguish between target and distractor items. The AD group overall had approximately twice as many false positive errors on recognition tests compared to the other groups. Regarding recognition discriminability, the PD and ET groups did not differ on the verbal recognition measures (i.e., List and Story Recognition) but the PD group did score lower on the visual recognition measure (i.e., Figure Recognition). The authors noted that the overlap in performance for the PD and ET groups provides further support for an association of ET with cognitive decline in at least a subset of those with ET.

This overlap in verbal recognition performance for those with ET and PD was also found in the next study, “Comparison of Core and Process Scores on the California Verbal Learning Test-3 for Parkinson’s Disease and Essential Tremor Patients.” Torres (2023) compared 70 patients with PD and 54 patients with ET on multiple traditional (e.g., short/long delay free/cued recall) and process scores (e.g., learning slope, intrusion errors, repetition errors, and serial position effects) on the California Verbal Learning Test-3 (CVLT-3). Scores were largely comparable between groups except for higher repetition errors in those with PD. Of note, this difference in higher repetition errors remained significant when comparing a subset of patients with PD and ET with normal memory as assessed by long delay free recall. It may be that this type of dysexecutive perseverative error precedes the onset of later memory problems. In further support of these repetition errors representing executive functioning deficits, they were correlated with worse performance on EF measures in the PD group, although correlations were small in magnitude. This study highlights the utility of examining process scores as the only difference between groups emerged for a process score measure. It also highlights the use of neuropsychological measures in non-memory domains to understand what cognitive processes may be driving differences observed for process scores on measures of learning and memory.

Finally, measurement issues in process scores are examined in “Serial position effects and mild cognitive impairment: a comparison of measures and scoring approaches.” A previous review of serial position effects in mild cognitive impairment (MCI) and AD (Weitzner & Calamia, 2020) identified heterogeneity in research findings that the authors speculated may have been related, at least in part, to differences in measurement approaches across studies (e.g., specific measure used, scoring approach used, which learning or memory trials were used to measure effects). In their manuscript in this issue, Weitzner and Calamia (2023) compared two scoring approaches for serial position effects on list-learning measures that vary in whether they express effects in terms of the total number of words recalled by an individual (standard scoring) or the total number of words in that region of the list (regional scoring). Standard scoring answers the question: “what percentage of your total recall is from the primacy region, middle, and recency parts of the list?” Regional scoring answers the question: “what percentage of primacy, middle, and recency words did you recall”? For example, assuming the first and last 4 words of the RAVLT were defined as the primacy and recency regions, an individual who recalled the first word and the last 2 words on the RAVLT would receive standard scores of 33% for primacy (1 primacy word divided by 3 total words recalled) and 67% for recency (2 recency word divided by 3 total words recalled) and regional scores of 25% for primacy (1 primacy word recalled of 4 possible words) and 50% for recency (2 recency words recalled of 4 possible words). Standard scores for primacy, middle, and recency are dependent on each other (as they are all based on the total number of words recalled) and sum up to 100%, while regional scores are independent of each other and each range from 0 to 100%. The RAVLT and CVLT-3 (or prior versions) are often used in serial position effect research, but RAVLT studies more often use regional scoring compared to CVLT studies which typically use standard scoring as this is what is used in the scoring software. The design of the CVLT, with semantically related words appearing scattered throughout the list, likely also affects serial position effects for those using a semantic clustering approach. In a sample of 57 healthy comparison older adults and 29 older adults with MCI, both the CVLT and RAVLT were administered, and scoring was done using both regional and standard scoring approaches. Serial position effect differences were more robust for regional scoring. Lower primacy, an expected finding for MCI, was observed consistently for the RAVLT but not CVLT-3. Serial position effects on delayed recall trials best distinguished groups. These results highlight how different measurement approaches drive heterogeneity in serial position effects studies of MCI and provide some preliminary evidence of which approaches may be best to use in future studies.

Future directions

Process scores have a rich legacy in neuropsychological assessment (Kaplan, 1990). With regard to learning and memory specifically, the findings in these manuscripts build upon work from decades ago; for example, prior work showing reduced primacy on list-learning measures in those with memory disorders (Mungas, 1983). The manuscripts in both issues include those examining process scores that have been studied for decades, more recent modifications of different ways of calculating process scores, and the use of statistical approaches to identify underlying patterns of performance on tests of learning and memory. They provide evidence for the validity of process scores in associations with AD biomarkers and clinical diagnoses.

The process scores featured in these manuscripts are but a subset of scores that could be derived from learning and memory measures. A weakness in the field is in the proliferation of novel indices at the expense of replication and extension of the clinical utility of previously derived scores. Incremental validity, both relative to traditional measures and currently widely used process scores, is often not examined, which leaves clinicians and researchers at a loss as to what scores are worthy of further attention. Additionally, there are vast differences in how consistently the same type of process scores are examined, thus adding unnecessary heterogeneity in the literature. For example, for learning slope on list-learning measures, a novel metric derived a few years ago, the Learning Ratio (Spencer et al., 2022) has now been the focus of a number of studies, some highlighted in this special issue, and thus quickly a literature has emerged on the utility of a specific scoring approach designed to improve upon a prior metric. In contrast, research on serial position effects is marked by variability in everything from the number of words considered in each region of a test (e.g., is the primacy region of the RAVLT the first 3, 4, or 5 words?), the use of simple versus ratio measures, the utilization of regional versus standard scoring, and whether the first trial, all learning trials, and/or memory trials are used. These choices are not always clearly described in studies and a lack of reporting of data in a common metric (e.g., means and standard deviations) precludes subsequent quantitative examination of the impact of these differences in systematic reviews. It is recommended that future work on process scores carefully delineate the methods used for calculation, provide descriptive statistics for their results, and examine the robustness of findings across different scoring choices. This will help to establish a common foundation to build evidence toward optimal methods for use in the lab and clinic.

Future investigation on the interaction between demographic factors and process scores appears to be warranted. A limitation noted by several authors was in the diversity of their samples (e.g., several noted overall high levels of education of their participants) and prior work has shown that racial differences in one type of learning process score, semantic clustering, may be related to differences in education quality (Fox et al., 2019). Gender differences found on traditional memory measures (e.g., delayed recall) also may or may not extend to process scores. For example, in this issue, Hall and colleagues found gender differences in cognitively healthy older adults for the Learning Ratio, while in a previous issue of this of this journal, Do and Suhr (2023) found a lack of gender differences on serial position measures in cognitively healthy older adults. Compared to published data on traditional measures, there is additionally often a lack of normative data to aid in the interpretation of process score measures although exceptions exist (e.g., the CVLT-3 contains normative data for a number of process scores as the CVLT was developed from within the Boston Process Approach [Milberg et al., 2009]). The manuscript in this issue by Spencer and colleagues provides a good example of how meta-normative data for process scores can be developed if sufficient data are reported in individual studies.

Several manuscripts in these issues used secondary analysis of data collected as part of longitudinal studies or clinical practice; there is ample room for future studies to make use of existing data in further exploring how robust process scores are to differences in measurement. There is also a need for studies to compare multiple process scores within the sample dataset to assess which scores (or combinations of scores) have the greatest clinical utility. Although this issue focused on measures of learning and memory, taking a broader perspective, it may be that process scores might be aggregated across measures within a battery (e.g., specific types of dysexecutive errors that may occur at a low base rate on an individual test) to improve reliability and validity. We hope that the set of manuscripts in these two issues will draw the attention of researchers and clinicians who are not already considering process scores to the potential benefits of their use. Many of these scores are easy to calculate with data already collected and we hope these manuscripts will encourage others to examine process scores in their own work and further grow this area of clinical neuropsychology research and practice. JCEN has published manuscripts on process scores for decades (e.g., Massman et al., 1990) and welcomes additional papers on this topic including, but not limited to, successful or unsuccessful replications of the findings in this special issue. Studying the same type of score measured in the same manner in future studies is critical to advancing the use of these scores in clinical practice.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.