Classification Of MeMory InTerventions: Rationale and developmental process of the COMMIT tool

ABSTRACT

Over the last decades, numerous memory interventions have been developed to mitigate memory decline in normal ageing. However, there is a large variability in the success of memory interventions, and it remains poorly understood which memory intervention programs are most effective and for whom. This is partially explained by the heterogeneity of memory intervention protocols across studies as well as often poor reporting of the study design. To facilitate a reporting framework that enables researchers to systemize the content and design of memory intervention paradigms, we developed the Classification Of MeMory InTerventions (COMMIT) tool using a 3-stage developmental process. Briefly, COMMIT was based on qualitative content analysis of already existing memory intervention studies published between April 1983 and July 2020, and iteratively validated by both internal and external expert panels. COMMIT provides an easily-applicable interactive tool that enables systematic description of memory intervention studies, together with instructions on how to use this classification tool. Our main goal is to provide a tool that enables the reporting and classification of memory interventions in a transparent, comprehensible, and complete manner, to ensure a better comparability between memory interventions, and, to ultimately contribute to the question which memory intervention shows the greatest benefits.

KEYWORDS:

• Ageing
• memory
• rehabilitation
• interventions
• interactive tool
• reporting

Introduction

Ageing is characterized by a decline in various cognitive domains, such as episodic memory, executive function, and processing speed (Salthouse, 2019), posing a threat to functional independence, autonomy, and quality of life in older age (Missotten et al., 2008). However, the process of cognitive ageing is highly heterogeneous. Individuals not only differ in their susceptibility to age-related cognitive decline and underlying neural changes, but also in the cognitive domains that are being affected (Cabeza et al., 2018). Together with the notion that cognitive and neural processes have the potential to be altered at advanced age (i.e., cognitive and neural plasticity, respectively), this suggests that optimizing cognitive functions may still be possible in the ageing population (Lövdén et al., 2010).

Various interventions have been developed to maintain or improve cognitive functions in older age (Kivipelto et al., 2018). While pharmacological interventions have not been effective in reducing ageing-related memory decline, non-pharmacological interventions (e.g., physical exercise, diet adjustments, cognitive trainings) have yielded more promising results (Cotelli et al., 2012; Yao et al., 2020). Long-term memory function is considered to be a major criterion for successful ageing (Depp & Jeste, 2006; Nyberg & Pudas, 2019). This cognitive domain is also particularly susceptible to the effects of advanced age and the accumulation of neuropathology, and therefore predominantly affected by ageing-related pathologies such as Alzheimer’s disease (Hedden et al., 2013; Qiu et al., 2019; Salthouse, 2019). One of the most commonly applied cognitive rehabilitation approaches therefore aims at improving memory functions. Memory interventions typically comprise two types of rehabilitation approaches: a restorative approach, consisting of exercises and/or training paradigms specifically designed to improve memory function (i.e., memory function training), and a compensatory approach which focuses on acquiring and applying compensatory strategies (i.e., memory strategy training). Both intervention approaches have been used to target cognitive decline in both healthy and pathological ageing (Gates et al., 2011; Lampit et al., 2014; Mowszowski et al., 2010).

Increasing evidence suggests that memory interventions indeed could be successful in improving memory functions across diverse populations (Gates et al., 2011; Gross et al., 2012; Hill et al., 2017; Lampit et al., 2014; Leung et al., 2015). However, the efficacy of these interventions varies greatly across studies (Eikelboom et al., 2020; Gross et al., 2012; Mewborn et al., 2017; Zelinski, 2009). These mixed results may be partially explained by heterogeneity in study paradigms, for example, in terms of the applied setting (e.g., single vs. group trainings), materials used (e.g., digital vs. paper-pencil trainings), or content of the particular intervention (e.g., strategy training vs. memory exercises; Kelly et al., 2014; Lampit et al., 2014; Mewborn et al., 2017). In addition, individual characteristics may also influence intervention success, such as cognitive status (e.g., presence of mild cognitive impairment), baseline performance (e.g., on memory), or sociodemographic variables such as age, sex, and education (Mewborn et al., 2017; Roheger et al., 2020a).

Nevertheless, systematic reviews and meta-analyses focusing on the overall effects of memory interventions or potential individual variables that influence the performance of memory interventions experienced difficulties and/or were unable to perform a proper evidence synthesis (Kueider et al., 2012; Roheger et al., 2020b; Traut et al., 2021; Zehnder et al., 2009; Zhu et al., 2016). These difficulties were not only attributed to the aforementioned variability in memory intervention components of the studies itself, but also to insufficient reporting of memory intervention studies, making it difficult to categorize such components for further analysis (Eikelboom et al., 2020; Roheger et al., 2020a).

Considering the potential impact of memory interventions in preserving or improving healthy ageing (Gross et al., 2012; Kueider et al., 2012; Mewborn et al., 2017), the reporting of memory interventions should facilitate transparency and allow researchers to critically appraise the methodological design and results. To our knowledge, however, none of the existing guidelines cover the specific aspects of memory intervention studies, such as the applied setting, materials used, or content of the particular intervention (see, e.g., the Enhancing the QUAlity and Transparency Of health Research [EQUATOR] network guidelines; Altman & Simera, 2016; Simera et al., 2010).

Consequently, we developed COMMIT (Classification Of MeMory InTerventions), a standardized scoring tool together with instructions for detailed reporting of memory interventions to facilitate a comprehensive reporting framework. The COMMIT tool specifically incorporates the diversity in intervention components that have been used across memory interventions to describe and classify the varying components of such intervention studies. COMMIT aims to: (1) systematically classify the content of memory interventions, and (2) assist authors to present their memory intervention studies in a transparent, comprehensible, and complete manner. Therefore, COMMIT enables researchers to accurately describe their own memory intervention studies, while also allowing researchers and publishers to critically appraise such studies and extract data for systematic reviews, meta-analyses, and mega-analyses.

Materials and methods

The development of COMMIT was based on the following 3-stage approach: (1) a systematic search for all memory intervention studies conducted among healthy ageing individuals published until January 2021; (2) a qualitative data analysis to derive a classification system of memory intervention studies of the methods section of each memory intervention in a stratified sample; (3) translation of the classification system into the COMMIT tool. For a flowchart of this process, see Figure 1.

Figure 1. Description of the different developmental stages of the COMMIT tool.

The COMMIT tool was designed using a 3-stage developmental process, consisting of (1) a systematic search for memory intervention studies, (2) qualitative data-analysis to derive a category system of memory intervention studies, and (3) translation of the category system to the COMMIT tool.

Development stage 1: Systematic search for memory intervention studies

Search and study selection: We performed the systematic search and study selection as a follow-up on a previously published review focusing on prognostic factors for change in memory functioning after a memory intervention among healthy older adults ≥ 55 years (Roheger et al., 2020a). Briefly, we performed a systematic search across MEDLINE, Web of Science Core Collection, CENTRAL, and PsycInfo until November 2019, and updated our search for the present study until January, 2021. We additionally searched the reference lists of all identified trials, relevant review articles, and current treatment guidelines for relevant content. When no full text was readily available, we contacted the authors to ask for the full text publications within a 2-week time frame. The systematic search and full search strings for each database are presented in Supplementary Material (“Overview of systematic search strings”).

Two authors initially screened the titles and abstracts in accordance with predefined eligibility criteria (MR and AKF) with the Covidence Software (Veritas Health Innovation). Full-text articles of the research papers were subsequently screened for inclusion in the present study by two authors (MGJ and MR). In case of uncertainty about inclusion, final consensus about particular studies was reached during consensus meetings with co-authors (AKF, JMO, EK).

Inclusion criteria: We included memory intervention studies with a minimum of two sessions focusing on healthy older adults from peer-reviewed journals, written in English. More specifically, we defined memory interventions as cognitive rehabilitation techniques with a clearly defined long-term memory intervention component, specifically incorporated to target memory-related outcomes (e.g., episodic memory, prospective memory or semantic memory). We excluded working memory interventions, as this cognitive domain more closely aligns with executive functions, and research on working memory intervention has become a field on its own (Gathercole et al., 2019; Oberauer et al., 2018; Soveri et al., 2017).

Our inclusion was not limited to studies that solely focused on memory interventions, but also covered those that combined memory interventions with other types of cognitive, pharmacological, or physical training. Furthermore, we put no restrictions in terms of the applied setting (e.g., individual or group setting) or materials used for the intervention (e.g., computerized and/or paper-pencil tasks). We also included pilot studies and study protocols. In this way, we were hoping to cover the whole range of memory intervention paradigms that have been used throughout time within our study. Nevertheless, the methodological description of a study needed to be available; if a study referred to a protocol paper for its design, we included the protocol paper instead. We excluded books, book chapters, abstracts, and conference papers.

Stratified sub-sample selection: We created a stratified sub-sample of all eligible studies (n = 274; see results section), as evaluation of the complete study sample is not required to obtain saturation (Morse, 1995). This resulted in a sub-sample of n = 40 studies. The stratified sub-sample was acquired using a three-step process. First, we arranged all eligible studies according to their publication date in 5-year time frames, starting from 1975 up until 1980, and counted the number of studies published in each time frame. Second, we calculated the relative amount of studies per time frame, and converted this number to our corresponding sample of 40 studies. For example, if n = 24 studies were published between 1996 and 2000, the relative contribution of these studies to the total sample n = 274 was 8.78%. Subsequently, from our sub-sample of n =40 studies, we randomly selected n = 4 studies that were published between 1996 and 2000, reflecting the previously calculated 8.78%. The selection of the studies within the stratified time blocks was achieved using R (https://www.R-project.org/).

Development Stage 2: Qualitative data analysis to derive initial category system

We used the qualitative data analysis software MaxQDA2020 (https://www.maxqda.com/) to apply qualitative content analysis to the method sections derived from the sub-sample of memory intervention studies, according to Kuckartz (2014). Research was performed in accordance to the COnsolidated criteria for REporting Qualitative research (COREQ; Tong et al., 2007). The quality of this process was additionally monitored by one author (DC) experienced in qualitative research and content analysis. Two authors (MR, MGJ) developed an initial category system, using both deductive and inductive categorizations as recommended in previous literature (Kuckartz, 2014). Deductive categories were derived using existing literature (Bamidis et al., 2014; Gates et al., 2011; Gross et al., 2012; Lampit, Hallock, & Valenzuela, 2014; Roheger et al., 2020a; Sitzer et al., 2006). The derived categories were subsequently revised during consensus meetings with co-authors (AKF, EK, JMO, RPCK) experienced in the field of memory intervention research, covering various specializations and subdomains relevant to disentangle memory intervention components (e.g., strategy-oriented and process-oriented memory interventions, utilization of training paradigms in healthy and/or pathological ageing, and both online and offline intervention application settings). New deductive and inductive categories were generated from the data during the content analysis of the stratified sample, where two authors (MR, MGJ) worked independently on the coding of this sample.

Two authors (MR, MGJ) subsequently discussed and harmonized the newly derived main and sub-categories, as well as on the individual coding of categories per study after each phase of independent coding, herewith optimizing the reliability and validity of the process. Whenever possible, subordinate categories were merged as long as this did not lead to loss of meaningful information during the process. Potential discrepancies were solved by consulting co-authors (AFK, EK, JMO, RPCK).

After we reached saturation on the stratified sub-sample (i.e., when no new main or sub-categories arose; Morse, 1995), a consensus meeting was organized to further refine the category system among all co-authors experienced in cognitive and/or memory intervention design (MR, MGJ, AFK, EK, JMO, RPCK; Eikelboom et al., 2020; Folkerts et al., 2017; Frankenmolen et al., 2018; Kalbe et al., 2020; Karssemeijer et al., 2017; Roheger et al., 2020b). Notably, if saturation was reached before finalizing the coding of the stratified sub-sample, we continued with processing this sample until completion. After this process, we extracted another stratified sub-sample (n = 5) among all memory intervention studies to test the newly derived category system to ensure quality and allow for potential further revision of categories. The latter process was repeated until no other revisions of the category system occurred. For a flowchart of the complete content analysis, see Figure 2.

Figure 2. Process flowchart of qualitative content analysis.

The category system of memory intervention studies was developed using qualitative content analysis of a stratified, randomized sub-sample (n = 40) of all eligible studies (n = 274). Two independent coders performed revisions, using input from an expert panel and another stratified, randomized sub-sample (n = 5), until consensus was reached.

Development Stage 3: Translation of category system into COMMIT tool

Based on the initial categorization system, we developed a first version of the COMMIT tool. During this developmental process, we aimed to make the COMMIT tool as comprehensive as possible while also maintaining an easily-applicable interface. We additionally developed detailed instructions (Table 1), describing each item and how to use the COMMIT tool.

Table 1. Guidelines to fill in the COMMIT Tool.

Section / Topic	Item	Checklist Item Explanation
Title
Title	1	Clearly identify your study as a memory intervention, directly targeting memory-related outcomes (instead of, e.g., multidomain cognitive training, working memory intervention, brain training etc.).
Goal
Goal	2	Briefly describe the overall goal of the memory intervention: • Which domains are aimed to be improved with the intervention? • In which population do you want to use the memory intervention? [e.g., “improve/maintain verbal long-term memory in healthy older participants”]
Target group(s)	3	Identify which patient / participant target group the intervention is administered to. [e.g., “healthy older adults aged > 50 years”, “patients with Parkinson’s disease”]
Description of training length
Description of duration	4	Identify how many minutes each memory intervention session lasts, how many sessions your intervention consists of per week, the total duration of all sessions in minutes, and the total duration in weeks. In case the intervention is conducted by the participants in their own speed and time, please tick “self-paced”. Note that if an intervention is not provided on a weekly basis, you could calculate the number of sessions per week by dividing the total number of sessions by the total duration of trainings per week to acquire the number of sessions per week (e.g., 6 sessions in total provided every other week for 12 weeks would result in 0.5 sessions per week [6 / 12 = 0.5]). A similar calculation could be made if the duration of each session differs across various sessions.
Booster training	5	Indicate whether your training includes a booster session (tick “booster training” in case it does), and specify the number of booster training sessions in total and the total duration in minutes. A booster session aims to review and/or refresh the content or skills provided throughout an intervention.
Other characteristics	6	Please identify any other characteristics of the training duration that are not covered by the aforementioned duration categories.
Trainer and/or therapist qualifications	7	Tick “trainer/therapist” if your training is administered by a specific trainer or therapist (rather than self-applied by the participants) and specify the qualification of the trainer (e.g., “certified trainer”, “psychology student with Bachelor degree trained for six weeks”). Also indicate whether supervision or guidance of the trainers takes place. If a learning tool or training is available for the trainer and/or therapist to perform this particular memory intervention, please indicate this too.
Training manual	8	Indicate whether the training manual is in the developmental stage or already in clinical use (tick the applicable box). Further, specify whether and where the training manual you used is available for use by others, e.g., published and available for clinicians.
Language(s) of training	9	Specify in which language(s) the training is available.
Setting
Group setting	10	Identify whether the training is performed in an individual setting (single icon), in a group-based setting (several icons), or in a combined setting (single icon + several icon). In case the setting is combined, describe how many minutes per session are conducted in each different setting at the “Other characteristics” of the “Description of duration”-section. [e.g., mixed setting, first 3 sessions á 60 min each in individual settings, 6 sessions á 60 min each in group setting]
Application setting	11	Identify whether the training is home-based (home icon), or performed at a clinical or institutional setting (hospital icon), or in a combined setting (home icon + hospital icon). In case the setting is combined, describe how many minutes per session are conducted in each different setting at the “Other characteristics” of the “Description of duration”-section. [e.g., mixed setting, first 3 sessions á 60 min each in clinical/institutional setting, 6 sessions á 60 min each in home-based setting]
Application mode	12	Identify whether the training is computerized (computer-icon), or in paper-pencil application modus (book icon), or in a combined application form (computer icon + book icon). In case the setting is combined, describe how many minutes per session are conducted in each different setting at the “Other characteristics” of the “Description of duration”-section. [e.g., mixed setting, first 20 min of each session in paper-pencil form, 30 min of each session computerized]
Supervision of training	13	Identify whether the training is self-administered, or supervised, or in a combined application form. In case the setting is combined, describe how many minutes per session are conducted in each different setting at the “Other characteristics” of the “Description of duration”-section. [e.g., mixed setting, 20 min self-administered preparatory assignments prior to each session, 40 min supervised session]
Memory intervention components
Ranking of relative contribution of components	14	Provide the relative contribution of the memory intervention components that are included in the memory intervention. The higher the contribution of a particular component, the lower the rank number (i.e., “1.” indicates the memory component that was most central in the intervention). If the memory intervention consists only of one memory intervention component, it suffices to only specify the first rank.
Internally applied memory strategies	15	Identify, which internally applied memory strategies are part of the memory intervention. You can tick internal memory strategies that are used in the present study, and further specify these in the box below. Note that this is only applicable if the memory intervention incorporates internally applied memory strategies. [e.g,. “Categorization”, Specify: Participants received a list of words and were asked to categorize the words, e.g., “animals”, “tools”]
Externally applied memory strategies	16	Identify, which externally applied memory strategies are part of the memory intervention. You can tick external memory strategies that are used in the present study, and further specify these in the box below. Note that this is only applicable if the memory intervention incorporates externally applied memory strategies. [e.g,. “external memory aids”, Specify: Participants were asked to create diary entries about specific events to help them memorize]
Practicing distinct memory tasks	17	Identify which distinct memory tasks are practiced in the memory intervention. You can tick distinct tasks that are used in the present study, and further specify these in the box below. Note that this is only applicable if distinct memory tasks are practiced in the memory intervention. [e.g., “transfer task”, Specify: Participants trained abstract word list learning and were then asked to learn a list of supermarket items]
Educative component Material/Content	18	Identify the content of the educative component(s) throughout your memory intervention and what materials are used. Specify how long the participants approximately spent on the educative component of your training. You can tick contents and materials that are used in the present study, and further specify these in the box below. Note that this is only applicable if the memory intervention contains an educative component. [e.g,. “Psychoeducation” “Lectures”, Specify: Participants received 4 h of psychoeducation on ageing and memory processes in form of lecturers taught by the training supervisor]
Additional components of training
Additional cognitive components trained	19	Identify, which additional cognitive components are trained during the memory intervention. Specify which tasks are used to train these components. You can tick the additional cognitive components that are used in the present study, and further specify these in the box below. Note that this is only applicable if the memory intervention contains additional components. [e.g., “attention”, Specify: Attention was trained with a specific version of the Example-Task by Example et al.]
Individualized	20	Specify whether the training is individualized in any form, e.g., by goal management or task adaptation strategies. [e.g., the task XY was adaptive, so that it increased its difficulty by one level in case the participant answered correctly]
Reinforcers during training	21	Specify whether any reinforcers were given during the training. In addition, indicate whether reinforcers were provided for adherence or performance. [e.g., for each successful training session, participants received a token]
Active social support	22	Specify whether any active social support is provided during the memory intervention, either by the trainer, the group, both, or other. [e.g., at the end of each session, participants shared their experiences of the training with the group; the trainer send motivational tips via an online platform]
Specify other applied components
Specify other applied components	23	Indicate, whether there are any other additional, non-cognitive training components either simultaneously and/or before/after the memory intervention, such as physical exercise, relaxation training, non-invasive brain stimulation etc. Specify details about frequency, duration, and content of the additional training. [e.g., the first 20 min of each memory intervention session were accompanied by anodal transcranial direct current stimulation.]
Feedback during the intervention
Feedback provided during intervention	24	Indicate whether any feedback to the patient or participant was provided during the intervention on adherence or performance. Feedback may have been provided on several outcome types, including near-transfer outcomes, far transfer outcomes, and patient- or subject-reported outcomes. [e.g., the outcomes of a custom-made progress questionnaire were used to provide feedback on adherence; episodic memory test XY was used to provide feedback on performance]
Briefly describe a typical memory intervention session
Briefly describe a typical memory intervention session	25	Provide a brief description of the content covered in a typical memory intervention session. This provides the reader with more contextual information with regards to the organization of the sessions. If appropriate, also indicate how much time was planned for specific tasks. [e.g,. the memory intervention typically starts with a 15-minute discussion about the homework assignments that were given in the previous session, subsequently …]
Other comments
Other comments	26	This section may be used for any additional comments about the memory intervention study that were not addressed in any of the other sections and/or to further specify certain components.

In order to improve and validate the COMMIT tool and corresponding instructions, we first acquired feedback from all co-authors with experience in memory intervention studies for internal validation (internal expert panel; MR, MGJ, AFK, EK, JMO, RPCK). For external validation, we reached out to 20 external investigators active in the field of memory intervention studies by them or by qualified members of their research groups. More specifically, we asked them about their willingness to help with the evaluation of COMMIT, together with a brief description of the purpose of COMMIT and how the evaluation process would work. Researchers were requested to fill in the COMMIT tool with the help of the instructions table (Table 1) for a memory intervention study conducted by themselves and to evaluate this process based on several aspects. These aspects included how practical it was to use the tool and corresponding guidelines, whether any categories or items were missing to correctly describe memory interventions, and whether there were any additional comments or suggestions to further improve the COMMIT tool. Once the researcher agreed with this procedure, we sent all relevant documents to facilitate their participation in our external expert panel. After receiving feedback, we revised the COMMIT tool accordingly and consulted both the internal and external expert panels for any further enquiries or revisions. Of the 20 researchers that were approached by us, five researchers replied and assisted in the evaluation of COMMIT. After processing the feedback from all parties involved, we send the revised COMMIT tool for any further feedback and/or approval.

Results

Systematic search of memory intervention studies

A complete overview of the included and excluded studies, together with reasons for exclusion, is presented in the flowchart in Supplementary Figure A. Our database search and search of included studies in previously published systematic reviews and meta-analysis on memory interventions, revealed a total of 12,975 studies. An updated search revealed 154 additional results in January 2021. After duplicate removal, we screened a total of 10,133 studies for eligibility based on predefined criteria. From the remaining articles (n = 692), we finally included 274 memory intervention studies.

Subsequently, the randomized, stratified sample of 40 studies was obtained for further analysis (see Materials and Methods). After the coding of 34 studies of the initial stratified sub-sample was completed, saturation was reached; however, we also coded the remaining 6 studies. Our category system was further refined during a consensus meeting with all co-authors. Subsequently, we derived a second stratified sub-sample of 5 studies. During coding of the latter sample, no new categories were established, indicating that saturation was sustained, and therefore we did not incorporate another stratified sub-sample. Our final stratified sample included memory intervention studies that were published between 1983 and 2020.

The main characteristics of the sub-sample of memory intervention studies are displayed in Supplementary Table A, including the sample size of the study, design, duration, and brief memory intervention description. Sample sizes of the intervention groups varied between n = 9 to n = 620 participants. Most of the studies included in this stratified sample involved randomized controlled trials (n = 36). We also included quasi-randomized controlled trails (n = 6), and (longitudinal) cohort studies (n = 3). Duration of conducted memory interventions was either self-paced (e.g., Lampit, Hallock, Moss et al., 2014), and varied from 20 min (e.g., Bureš et al., 2016) to 3 h per session (Chapman et al., 2016). In addition, a high heterogeneity regarding training frequency was found, varying from 1 session per week (e.g., Wiegand et al., 2013) to daily sessions (e.g., Antonenko et al., 2018). With regards to the duration of the training, this varied from 3 consecutive days (e.g., Antonenko et al., 2017) to 2 years (e.g., Buiza et al., 2008).

With regard to the conducted memory intervention paradigms, descriptions were highly heterogeneous, such as the naming and definition of the memory interventions (e.g., “number-consonant mnemonic training”, Hill et al., 1997; “imagery-based strategy training”, Carretti et al., 2007; “multi-factorial training program focusing on memory”, West et al., 2008; “multi-domain memory and ageing program”, Wiegand et al., 2013; “cognitive restructuring memory intervention”, Caprio-Prevette & Fry, 1996). Accordingly, not only the memory intervention content differed across studies, but also additional training components varied greatly, such as educational support, supervision of the trainings, other cognitive domains that were trained within the memory intervention, and home exercises. Altogether, these findings further emphasize the need to classify memory intervention paradigms in a more systematic manner.

Memory intervention classification system

An overview of the final category system is given in Figure 3. The coding system follows a hierarchical design with the main categories displayed on the left side of the Figure (level 1), and the sub-categories on the right side, comprising increasingly more details with every step in the hierarchy (levels 2–4).

Figure 3. Hierarchical structure of memory intervention components.

The category system of memory interventions follows a hierarchical design with 5 levels, comprising increasingly more details with every step in the hierarchy. For example, from memory intervention components (level 1) to mental imagery (level 4).

Our analysis showed that the training content of memory intervention studies could be broadly divided into two main categories (level 1). The first category involves memory intervention components and describes all training components that have been incorporated within an intervention paradigm with the direct goal to improve memory performance. The second category involves additional components of intervention. Although the elements described in the latter category are not directly incorporated to improve memory functions, these components play an important role with regards to the intervention design and conduction of the memory intervention, and may conversely also influence training efficacy (e.g., executive function training and personalized training approaches).

Memory intervention components (level 1) can be further subdivided into content of memory intervention and educative components (all level 2). Regarding the content of memory intervention (level 2), a distinction can be made between internally applied memory strategies, externally applied memory strategies, and practicing distinct memory tasks (all level 3). Internally applied memory strategies (level 3) are defined as memory strategies that do not need any external help or anchor in their execution, and involve mental manipulations to facilitate memory of targeted stimuli such as chunking (i.e., taking individual pieces of information and grouping them into larger units), associations (i.e., making an association between items), categorizations (i.e., grouping items together based on common features), use of rhymes and patterns to memorize information (i.e., creating a rhyme that involves several stimuli), and mental imagery (i.e., eliciting a perceptual experience that occurs mentally in the absence of the appropriate external stimuli; all level 4). Externally applied memory strategies (level 3), on the contrary, actively help and/or cue an individual with the information that has to be memorized. This may involve, e.g., using external aids like shopping notes, calendars, placing things in conspicuous places, and asking someone to help you remember (all level 4). Practicing distinct memory tasks (level 3) refers to the repetitive execution of diverse standardized tasks with the sole aim to practice and stimulate the function that is probed in that task, for example, by practicing list- or location learning, spaced repetition, or transfer tasks where participants are stimulated to actively practice in everyday situations (all level 4).

Educative components (level 2) is the second sub-category of memory intervention components. Here, we can distinguish between the material (level 3) that is used to facilitate transfer of knowledge (e.g., lectures that were given, homework material, self-study material; all level 4), and the content of the education (level 3), such as psychoeducation, experiential learning of training content (i.e., learning through exercises that actively facilitate experience with the topic), or traditional learning of training content (i.e., learning course concepts through lectures and reading material; all level 4).

Additional components of intervention (level 1) can be further classified into additional cognitive domains trained (level 2), such as executive functions, visuo-cognition, language, global cognition, attention or other cognitive domains (all level 3), individualized training components (level 2), such as individual training task adaptation and individual goal management (level 3), and reinforcers during training (level 2), either provided for adherence or for performance at the training (level 3). Furthermore, these additional intervention components could involve active social support (level 2), either by the trainer, the group, or both (level 3), and other additional applied components (level 2, e.g., physical exercise, medication, reminiscence therapy, etc.).

Translation of category system to COMMIT tool

After transforming the initial category system to the PDF format, we consulted the internal expert panel for feedback. Subsequently, we revised of the COMMIT tool primarily in terms of its lay-out, user-friendliness, and practical aspects. For example, we included sections that allowed for the specification of “title”, “goal”, “setting” and “description of duration” of the performed intervention. We also included the sections: “trainer and/or therapist qualifications”, “training manual”, “target group(s)”, “language(s) of training”, and a section to describe a typical memory intervention session. Although the classification system of COMMIT originates from memory interventions among healthy older adults, the latter sections were incorporated to enable the description of more clinically oriented work. Besides this, to emphasize the relative contribution of different memory intervention components, we incorporated a section that allows to rank these various aspects relatively to each other.

The external validation procedures resulted in feedback from five different external researchers (see Acknowledgments for an overview of the researchers involved). All researchers noted that the COMMIT tool was overall easily-applicable and practical to use. Furthermore, the tool was described to be comprehensive, and the initial classification system remained undisputed. With regards to the adjustment or addition of content, the researchers noted that the duration, reward, and feedback sections could be more thoroughly described. Therefore, we revised these sections accordingly, for example, by adding the possibility to describe for which measurement instruments feedback was provided. We also clarified the instructions for the corresponding items. Furthermore, to allow for any further comments about the intervention for potential aspects or specifications that were not covered by the previous sections, we added an “other comments” section. As some researchers noted that they preferred more detailed examples with regards to using COMMIT, we created examples based on existing memory intervention studies. The subsequent version was approved by all involved researchers (internal and external).

The resulting COMMIT tool comprises an interactive PDF with 26 items, some of which include sub-items (Supplementary Material, “COMMIT tool”). In addition, we provide detailed instructions with regards to how each item should be effectively incorporated within the tool (see Table 1). To facilitate the use of the COMMIT, we make the tool, corresponding instructions, and pre-filled examples openly available via OSF (https://osf.io/bg9q8/).

Discussion

COMMIT was developed as an easily-applicable interactive tool with detailed instructions and examples, to facilitate a reporting framework that enables researchers to systemize the content and structural aspects of memory interventions. COMMIT is the first tool that takes the heterogeneous nature in the content of memory interventions into account, as its classification system is based on qualitative content analysis of a stratified sub-sample of previously published memory interventions. The studies included in this sub-sample were highly variable in sample size, design, and memory intervention components described, further emphasizing the need for more systematic categorization of such studies. Together with incomplete reporting of memory intervention paradigms, this heterogeneity has led to methodological limitations in determining the efficacy and efficiency of memory interventions in previous studies (Eikelboom et al., 2020; Roheger et al., 2020a). COMMIT aims to tackle these difficulties by providing a categorization system for standardized reporting of memory interventions.

We propose that all memory classification components as listed in COMMIT need to be separately addressed in the main report of memory intervention studies. We additionally encourage to upload the filled-in COMMIT tool as additional material either in the supplementary material or in open data registries (such as OSF; https://osf.io/). A link or reference to the checklist can be provided in the main paper. If it is not possible to describe all memory classification components in the main text (e.g., due to word count restrictions), we recommend uploading the filled-in COMMIT tool as an online supplement. In this way, COMMIT continues to assist researchers in describing all crucial components of a memory intervention study and ensures that other researchers can still easily appraise the memory intervention. Furthermore, we strongly encourage using the instructions and examples in conjunction with the checklist to foster complete, coherent, and transparent reporting.

Although COMMIT addresses the diverse approaches that were incorporated in previous memory interventions, it does not include recommendations for designing, conducting, and analysing trials. Therefore, this tool should be used as an addition to previously established reporting guidelines from the EQUATOR network, for example, the Consolidated Standards of Reporting Trials (CONSORT) statement for randomized trials (Schulz et al., 2010) and the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement for observational studies (von Elm et al., 2007). Nevertheless, COMMIT does provide a template for standardized reporting of memory intervention studies which eases the comparison between studies and may also identify missing aspects in the design and/or conduct of a particular study. Potential users of COMMIT thus may not only include researchers and clinicians who conduct memory interventions or who want to synthesize evidence on different memory interventions, but also organizations offering memory interventions, journal editors, manuscript reviewers, and readers who want to get a detailed insight in memory intervention studies. Therefore, COMMIT could serve multiple purposes in the scientific community, not only allowing authors to present memory intervention studies in a transparent yet comprehensible manner, but also by aiding in the critical appraisal of the content of such studies and hence extracting useful information for proper data synthesis in systematic reviews and meta-analyses. In this way, COMMIT may also complement the recently developed Cognitve Treatments Article Library and Evaluation (CogTale), an online repository for cognitive intervention trials which directly facilitates the performance of meta-analyses through its own platform (Sabates et al., 2021).

Although COMMIT focuses on the reporting of memory interventions in cognitive ageing, these interventions are also highly relevant to improve memory functioning in, for example, traumatic brain injury, stroke, multiple sclerosis, and Parkinson’s disease (Elliott & Parente, 2014; Taylor et al., 2021; Leung et al., 2015; Kalbe et al., 2020). Furthermore, memory interventions are only one part of the huge range of cognitive training approaches which target, amongst others, the domains of working memory, attention, visuo-cognitive or a mix of all or of some of these domains (Jiang et al., 2017; Mewborn et al., 2017; Parsons et al., 2016; Soveri et al., 2017). COMMIT can be seen as a starting point for the development of classification and reporting recommendations for cognitive trainings across diverse populations, as this research field is embossed with high heterogeneity and complexity (Butler et al., 2018; Roheger et al., 2021; Shani et al., 2021; Traut et al., 2021). Besides this, we emphasize that the COMMIT tool represents an evolving recommendation, which still requires perpetual reappraisal and, if necessary, modifications. Therefore, we would like to encourage the scientific community to focus their future efforts in revising COMMIT.

Taken together, we present an interactive tool that allows for the systematic classification and description of memory interventions: COMMIT – a reporting framework that enables researchers to report and systemize the content and structural aspects of memory intervention paradigms.

Declaration of interest statement

AKF and EK are authors of the cognitive intervention series “NEUROvitalis” (ProLog, Therapie und Lernmittel, Cologne, Germany), but receive no corresponding honoraria. RPCK is partially supported by EFRO OP-Oost grant #00928.

Acknowledgements

We thank Dr. Susan Vandermorris, Dr. Jonathan Evans, Dr. Anne Cecilie Bråthen, Dr. Anne Corbett, and Dr. Clive Ballard for their time, constructive feedback, and useful suggestions to improve the COMMIT tool.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The latest version of the COMMIT tool is accessible through OSF (https://osf.io/bg9q8/). The version of the COMMIT tool used at the time of publishing is provided in our Supplementary Material. For further details about the total amount of the different memory intervention categories and coding for each individual included study (i.e., code book of qualitative analysis in MaxQDA), please send an e-mail to: [email protected].

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.