Feasibility, usability and clinical value of intensive longitudinal diary assessments in older persons with cognitive impairment and depressive symptoms

Abstract

Objectives

To evaluate the feasibility, usability and clinical value of daily diary assessments combined with actigraphy in older persons with cognitive impairment.

Methods

For 63 days, patients ≥60 years with cognitive impairments filled out a daily diary (including standardized questionnaires and cognitive test battery), and wore an actiwatch (sleep). After the study, participants and clinicians received personal feedback about patterns and daily triggers of depressive symptoms, sleep and cognitive performance. We assessed feasibility (participation rate, compliance and subjective burden), usability (variability and floor- or ceiling effects) and clinical value for patients and their clinicians (questionnaires).

Results

Of 96 eligible patients, 13 agreed to participate (13.5%). One patient dropped out after 2 days, another after 37 days, and another did not complete the cognitive test battery. Compliance rate was high (6.7-10% missing values). Subjective burden was relatively low. Time-series data showed sufficient variability and no floor- or ceiling effects, except for one relevant ceiling effect on the One Back task. The personal feedback report was considered insightful by 4 out of 11 participants and 5 out of 7 clinicians.

Conclusion

Daily assessments are suitable for a minority of cognitively impaired older persons, but is helpful to increase insight into their symptoms.

Keywords:

• Digital monitoring
• ambulatory assessment
• actigraphy
• single-subject
• idiographic

Introduction

Intensive longitudinal ambulatory assessments using diaries and wearables are promising to achieve a more personalized approach in mental health care and research (Reichert et al., 2021; Zuidersma et al., 2020). Daily ambulatory diary assessments can yield detailed insight into patterns and relation over time of behaviors, symptoms and contextual factors with high ecological validity (Ebner-Priemer & Trull, 2009). Such assessments provide 1) in-the-moment information, reducing retrospective bias, 2) increased reflections and insights in patient’s psychopathology by intensive self-monitoring, and 3) the resulting time-series data can be summarized into personalized feedback reports that contain verbal, written and graphical illustrations of relations of activities, events and social interactions in daily life with affective responses (Bastiaansen et al., 2020a; Kramer et al., 2014). In this way, the daily assessments can help support the case conceptualization process, during which patients and clinicians closely collaborate on a working theory of the patient’s psychopathology (Riese et al., 2021). Results from such ambulatory diary assessments can be used for tailored treatment or behavioral advice in order to increase self-management (Bastiaansen et al., 2020a; Kramer et al., 2014; Van Roekel et al., 2017). While most studies typically used standardized diaries in which the same items are used for all patients, but a recent study has incorporated personalized diaries where the patient and clinician select the items (Riese et al., 2021). Ambulatory assessments may also increase ecological validity for cognitive performance assessments: in standardized laboratory settings some participants may perform better on cognitive tasks due to social facilitation (Strauss, 2002), while others may score worse on cognitive tasks due to the stereotype threat effect (Schmader et al., 2008).

Particularly in old age psychiatry, ambulatory assessments to personalize clinical care and research approaches are important, because 1) treatment efficacy for mental disorders decreases with age (Schaakxs et al., 2018) and 2) diversity between patients increases with age due to age-related somatic comorbidity, physical frailty, cognitive impairment, and age-related psychosocial adversities (Oude Voshaar et al., 2019). Yet, intensive ambulatory assessments have mainly been applied in younger patients (Bastiaansen et al., 2020a; Kim et al., 2020; Kramer et al., 2014; Van Roekel et al., 2017).

A potential reason for the lack of studies in older persons is that the feasibility of diary studies with intensive repeated assessments in older persons is often questioned. Older persons, and particularly those with cognitive impairments, report more difficulties with technologies (Jekel et al., 2015). This may negatively affect the response and compliance in intensive diary studies, and therefore their feasibility.

Some previous studies have shown that ambulatory diary assessments are feasible in older persons when administered over relatively shorter time periods (Bartels et al., 2020; Maher et al., 2018; Moore et al., 2016; Ramsey et al., 2016). For predictive within-person modeling, however, at least 50 assessments are needed (Lütkepohl, 2006). The time interval between these assessments depends on a priori expectations about the lag between the associations, but also on how often the assessed variable occurs. For example, when assessing nighttime sleep (which obviously occurs only once a day), the study must include at least 50 assessment days. According to our knowledge, no study evaluated the feasibility (i.e. response rate, compliance and burden) of such long periods of diary assessments in older persons with cognitive impairments.

Previous studies have shown that up to 100 daily assessments of cognitive functioning is feasible in healthy younger and older adults (Allaire & Marsiske, 2005; Salthouse & Berish, 2005; Schmiedek et al., 2010; Verhagen et al., 2019). Nonetheless, feasibility among older adults with cognitive impairments has not been examined yet. In addition, sensors might be particularly useful in gathering daily time series data using intensive repeated assessments for older persons with cognitive impairments. Actigraphy is a valid and reliable way to assess sleep disturbances based on movement data obtained by a wristwatch (Ancoli-Israel et al., 2003; Fekedulegn et al., 2020). It uses validated algorithms to estimate several sleep parameters, such as the sleep efficiency, total sleep time or sleep onset latency and is typically used for periods up to 14 days. Few studies have assessed the feasibility of actigraphy in older persons with cognitive impairments for longer periods. One study demonstrated feasibility of actigraphy for 63 days in 5 nursing home residents with dementia (Wijbenga et al., 2021).

In addition to the feasibility of intensive longitudinal assessments, it is also important to know whether the data is usable for analyses at the within-person level. First, there must be sufficient variability in the measured construct. A variable or construct that does not vary enough over time in a specific person is not relevant for dynamic modeling at the within-person level (Brose & Ram, 2012). Second, floor- or ceiling effects may indicate that a variable is not suitable for measuring the underlying construct.

Finally, no study evaluated the clinical value of personalized feedback in older persons with cognitive impairments. That is, the personalized feedback may help patients and therapists in identifying personal predictors of symptoms, and consequently defining a proper treatment or behavioral advice. Older persons with late-life depression and cognitive impairments are a vulnerable group at risk of losing functional independence, nursing home admission, and adverse health outcomes (Mourao et al., 2016; Xiang & An, 2015). Therefore, ambulatory assessments to identify daily triggers of important health outcomes, such as sleep, depressive symptoms and cognitive functioning might be particularly important in this group to enhance independence and daily functioning. In addition, self-management is likely reduced in older persons with cognitive impairments, as the cognitive deficits negatively impact feelings, behaviors, self-perception, and social interactions (Parikh et al., 2016). Therefore, increasing self-management by offering detailed insight in daily patterns of behaviors, symptoms and contextual factors may be particularly important for this group.

In this study we assessed the feasibility, usability and clinical value of daily diary assessments including a neurocognitive test battery, and combined with actigraphy over a period of 63 days in older persons admitted to an outpatient psychiatric or memory clinic with clinically relevant depressive symptoms and cognitive impairment.

Methods

Study design and participants

The ‘idiographic study on cognition affect and sleep in the elderly’ (i-CASE) adopted a single-subject study design. Older persons were recruited between January 2016 and January 2019 from the University Medical Center Groningen, and a mental health care institute in Groningen (Lentis), the Netherlands (Zuidersma et al., 2021). For 63 consecutive days (9 weeks) participants filled out a standardized diary on a laptop twice a day and they wore a wrist actiwatch.

Eligibility criteria were: age ≥60 years, a Geriatric Depression Scale (GDS) score ≥3 or major depressive disorder according to Diagnostic Statistical Manual (DSM) criteria, cognitive impairments according to the team of the memory clinic or Mini Mental State Examination (MMSE) score <25 or Montreal Cognitive Assessment Score <26. Exclusion criteria were presence of a somatic disorder influencing short-term survival, a clinical dementia rating scale >1, bipolar-, psychotic- or substance use disorder during the past 2 years, unable to participate (due to for example functional or language impairments), or being mentally incompetent to give consent for study participation.

The study protocol was approved by the institutional review board at the University Medical Center Groningen, and written informed consent was obtained from each study participant (METc 2013/019). Participants received a financial incentive of 90 euros for participating within several weeks after the study end.

Procedures

A research assistant administered the baseline assessment and instructed participants regarding all study procedures. Hereafter, participants continuously wore the actiwatch and filled out a standardized diary on a laptop twice a day: once in the morning (within 1 h of waking up) and once in the evening (within 1 h before going to bed). After 3 days, the research assistant called the participant to evaluate and support adherence to the study procedures.

During the whole study period all study participants could continue their therapy and medical treatment as usual. The research assistant was 7 days a week available by mobile phone for participants who had any problems or questions. If necessary, the research assistant visited the participant to solve any problems. The research assistant contacted each participant once a week to assess changes in treatment or medication and relevant events with impact on results, and evaluated whether the battery of the actiwatch was still working. On the last day, an extra questionnaire was prompted directly after the morning diary including questions about how the participant experienced the study.

At one-month follow-up, the researcher discussed the personal feedback report with the patient at the participants home. This report showed descriptive outcomes, such as patterns over time, means and range of depression/affect, sleep and cognitive performance. It also showed their values at moments when the participant reported a relevant event (such as having the flu or a holiday). In addition, it showed temporal associations (from one day to the next) between depressive symptoms/affect, sleep and cognitive function and their association with daily behaviors (physical activity, social interactions) and relevant events. These temporal associations were derived from vector autoregressive modelling (Zuidersma et al., 2021).

At two-month follow-up, the participant filled out a questionnaire asking whether the study or the report helped to give insight into potential triggers of their depressive symptoms, sleep problems and cognitive performance. In addition, it asked whether they discussed the report with their clinician.

At 3-month follow-up, the clinician of the participant also received the personal report of the patient, which was discussed with the researcher. At 4-month follow-up, the clinician received an insight-questionnaire asking whether the report gave them insight into the potential triggers of the depressive symptoms, sleep problems and cognitive performance of the participant (see Figure 1).

Figure 1. Time schedule of whole study.

Assessments/measures in the morning diary

The morning diary consisted of 4 question(naire)s and took about 2-3 min to complete.

• Sleep disturbances during previous night: The Pittsburgh Sleep Diary (morning-version; (Monk et al., 1994)) was administered to evaluate sleep habits, including a sleep diary assessing sleep and wake times, and disturbances and subjective sleep quality during the previous night.

• Sleeping-pill use: Single question asking whether the participant used a sleeping-pill for last night’s sleep.

• Depressive symptoms: Momentary severity of depressive symptoms was assessed with the 8-item Patient Health Questionnaire (PHQ-8; (Kroenke et al., 2009)). The PHQ-8 is a brief and well-validated measure to evaluate current presence and severity of eight out of the nine DSM-IV depressive symptoms. In the present study, each item was rated from 1 (absent) to 7 (most severe), so the total score of the PHQ-8 in the morning diary could range from 7 to 56.

• Affect: Momentary affect was assessed with 6 positive affect (PA) items (relaxed, energetic, enthusiastic, content, calm, cheerful) and 6 negative affect (NA) items (gloomy, anxious, nervous, irritable, dull, tired) as consistent with the circumplex model of affect (Yik et al., 1999). Items were rated on a visual analog scale from 0-100. The 6 PA and the 6 NA items were averaged to create a positive affect (PA) and a negative affect (NA) scale, each with a possible range of 0-100.

Assessments/measures in the evening diary

The evening diary included several questionnaires, and a cognitive test battery and took about 20 min to complete. The content was as follows:

• Subjective memory complaints: Single question asking whether the participant experienced memory problems during the day with 3 possible answer categories: no, yes but I have no worries about it, yes and I am worried about it.

• Depressive symptoms: Momentary depressive symptoms were assessed as in the morning diary, except the item about sleep was removed as this was already asked out in the morning diary. This resulted in 7 items being asked on a scale from 1 (absent) to 7 (most severe), resulting in a possible total score from 7 to 49.

• Affect: Momentary positive and negative affect were assessed as in the morning diary.

• Sleepiness: The Karolinska Sleepiness Scale (KSS; (Åkerstedt & Gillberg, 1990)), a one-item scale asking ‘how sleepy were you today’, was used to rate sleepiness during the day on a scale from 1 (not sleepy at all) to 9 (very sleepy).

• Naps: Single question asking the total number of minutes spent on napping.

• Physical activity: The International Physical Activities Questionnaire (IPAQ; (Craig et al., 2003)) was administered to assess the total number of minutes spent during that day on sitting, walking, moderate physical activities and intensive physical activities.

• Consumption: The number of cigarettes/pipes/cigars, alcoholic drinks and caffeine drinks consumed during the day.

• Social interaction: The number of social contacts during the day, and the pleasantness of these contacts on a scale from 1 to 7.

• Pain: The severity of experienced pain during the day from 1 to 7.

• Actiwatch: Single question asking whether participant took off the actiwatch during the day. If yes, free text to fill out at what time and for how long.

• Important events: Free text to fill out anything (pleasant, neutral or unpleasant) that happened during the day that could have influenced their sleep, depression and cognitive function.

• Cognitive test battery: the Cogstate Brief battery, a computerized cognitive test battery developed for unsupervised repeated testing of individuals, resulting in valid measure of cognitive functions with minimal practice effects (Falleti et al., 2006). It includes four culturally independent game-like playing card tasks in order to assess psychomotor reaction time, attention, visual learning and memory, and working memory (Hammers et al., 2012). Psychomotor reaction time was assessed with the detection task (DET), asking to press the ‘yes’ button as quickly as possible if the playing-card shown on the screen is turned face-up (2 minutes). Attention was assessed with the Identification Task (IDN), asking to press the ‘yes’ button as quickly as possible if the card shown on the screen was red (and do nothing if the card was black; 2 minutes). Visual learning and memory were assessed with the One Card Learning task (OCL), asking to press the ‘yes’ button if the card shown on the screen has been shown before during this trial, and the ‘no’ button if not (5 minutes). Working memory was assessed with the One Back Task (OBK), asking to press the ‘yes’ button as quickly as possible if the card shown on the screen was the same as the previous card, and the ‘no’ button if the card was not the same (3 minutes). For DET and IDN, the reaction times for correct responses were the primary outcomes. For OBK and OCL, the accuracy measure was the primary outcome (Hammers et al., 2012).

Actiwatch

Participants wore a MotionWatch8 (CamnTech) during the whole study period of 63 days in order to estimate sleep. Actigraphy is a previously validated non-invasive method providing information about sleep- and wake patterns through an accelerometer that detects wrist movement (Ancoli-Israel et al., 2003; Fekedulegn et al., 2020). From actigraphy, we derived total sleep time (TST): the total time during the sleep period that the person was actually asleep.

Feasibility for the patient to participate

The feasibility was investigated by assessing response rate, compliance, and personal experiences. The response rate was defined as the proportion of the eligible patients who agreed to participate. Compliance was expressed as the proportion of the assessment points that were filled in. This was regarded as satisfactory when >70% of the assessments were filled in or, in case of actigraphy data, obtained (Bartels et al., 2020). We evaluated the personal experience of using the diary and wearing the Motionwatch with a self-developed questionnaire after the end of the study.

Usability of the daily assessments for the researcher

Variability of the main outcome parameters

We evaluated whether the variability was sufficient for the items of the main outcome parameters of the present study (depressive symptoms, affect, cognitive performance and sleep (Zuidersma et al., 2021)). We defined sufficient variability if: 1) for a specific person <80% of the scores over time were the same (Brose & Ram, 2012), and 2) if the intra-individual variability exceeded 10% of the known between-subject variability (Brose & Ram, 2012). For this latter purpose, we calculated the root-square of the mean squared successive differences (rMSSD; (von Neumann et al., 1941)) for these variables for each participant. The MSSD is a measure of intra-individual variability and accounts for temporal trends (Jahng et al., 2008). It is obtained by averaging the squares of each successive time difference between assessments, which is square-rooted for direct comparability with the between-person SD of the scale. In this way, the MSSD does not get inflated by time trends, such as learning effects. Sufficient variability was determined when the rMSSD for a specific person was at least 0.10 of the known between-person SD of that scale (Brose & Ram, 2012, see Supplemental file for how we derived the between-person SD of each outcome).

Floor- and ceiling effects

Floor- or ceiling effects of the main outcome parameters (depressive symptoms, affect, cognitive performance and sleep) were considered present if >15% of the assessments were at the maximum or minimum level of the scale (McHorney & Tarlov, 1995).

Clinical value for the patient and his/her clinician

The personal feedback report contained information about courses over time of sleep, depressive symptoms and cognitive performance, and their relation with daily events, activities and social interactions. With the insight-questionnaires we evaluated clinical value of the personal feedback report for patients and clinicians. The questionnaire for patients included multiple choice questions asking whether participating in the study or the personal feedback report gave them insight into the triggers of their complaints, and whether it improved their complaints. This was done separately for depressive symptoms, sleep and cognitive performance. An additional question asked whether the patient discussed the personal report with the clinician. The questionnaire for the clinician included multiple choice questions asking whether the personal report gave them insight into the triggers of the complaints of the patient, and whether it gave them clues for the treatment of the complaints. This was done separately for depressive symptoms, sleep and cognitive performance.

Results

Feasibility for the patient to participate

Response rate

During the inclusion period between January 2015 and June 2019, 96 eligible patients were asked for participation. Of these, 83 did not want to participate or did not respond to the invitation letter, and 13 started the study (13.5%). For two participants (participants 1 and 3), the depressive symptoms were in remission before the start of the study, and one participant (participant 12) appeared not to fulfill criteria for cognitive impairments by hindsight. Nevertheless, these 3 participants are included in the present results as feasibility data is present for them.

Compliance

Of the 13 participants, one participant (Participant 13) stopped after 2 days, because it appeared too difficult to fill out the diaries. Another participant (Participant 2) stopped after 37 days because it was too burdensome. Of the 11 participants who completed the study, one participant (Participant 6) found the cognitive test battery too burdensome and therefore stopped the cognitive tests after 10 assessments, but completed all other measurements for 63 days. In the 12 participants with a total time series length of 772 assessments, 52 (6.7%) of the possible morning diaries were missing. This was 53 (6.9%) for the evening diaries, and 130 (16.8%) for the cognitive tests (this was 77 (10.0%) when deleting Participant 6), and 54 (7.0%) for the Motionwatch data. So, compliance rates were all regarded as satisfactory (i.e. >70% of assessments were present). Most frequently, missing values were due to technical problems (i.e. failure of the laptop or dead battery of the Motionwatch, see Table 1 for details).

Table 1. Time series length, amount of missing data and reasons for missing data for each participant.

Participant	Time-series length	Reason deviation study-length from 63	N missing morning diary	N missing evening diary	N missing cognitive tests	N missing motionwatch	N missing per person
1	53	Holidays	2^1	1^1	1^1	6^3	10/212 = 4.7%
					2^2
2	37	Too much burden	4^2	4^2	4^2	0	12/148 = 8.1%
3	63		1^2	1^2	1^2	1^2	4/252 = 1.6%
4	94	Lost motionwatch for a while, and wanted to compensate	2^2	2^2	4^2	21^3	40/376 = 10.6%
						11^4
5	63		1^2	2^2	2^2	0	5/252 = 2.0%
6	63		1^5	1^5	53^6	0	56/252 = 22.2%
			1
7	63		3^2	2^7	2^7	0	10/252 = 4.0%
			2^7
			1^8
8	63		10^2	7^2	13^2	9^3	42/252 = 16.7%
						3^9
9	63		0	3^2	1^2	0	16/252 = 6.3%
					12^10
10	77	Wanted to go on longer to account for missing values due to laptop failures	9^1	11^1	11^1	2^3	34/308 = 11.0%
						1^4
11	70	Study end was during holidays, and participant wanted to end study after holidays	4^1	5^1	5^1	0	51/280 = 18.2%
			4^2	5^2	10^2
			2^11	8^11	8^11
12	63		1^6	1^6	1^6	0	7/252 = 2.8%
			3^2
			1^12
total	772		52 (6.7%)	53 (6.9%)	130 (16.8%)	54 (7.0%)

¹Laptop failure, ²Reason missing unknown ³Motionwatch battery dead, ⁴Lost Motionwatch, ⁵Received no link to the questionnaire, ⁶Found cognitive tests too burdensome, so did not proceed, ⁷Family or friend visit, ⁸Forgot to fill out, ⁹Did not wear Motionwatch, ¹⁰Could not start cognitive test battery, ¹¹No WiFi in the evening on the camping, ¹²Could not fill out because of grandchildren staying over.

Burden for- and experiences from participants

One participant (Participant 2) who stopped earlier, did not fill out the evaluation questionnaire at the end of the study, resulting in 11 participants that filled it out. Most participants found the study not or only little burdensome, with a few reporting moderate burden, and none reporting high burden (Table 2). None of the participants indicated burden from the motionwatch. Participant 7 rated the study as moderately burdensome (5 out of 7), found the diary questions too confronting and found it difficult (for filling in such personal information) that he did not know the principal investigator. Participant 6 found the study moderately burdensome (4 out of 7), and also rated the cognitive tests as burdensome (this was also the patient that stopped the cognitive tests). None of the remaining participants indicated problems in multiple domains and most questions were answered generally positive by most participants (see Table 2).

Table 2. Burden and subjective experiences of participants associated with the study.

	N = 11 participants (participant 2 did not fill out)
Burden study
1 no burden at all	3 (participants 3, 9 and 10)
2	4 (participants 1, 5, 8 and 11)
3	1 (participant 4)
4	2 (participants 6 and 12)
5	1 (participant 7)
6	0
7 very burdensome	0
Length of study
I could have done this longer	2 (participants 4 and 10)
Exactly good	7 (participants 1, 3, 5, 6, 7, 8, and 9)
Too long	2 (participants 11 and 12)
Diary questions	Participants 1, 6, 8, 9 and 10 reported no problems with specific questions
	Three participants (participants 3, 5 and 12) had difficulties with the sleep questions (the scroll down menu was confusing and it was difficult to remember the exact bed times)
	Participant 4 found the appetite question difficult. He had no appetite, but that was not a burden for him. So, it was confusing how to answer it.
	Participant 11 said it was annoying that it concerned the same questions all the time
	Participant 7 found the questions confronting, the answer options arbitrary, difficult to answer exactly how he felt
Difficulty cognitive tasks
Too easy	1 (participant 8)
Exactly good	8 (participants 1, 3, 5, 7, 9, 10, 11 and 12)
Too difficult	2 (participants 4 and 6)
Burden cognitive tasks
Not burdensome, could have done more	1 (participant 10)
Exactly good, but no more should have been asked from me	9 (participants 1, 3, 4, 5, 7, 8, 9, 11 and 12)
Too burdensome, was too much	1 (participant 6)
Experiences cognitive tasks	Participants 4, 5, 8, 9 and 12) found the third task (OCL) difficult.
	Participants 4 and 8 found the tasks confronting.
	Participant 10 found it confusing that there was a practice session before the third task (i.e. difficult to distinguish the cards remembered from the practice session from those of the real session).
Motionwatch experience
No nuisance at all	8 (participants 1, 3, 4, 5, 7, 9, 10, 11)
I felt it a bit, but was not a nuisance	3 (participants 6, 8, 12)
I felt it and it was a nuisance	0
Experiences motionwatch	Participant 8 had eczema, that made her feel the motionwatch. But it was not a nuisance.
Frequency of contact with research assistant
Not enough	0
Exactly good	10 (participants 1, 3, 4, 5, 6, 8, 9, 10, 11, 12)
Too much	1 (participant 7)
Experiences contact with research assistant	Participants 4, 8, 9, 10 and 12) said it was very pleasant
	Participant 7 said he did not know the principal investigator, which made him feel uncomfortable sharing his feelings.
Positive experiences with the study	Participants 4 and 5 indicated that it made them feel good to have helped others by participating in the study
	Participants 7 and 10 indicated that it gave them insight in their mood fluctuations over the day
	Participant 8 indicated it helped her to learn to focus and have patience
	Participant 1 indicated that the study helped in being more conscious of matters
Negative experiences with the study	Participant 6 indicated it was somewhat a burden
	Participant 1 indicated that initially the evening diary was too late in the evening, but this could be adapted during the study.
	Participant 8 indicated that she had to remember to be home early to fill out the evening diary (as the evening diary was available until 2 pm)
	Participant 10 had a laptop that sometimes malfunctioned, which resulted in some missing assessments, which was annoying for her

Usability of the data for the researcher

Within-person variability

For participants 9 and 10 > 80% of the One Back Task scores were the same (i.e. scores of 100% accuracy). For the other variables and participants, the frequency of the mode was always smaller than 80%. The rMSSD of all variables exceeded 0.1 of the known between-person SD for all persons (see Table 3). For cognitive performance, we describe elsewhere that the intra-individual variability exceeds at least 0.5 SD of the between-person variability for most persons (Zdanowski et al., 2022). In fact, for none of the participants the rMSSD of the cognitive measures was lower than 0.10 of the between-person SD (Zdanowski et al., 2022).

Table 3. Participant-specific rMSSD and between-person SD of sleep, depressive symptoms and affect.

	Total sleep time (TST), hours	Depressive symptoms morning diary (PHQ-8)	Depressive symptoms evening diary (PHQ-7)	PA morning diary	NA morning diary	PA evening diary	NA evening diary
1	0.84	1.82	1.21	X	X	X	X
2	1.68	3.33	3.31	X	X	X	X
3	1.29	3.95	2.72	X	X	X	X
4	1.67	4.07	2.25	5.14	5.85	6.16	6.52
5	0.82	6.58	2.66	7.04	7.92	6.12	6.43
6	1.33	7.09	10.27	15.72	9.80	27.80	20.34
7	0.72	2.70	3.28	5.02	8.42	7.58	6.35
8	1.69	8.16	5.90	27.36	22.42	18.43	17.57
9	1.10	2.75	2.37	7.25	3.22	8.71	5.07
10	1.47	3.93	5.59	13.52	8.39	19.35	10.46
11	1.59	4.82	4.69	11.12	10.92	7.35	10.95
12	1.30	5.46	3.54	13.01	9.67	11.55	9.01
Between-person SD^1	0.86	9.66	8.45	15.96	10.91	15.96	10.91

X = not assessed.

1 See Supplemental file how we derived the between-person SD. For cognitive performance, see Zdanowski et al. (2022).

Floor- and ceiling effects

For all but one participants the One Back task showed ceiling effects (e.g. >15% of the values at the ceiling). Specifically, the percentage of test values at the ceiling (100% accuracy) was 38%, 21%, 38%, 38%, 18%, 44%, 88%, 88%, 37% and 37% for participants 1, 3, 4, 5, 7, 8, 9, 10, 11 and 12 respectively. None of the other main outcome parameters showed floor- or ceiling effects.

Clinical value for the patient and his/her clinician

Each patient received a personal feedback report indicating which daily events, activities and social interactions preceded changes in their sleep, depressive symptoms and cognitive performance. These results differed between patients. For example, for one patient (Participant 8), worse self-reported sleep quality preceded depressive symptoms the following day. For another patient (Participant 11), more walking during the day preceded less depressive symptoms in the evening.

Eleven participants returned the insight questionnaire. Four of them (Participants 6, 8, 9 and 12) indicated that they got more insight into their sleep-, mood- or cognitive problems due to the study or the feedback report. One (Participant 12) indicated that the personal report helped improve the complaints. Two (Participants 2 and 5) indicated that they shared the study results with their clinician, but none indicated that this has led to another (treatment)advice. Seven clinicians returned the insight questionnaire. Five of them (from Participants 2, 4, 5, 7 and 11) indicated that the feedback report gave them more insight into the sleep-, mood-, or cognitive problems of their patient, and that the feedback report also gave them clues for treatment options.

Discussion

Principal findings

This study evaluated the feasibility, usability and clinical value of daily ambulatory diary assessments and actigraphy in older persons with depressive symptoms and cognitive impairments. Several important findings emerged: (1) only 13.5% of eligible older persons with cognitive impairment were willing to participate, (2) of those who participated, 3/13 (23.1%) had difficulties adhering to the protocol, (3) the compliance rate was high among study completers and subjective ratings of the study procedures were positive, (4) most frequently, missing values were due to technical problems, (5) data on sleep, depressive symptoms, affect and cognitive performance showed sufficient daily fluctuations, (6) there were no floor or ceiling effects in sleep and affect data, nor in cognitive tasks, except in the accuracy scores of the One Back task. Finally, the personal feedback report appeared useful for some of the participants (4 out of 11) and their clinicians (5 out of 7). Taken together, only a minor part of the older persons with cognitive impairments and depressive symptoms admitting to a memory clinic or psychiatric care are willing to participate in intensive longitudinal ambulatory assessments, but for some patients and their clinicians it can help them give insight into their complaints. There are several practical issues which we will discuss below including possible suggestions for future studies.

Feasibility for the patient to participate

Low participation rate

The low participation rate of 13.5% may have several causes. First, the moment of asking for participation was at the same time that many patients also received the message they have been diagnosed with a cognitive disorder. This might have impacted the patients so that they were not capable of participating at this moment. Second, patients may see the 63 daily assessments as too much burden. The cognitive test battery comprised the heavy part of the diary assessments. It took 12 min to complete. Future studies may use a less heavy cognitive battery, but only if that battery is suitable for unsupervised administration, is sensitive to real fluctuations in cognitive function, has multiple alternating versions of a task, and has minimal practice effects. Third, working with a laptop may have demotivated patients. For future studies, one should consider a device more easy to use like a smartphone or tablet with a touch-screen.

Missing data due to technical problems

These problems were mostly related to loss of internet connection, and failure of the laptop (mostly because the laptop needed to install updates, and the participant did not know how to handle this). This may be solved in future studies by using a device that works for a longer period without any problems, and diaries and tests that can be done offline. Important to know is that our research assistant was available 7 days a week and promptly solved the technical problems, so missing data was reduced to only one or a few assessments per person. For the actigraphy, missing data occurred due to battery failure. The battery had to be replaced by the researcher after reading out the data. Future studies making use of actigraphy for longer periods must look for devices with long battery life.

Usability of the data for the researcher

All measures for depressive symptoms, affect, cognitive performance and sleep fulfilled criteria for within-person variability (i.e. within-person rMSSD > 0.1 between-person SD). The accuracy measure of the One Back task showed ceiling effects in all but one participant. This suggests that the accuracy on this task is not suitable for this group of patients, although it might be more suitable for patients with more severe cognitive problems. For the current patient group a reaction time measure of this task might be more suitable, as this is less prone to ceiling effects.

Clinical value

Our findings suggest that personal feedback differs between patients and providing personal feedback might be helpful for some patients and clinicians. This is in accordance with other similar studies that also showed that patients perceived personal feedback to be helpful (Bastiaansen et al., 2020a; Bos et al., 2020; Kramer et al., 2014; Van Roekel et al., 2017). However, there are several obstacles that need to be overcome before single-subject study results can be implemented in clinical practice. First, results of VAR analyses must be interpreted with caution, as they do not infer causality, they do not take into account changes in associations over time, they yield different results when looking at different time intervals, and they are subject to specific analytic choices (Bringmann, 2021). The impact of specific analytic choices has been demonstrated in a study in which data of one patient has been analyzed by 12 different research groups yielding different outcomes (Bastiaansen et al., 2020b). These obstacles might explain why not all patients in our sample found the feedback useful, and they should have been solved before implementing these complex analytic methods in clinical care. Until that time, we can provide feedback on simple and descriptive patterns and associations only, which is also perceived as helpful (e.g. (Bos et al., 2020; Kramer et al., 2014)). Another obstacle before implementation in clinical care can take place is that pre-processing of data and analysis are often a burden for researchers and particularly clinicians. Therefore, it would be ideal if pre-processing of data and analyses can be performed automatically. As such, packages for automatic time-series analysis have been developed (AutoVAR; (Emerencia et al., 2016)), as well as a package to automate preprocessing of actigraphy data into workable circadian rhythm variables (ACTman; (Kunkels et al., 2020)), and the possibility to integrate data from sensors and diary data (Blaauw et al., 2016).

Methodological considerations

The making of the personal reports was not done according to standardized algorithms. For different participants different variables were included which may have biased the findings. However, choices were mainly based on the level of variation over time, and excluding those with little variance may not impact on individual results. Another important issue in all single-subject diary studies is that psychometric properties of a questionnaire obtained from group-based results can be different for a specific person. Therefore, it remains unclear to what extent fluctuations are due to real fluctuations or due to random fluctuations (noise). Another point worth mentioning is that in the present study standardized diaries were used with the same set of questionnaires for all patients. Personalized diaries, in which patient and clinician select the most relevant questionnaires, may further improve clinical value because these can be tailored to the patients’ needs and situation (Riese et al., 2021). Finally, because we only included participants with mild cognitive impairments, these results cannot be generalized towards patients with severe cognitive impairments. It may be expected that feasibility of daily diary assessments is lower in persons with severe cognitive impairments. Also, participant 12 did not have cognitive impairments in hindsight and was the only participant mentioning that the study or personal feedback report improved the complaints. Therefore, based on the present study we may not conclude that intensive monitoring and a personal feedback report improve complaints of persons with cognitive impairments.

Conclusion

The present study protocol with intensive repeated diary assessments including questionnaires and a cognitive test battery combined with actigraphy is suitable for a minor subgroup of older persons with depressive symptoms and cognitive impairment. This yields data suitable for scientific research in order to get more insight into associations at the level of the individual person. In addition, for some patients such data can help patients and clinicians to get better insight into their symptoms. Future studies should evaluate the implementation of such algorithms on larger scale and their effectiveness in identifying personal predictors of symptoms, and consequently defining a proper treatment or behavioral advice. A summary of recommendations for future studies can be found in the Box.

Box: Recommendations for future studies

• Research assistants must be available 7 days a week to solve technical or other problems participants have.

• A minority of patients may find the diary questions confronting (particularly if they are suffering from a depressive disorder and are asked about their symptoms every day).

• Make sure the patient understands how to interpret the questions.

• Let participants get acquainted with the principal investigator(s), in order to avoid the participant feeling uncomfortable sharing their feelings.

• When approaching patients just after dementia diagnosis for a study, it is better to wait with approaching them until a next clinical appointment. When recruiting persons with cognitive impairments from a memory clinic, be aware that the heavy message of a possible cognitive disorder might demotivate the person to participate in a study.

• Use question(naire)s that have enough intra-individual variability and no floor- or ceiling effects.

• Make sure the cognitive task is difficult enough to avoid ceiling effects. The more cognitively healthy your sample, the more difficult the task should be. Tests evaluating reaction times are less vulnerable to floor- or ceiling effects than tests evaluating accuracy.

• Minimize burden of a cognitive test battery without losing validity: look for a short but sensitive cognitive test battery that can be used in unsupervised settings with minimal practice effects.

• Use an actiwatch that has sufficient battery life.

Acknowledgements

We would like to thank all the participants of this study for their participation.

Disclosure statement

No potential conflict of interest was reported by the authors.

Conflict of interest

Richard Oude Voshaar is associate editor of Aging & Mental Health. Further, we declare no conflicts of interest.

Data availability statement

The data used in this study can be provided upon request.

Additional information

Funding

This work was supported by Stichting MIND (project code 2013 6740) and Alzheimer Nederland (project code WE 09-2014-01).