Physician estimated vs. self-reported subjective memory in depressed patients treated with electroconvulsive therapy

Abstract

Background: Subjective memory deficits are common in depression and during series of treatment with electroconvulsive therapy (ECT). There is a need for feasible assessment of memory deficit. In the Swedish National Quality Register for ECT, patients’ subjective memory function is rated by a clinician. Self-ratings would be easier to administer.

Objectives: The aim of this study was to analyze the consistency between self-reported and physician estimated subjective memory in depressed patients treated with ECT.

Methods: Fifty-two inpatients treated with ECT for major- or bipolar depression were recruited and 41 of them completed the study protocol. Each patient rated their own subjective memory and had it rated in an interview by a physician both before/in the beginning of the ECT series and after the ECT series. The patients’ memory was rated and self-rated with the memory item in the Comprehensive Psychopathological Rating Scale (CPRS). We then analyzed correlations, and differences in distributions, between self-reported assessment and physician estimates of patients’ subjective memory.

Results: The correlations between the self-reported and the physician estimated ratings of subjective memory were 0.699 (p < .01) in baseline ratings and 0.651 (p < .01) in post-treatment ratings. These correlations were relatively high compared to a previous study on self-reported vs. physician estimated CPRS ratings.

Conclusions: Based on the results in this study, we propose that patients’ self-ratings of subjective memory in association with ECT can be used instead of a physician’s rating of patients’ subjective memory.

Keywords:

• Electroconvulsive therapy
• subjective memory
• self-rating scale

Introduction

Major depressive disorder is one of the leading causes of disability worldwide [1] and the average lifetime prevalence has been reported to be 15% in high-income and 11% in low- to middle-income countries [1]. The treatment options for major depressive disorder include psychotherapy, antidepressants and, for severe depression, electroconvulsive therapy (ECT) [2]. In Sweden, ECT-related data are collected in the Swedish National Quality Register for ECT (Q-ECT). About 3973 individuals received ECT in Sweden and were included in the Q-ECT during the year 2016 [3].

ECT is considered the most effective antidepressant treatment [4]. One characteristic limiting the use of ECT is its cognitive side effects [4] and transient memory disturbance is considered a common side effect of ECT [5]. Up to three days after ECT, medium to large deficits in episodic memory can be expected. Most patients recover to baseline within two weeks. After 15 days many cognitive functions have been shown to improve beyond baseline levels in most subjects, although there is a considerable interindividual variability [6]. Some ECT protocols (bilateral electrode placement, three treatments per week and higher doses) tend to be associated with more cognitive side effects, primarily anterograde and retrograde amnesia [7].

A complicating factor in the assessment of memory effects due to ECT is that memory deficit is a characteristic of major depressive disorder per se, and for many patients, this persists beyond recovery from mood symptoms [8]. For example, a study of depressed inpatients showed pronounced subjective complaints of deficits in memory at admission and these complaints were considerably less at time of discharge [9]. It has thus been suggested that subjective memory partly derives from current memory performance but also from other variables such as depressive symptoms [10].

In order to better understand the prevalence and severity of cognitive side effects associated with ECT, the Q-ECT [11] includes data of patients’ subjective memory before and after each ECT series [11]. In this register, subjective memory is rated using the memory item in the Comprehensive Psychopathological Rating Scale (CPRS), CPRS-M, [12] by a medical professional. Up until now no self-rating version of CPRS-M has been validated.

It would be beneficial if physicians’ ratings of patients’ subjective memory could be replaced with patients’ self-reported ratings of subjective memory. At least the following benefits can be expected:

• It would facilitate logistics: Rating of subjective memory could be done after the discharge of patients and it would be time-saving for the physicians at the ward.

• It would make follow-up ratings of subjective memory by mail possible, enabling higher temporal resolution of subjective memory impairments following ECT.

• It would be neutral in the sense that patients do not rate their subjective memory in an interview with their attending physician.

• It would be an effective method to avoid inter-rater variability which is expected to introduce noise in the CPRS data in the Swedish Q-ECT.

The primary aim of this study was to analyze the consistency between self-reported and physician rated subjective memory in depressed patients before and after ECT treatment. This is important for the ability to compare self-ratings with previous reports using clinician’s ratings [13]. The hypothesis of the study was that there is a high and significant consistency that supports using patients’ self-reported subjective memory in clinical practice and in the Swedish Q-ECT. The secondary aim was to explore the potential confounding influence of differences in demographic variables, illness severity, rater, and ECT administration.

Materials and methods

The study was approved by the regional ethical review board, Stockholm, Sweden. All study procedures took place after oral and written consent.

Subjects were recruited from Northern Stockholm Psychiatry, an inpatient clinic in Stockholm, Sweden, between the years 2013 and 2015. All subjects were planned to start or had recently started ECT treatment for major depressive disorder or for bipolar depression. Fifty-six patients consented to participate and started to follow the study protocol. The study was observational, i.e. it did not interfere with standard treatment routines such as medication, ECT treatment protocol or other factors of treatment or care of the patients.

All subjects referred for ECT were considered for inclusion. The inclusion criterion was that the subject was suggested for treatment with ECT by their physician, due to major depression or depressive episode of bipolar disorder, as defined in ICD-10 [14]. Exclusion criteria were alcohol or substance abuse, dementia, schizophrenia, Parkinson’s disease, acquired brain injury, mental retardation, intellectual disability, and having received ECT during the last two months.

The CPRS [12] consists of 65 items measuring psychopathological characteristics of psychiatric illness as rated by a clinician. The complete CPRS can be used, or selected items can be used alone or together. Several CPRS items, although not its memory item, has been shown to have high inter-rater reliability [15]. The memory variable in CPRS, CPRS-M, represents the present subjective disturbance of memory recall compared with previous ability. CPRS-M is rated on a 7-grade scale where even numbers are specified as 0 = memory as usual, 2 = occasional increased lapses of memory, 4 = reports of socially inconvenient or disturbing loss of memory and 6 = complaints of complete inability to remember [12] and uneven numbers are used when the deficit is rated somewhere in-between the specifiers. In this study, we used the CPRS-M when the physicians rated subjects’ subjective memory and a modified version of it using the same grading and specifiers, CPRS-M-S, when the subjects rated their own subjective memory. A clinically meaningful change in subjective memory was defined as a change in CPRS score of at least 2. CPRS-M and CPRS-M-S were administered in Swedish and are presented in the Supplementary material.

The depressive symptoms were rated with the Montgomery Åsberg Depression Rating Scale (MADRS) which is a rating scale for depression consisting of 10 CPRS variables [16]. The MADRS score span is 0–60. The MADRS score can be interpreted as 7–19 = mild depression, 20–34 = moderate depression, 35–60 = severe depression [17]. Remission after ECT treatment was defined as a MADRS score <11.

The Clinical Global Impressions-Severity Scale [18], CGI-S, is a rating of the overall clinical impression of the severity of mental illness. The subjects were rated with the CGI-S by the physicians. The CGI-S has 7 steps spanning from 1 = not at all ill, to 7 = among the most extremely mentally ill patients.

The AUDIT alcohol consumption questions [19], AUDIT-C, were used to rate the subject of alcohol consumption. AUDIT-C consists of three questions with a maximum score of 12. A score of 5 or more for men or a score of 4 or more for women indicates hazardous drinking [20].

Data for this study were collected at two separate time points for each subject. The first time point was at baseline, closely before (±3 days) or in the beginning of the subject’s ECT series, and the second time point was for most subjects within three days after completion of the ECT series.

At baseline, each subject met the physician assessing the subject. The subject was informed about the study and signed an informed consent. The subject was interviewed to collect background information, after which the physician left the room and the subject answered the single question CPRS-M-S question and the AUDIT-C, which were put in an envelope and sealed. Then the physician came back and completed a MADRS rating and the CPRS-M, blinded to the subject’s rating. Regarding the subjective memory question, the physician assessed and rated the subject’s subjective experience of failing memory and did not attempt to assess and rate the subject’s objective failing memory. Finally, the physician rated the severity of the subject’s mental illness using CGI-S. The procedure of the follow-up was identical to the first, except collecting background information and the alcohol habits form.

All assessments but one were performed by three physicians. To educate the raters and to estimate inter-rater variability, all physicians rated a filmed patient. Altogether eight physicians watched the film and rated CPRS-M. Intra-class correlation for CPRS-M was 1, i.e. all physicians rated the same.

The ECT was given three times per week. Electrode placement was either unilateral according to d’Elia, or bitemporal, using the Thymatron IV (Somatics, LLC., Venice, FL, USA) fixed current 900 mA, pulse width 0.5–1 ms (see Table 3). Age-based dosing was applied at first treatment and subsequently adjusted depending on seizure quality. For example, a 50-year-old patient receives 50% of the Thymatron’s maximum dose of 504 mC, a 25-year-old receives 25% and so on. The anesthesia given during ECT treatments was according to current routines with thiopental as the primary choice for general anesthesia and succinylcholine as muscle relaxant. Termination of the ECT series was decided by the senior psychiatrist in the psychiatric ward.

Data analyses were performed using IBM SPSS v21 (IBM Corp., USA). To assess whether CPRS-M and CPRS-M-S were from populations with different distributions the nonparametric Wilcoxon signed-rank test was used. Correlations between CPRS-M and CPRS-M-S were estimated with the nonparametric correlation estimate Kendall’s tau-B. Sub-analyses, based on demographics, illness type and severity, treatment effect, etc., were performed post-hoc to assess the results’ representativeness for a general population of patients treated with ECT for depressive episodes, and to check for potential confounders. All variables are presented in Table 1. The nonparametric Mann–Whitney U test was used to assess differences in consistency of CPRS-M and CPRS-M-S between the physicians and for differences in consistency of CPRS-M from baseline to follow-up for each physician. Significance level was set to p < .05 and all tests were two-tailed.

Table 1. Categorical clinical and demographic characteristics.

Categorical variable	Group	Baseline (N (%))	Follow-up (N (%))
Gender	Male	12 (23)	8 (20)
	Female	40 (77)	33 (80)
Age group (years)	18–39	11 (21)	10 (24)
	40–64	28 (54)	20 (49)
	65+	13 (25)	11 (27)
Education level	Post-secondary or higher	26 (50)	19 (46)
	Upper secondary school	18 (35)	14 (34)
	Elementary school	5 (10)	5 (12)
	Not completed elementary school	3 (6)	3 (7)
ECT indication	MDD, single episode	8 (15)	5 (12)
	MDD, recurrent episode	28 (54)	23 (56)
	BPD, depressive episode	16 (31)	13 (32)
Psychotic symptoms	Yes	6 (12)	5 (12)
	No	46 (88)	36 (88)
MADRS score group	0–6	0 (0)	16 (39)
	7–19	5 (10)	18 (44)
	20–34	30 (58)	7 (17)
	35–60	17 (33)	0 (0)
Previous ECT treatment	Yes	34 (65)	29 (71)
	No	18 (35)	12 (29)
Lithium	Yes	11 (21)	8 (20)
	No	41 (79)	33 (80)
Rater	Physician #1	13 (25)	12 (29)
	Physician #2	21 (40)	20 (49)
	Physician #3	18 (35)	8 (20)
Remission after ECT	Yes	N/A	22 (54)
	No	N/A	19 (46)
Electrode position (last	Unilateral	N/A	37 (90)
ECT)	Bitemporal	N/A	4 (10)
Pulse width (last ECT; ms)	0.50	N/A	26 (63)
	0.75	N/A	13 (32)
	1.00	N/A	2 (5)
Total		52	41

BPD: bipolar disorder; ECT: electroconvulsive therapy; MADRS: Montgomery Åsberg Depression Rating Scale; MDD: major depressive disorder.

Results

Of the 56 patients assessed for inclusion in the study, 4 were excluded due to exclusion criteria. Two had acquired brain injury, one had schizophrenia and one suffered from alcohol abuse. Thus, 52 subjects were included in the study and out of them, 41 subjects completed the whole study protocol. Eleven subjects completed baseline measurement only. Clinical and demographic characteristics of the subjects included in the study are presented in Tables 1 and 2.

Table 2. Continuous clinical and demographic characteristics.

Continuous variable	Baseline (mean (SD))	Follow-up (mean (SD))
Age (years)	53.3 (16.4)	52.6 (17.3)
MADRS (Σ)	30.9 (8.0)	10.7 (8.0)
AUDIT-C (Σ)	1.5 (1.8)	N/A
CGI-S	4.6 (0.8)	2.5 (1.2)
ECT treatments before baseline (N)	0.4 (0.8)	0.32 (0.76)
ECT treatments between baseline and follow-up (N)	N/A	6.0 (1.9)
Time from last ECT treatment to follow-up (days)	N/A	1.8 (1.8)
Pulse width (final ECT (ms))	N/A	0.6 (0.2)
Total (N)	52	41

AUDIT-C: Alcohol Use Disorder Identification Test – Consumption; CGI-S: Clinical Global Impression – Severity; ECT: electroconvulsive therapy; MADRS: Montgomery-Åsberg Depression Rating Scale; SD: standard deviation.

Of the 52 included subjects, 24 had one or more comorbid conditions. The following psychiatric comorbidities were the most common: anxiety disorder unspecified (10 subjects), general anxiety disorder (7 subjects), attention deficit hyperactivity disorder (3 subjects), attention deficit disorder (3 subjects), Asperger’s syndrome (2 subjects), personality disorder unspecified (2 subjects), and borderline personality disorder (2 subjects).

20% of the subjects had a clinically meaningful worsening of CPRS-M score and 10% a clinically meaningful improvement. 27% of the subjects had a clinically meaningful worsening of CPRS-M-S score and 10% had a clinically meaningful improvement. For 12% of the subjects, there was a clinically meaningful worsening in both CPRS-M and CPRS-M-S.

At baseline, Kendall’s tau-B correlation between CPRS-M and CPRS-M-S was 0.699 for all subjects. There was no difference in median CPRS-M and CPRS-M-S, and the CPRS-M and CPRS-M-S scores were not from statistically different distributions.

After treatment, the Kendall’s tau-B correlation between CRPS-M and CPRS-M-S was 0.651 for all subjects. The difference in median CPRS-M and median CPRS-M-S was 1 for all subjects and the CPRS-M and CPRS-M-S scores were not from statistically different distributions

Secondary analyses showed that at baseline and follow-up, Kendall’s tau-B was higher than 0.6 in most subgroups (see Table 3 and Figure 1). At baseline, the CPRS-M and CPRS-M-S scores in the subgroups were not from statistically different distributions. The correlations decreased from baseline to follow-up for the subjects assessed by physician #3 and for the male subjects and these correlations were not statistically significant at follow-up (eight subjects in these subgroups at follow-up). For all other secondary analyses of groups containing more than five subjects, the correlations were statistically different from zero at follow-up (see Tables 1 and 3). At follow-up, the CPRS-M and CPRS-M-S scores were not from statistically different distributions except for the subgroups: MADRS score 20–34 (seven subjects in this subgroup), #2 physician, and non-remitters (see Table 4).

Figure 1. Correlation between CPRS-M and CPRS-M-S at baseline (a) and follow-up (b). The size of the bubbles represents the number of subjects within each combination of CRPS-M and CPRS-M-S.

Table 3. Kendall’s tau-B correlations between CPRS-M and CPRS-M-S with p-values for non-zero correlation.

Categorical variable	Group	Baseline (correlation (p-value))	Follow-up (correlation (p-value))
Gender	Male	0.82 (<.01)	0.49 (.127)
	Female	0.62 (<.01)	0.68 (<.01)
Age group (years)	18–39	0.63 (.016)	0.65 (.017)
	40–64	0.71 (<0.01)	0.39 (.039)
	65+	0.71 (<.01)	0.81 (<.01)
Education level	Post-secondary or higher	0.72 (<.01)	0.75 (<.01)
	Upper secondary school	0.83 (<.01)	0.65 (<.01)
	Elementary school	0.87 (.068)	−0.57 (.213)
	Not completed elementary school	−0.50 (.480)	1.00 (<.01)
ECT indication	MDD, single episode	0.64 (.037)	0.89 (.037)
	MDD, recurrent episode	0.70 (<.01)	0.63 (<.01)
	BPD, depressive episode	0.79 (<.01)	0.61 (<.01)
Psychotic symptoms	Yes	0.93 (.014)	0.95 (.023)
	No	0.68 (<.01)	0.60 (<.01)
MADRS score group	0–6		0.72 (<.01)
	7–19	0.80 (.078)	0.63 (<.01)
	20–34	0.74 (<.01)	0.83 (.027)
	35–60	0.64 (<.01)
Previous ECT treatment	Yes	0.68 (<.01)	0.70 (<.01)
	No	0.76 (<.01)	0.57 (.023)
Lithium	Yes	0.83 (<.01)	0.94 (<.01)
	No	0.64 (<.01)	0.60 (<.01)
Rater	Physician #1	0.66 (<.01)	0.77 (<.01)
	Physician #2	0.68 (<.01)	0.81 (<.01)
	Physician #3	0.78 (<.01)	0.56 (.089)
Remission after ECT	Yes	N/A	0.70 (<.01)
	No	N/A	0.58 (<.01)
Electrode position (last	Unilateral	N/A	0.63 (<.01)
ECT)	Bitemporal	N/A	0.40 (.444)
Pulse width (last ECT; ms)	0.50	N/A	0.62 (<.01)
	0.75	N/A	0.63 (<.01)
	1.00	N/A
Total		0.70 (<.01)	0.65 (<.01)

BPD: bipolar disorder; ECT: electroconvulsive therapy; MADRS: Montgomery-Åsberg Depression Rating Scale; MDD: major depressive disorder.

Table 4. CPRS-M and CPRS-M-S: median value, minimum value and maximum value. p-values from Wilcoxon signed-rank test.

		Baseline			Follow-up
Categorical variable	Group	CPRS-M	CPRS-M-S	p-value	CPRS-M	CPRS-M-S	p-value
All subjects		2, 0, 5	2, 0, 6	.560	3, 0, 6	2, 0, 6	.233
Gender	Male	0.5, 0, 5	1, 0, 5	.564	1.5, 0, 5	2, 0, 5	.705
	Female	2, 0, 5	2, 0, 6	.443	3, 0, 6	3, 0, 6	.110
Age group (years)	18–39	2, 0, 5	2, 0, 4	.197	3.5, 0, 6	3, 0, 5	.084
	40–64	2, 0, 5	2, 0, 5	.438	2, 0, 5	2, 0, 6	.963
	65+	2, 0, 5	2, 0, 6	.366	2, 0, 5	2, 0, 4	.480
Education level	Post-secondary or higher	1.5, 0, 4	2, 0, 4	.331	2, 0, 5	2, 0, 4	.096
	Upper secondary school	2, 0, 5	2, 0, 6	1.000	3, 0, 6	3, 0, 6	.773
	Elementary school	4, 0, 4	1, 0, 2	.102	5, 0, 5	4, 2, 4	.715
	Not completed elementary school	2, 2, 4	2, 2, 3	.655	0, 0, 2	0, 0, 3	.317
ECT indication	MDD, single episode	2, 0, 5	2.5, 0, 5	.739	3, 0, 5	2, 0, 5	.157
	MDD, recurrent episode	2, 0, 5	2, 0, 6	.204	2, 0, 5	3, 0, 4	.869
	BPD, depressive episode	1, 0, 5	1, 0, 4	.782	3, 0, 6	2, 0, 6	.222
Psychotic symptoms	Yes	2, 0, 5	1, 0, 5	.083	3, 0, 5	2, 0, 5	.157
	No	2, 0, 5	2, 0, 6	.851	2.5, 0, 6	2, 0, 6	.360
MADRS score group	0–6				2, 0, 5	2, 0, 6	.527
	7–19	2, 0, 3	0, 0, 2	.102	3, 0, 6	3, 0, 5	.458
	20–34	2,0, 5	2, 0, 6	.742	3, 2, 5	2, 2, 5	.038
	35–60	2, 0, 5	2, 0, 4	.627
Previous ECT treatment	Yes	2, 0, 5	2, 0, 6	.162	3, 0, 6	3, 0, 5	.225
	No	1.5, 0, 4	1.5, 0, 4	.248	2.5, 0, 5	2, 0, 6	.660
Lithium	Yes	0, 0, 5	1, 0, 5	.102	3, 0, 5	2, 0, 6	.564
	No	2, 0, 5	2, 0, 6	.209	2, 0, 6	2, 0, 5	.286
Rater	Physician #1	2, 0, 4	2, 0, 4	.317	2, 0, 5	2, 0, 6	.18
	Physician #2	2, 0, 5	2, 0, 5	.054	3.5, 0, 6	2, 0, 5	<.01
	Physician #3	2, 0, 5	2, 0, 6	.366	0.5, 0, 4	3, 0, 4	.109
Remission after ECT	Yes	N/A	N/A	N/A	2, 0, 5	2, 0, 6	.589
	No	N/A	N/A	N/A	3, 0, 6	3, 0, 5	.047
Electrode position (last ECT)	Unilateral	N/A	N/A	N/A	2, 0, 6	2, 0, 6	.342
	Bitemporal	N/A	N/A	N/A	4, 0, 5	3.5, 0, 4	.414
Pulse width (last ECT; ms)	0.50	N/A	N/A	N/A	2.5, 0, 5	2, 0, 6	.439
	0.75	N/A	N/A	N/A	2, 0, 5	2, 0, 4	.429
	1.00	N/A	N/A	N/A	5.5, 5, 6	5, 5, 5	.317
Total (N)		52			41

BPD: bipolar disorder; ECT: electroconvulsive therapy; MADRS: Montgomery-Åsberg Depression Rating Scale; MDD: major depressive disorder.

Discussion

In this study, we have shown that self-rating of subjective memory has high and significant consistency with clinicians’ ratings. The correlations between CPRS-M and CPRS-M-S were positive and statistically significant for all subjects and for all larger subgroups of subjects. In a study comparing 19 CPRS variables rated by patients and a physician [21], the median correlation was 0.54 (range 0.35–0.72) which was lower than our correlations for all subjects. This indicates relatively high correlations between CPRS-M and CPRS-M-S in the present data, supporting the implementation of CPRS-M-S for quantification of subjective disturbance of memory recall compared with previous ability in patients referred to ECT for the treatment of depressive episodes.

The mean CPRS-M and CPRS-M-S were numerically close to each other at baseline and follow-up. The mean CPRS-M and CPRS-M-S scores were close in most larger subgroups (Table 1). This supports the generalizability of the results.

To evaluate the generalizability further, we compared our study sample with that of the Swedish Q-ECT for the year 2016. The Swedish Q-ECT holds nationwide data about ECT patients and treatments in Sweden since 2011 and for 2016 it covers 89% of the patients treated with ECT in Sweden [3]. The gender distribution in our data and Q-ECT both showed an overweight for women, although this was more pronounced in our data (77%) than in Q-ECT (62%). The mean age in the Q-ECT was 54 years which was close to the mean age in our study sample. Compared to our study sample the Q-ECT contains relatively more patients with single-episode depression (41%) and also relatively fewer patients with bipolar disorder with depression (19%). Of the patients treated for depression in the Q-ECT 22% had psychosis compared to 12% in our study. While the relatively large group of bipolar patients in our data allows for more robust interpretation of the correlation coefficients, the small sample of patients with psychotic symptoms warrants caution. Possibly CPRS-M-S is less well correlated to CPRS-M in this group. A recent study based on the Q-ECT data showed subjective memory worsening for 26% [13] compared to 20% (CPRS-M) and 27% (CRPS-M-S) in our study. Taking all this into account and bearing the limited sample size in mind, our study sample differs in some aspects from the Q-ECT and therefore does not fully represent the population in the Swedish Q-ECT.

Comparing the CPRS-M to CPRS-M-S, there was a group effect by physician. The group rated by #2 physician was large, consisting of 20 (21 at baseline) subjects and the mean difference between CPRS-M and CPRS-M-S in this subgroup was 0.90 (0.48 at baseline). It is notable that the p-value for this subgroup at baseline was only marginally higher than the significance level indicating relatively different distributions. We suspected that physician #2’s ratings were less consistent with the subjects’ self-reported ratings than the ratings by the other physicians. To analyze the consistency of CPRS-M and CPRS-M-S between the three physicians, we compared the difference of CPRS-M and CPRS-M-S in the data grouped by physician, using the Mann–Whitney U test. At baseline, there was no significant difference between physician #1 and #2 (p = .079) and #1 and #3 (p = .929), but significant difference between physician #2 and #3 (p = .042). At follow-up, there was no significant difference between physician #1 and #3 (p = .455) but significant difference between physician #1 and #2 (p < .01) and #2 and #3 (p < .01). We also assessed differences in consistency of CPRS-M from baseline to follow-up for each physician for the subjects assessed at both occasions. This was not significant for any of the physicians (p = .896 for physician #1, p = .177 for #2 and p = .303 for #3). The by-rater analyses indicate non-perfect inter-rater reliability. In the present study, the short co-rating feedback exercise that preceded the study showed very high consistency. In spite of this training, we found a significant confounding rater effect. Since such educations are rare, by-rater differences could be even more pronounced in a clinical setting. This suggests that CPRS-M-S may be preferred to CPRS-M.

The secondary analyses (Table 4) also show that the subjects that did not respond to ECT self-rate their memory function as significantly better than the physicians rate it. The reasons for this difference are unknown. One explanation might be that raters interpret other depression-related cognitive deficits as memory impairment. Another option is that the patients that are still depressed after ECT are less aware of their memory impairment. Objective measurements of memory are needed to explain this difference.

There are some weaknesses in this study. First, the CPRS-M does not objectively measure memory function or any domain thereof, but perceived memory function. Second, in the study protocol, the subjects rated their subjective memory closely before the physician and the order was not randomized. This may have resulted in the subjects remembering the syntax of their self-reported subjective memory rating making matched ratings more likely than otherwise would have been the case. However, the design was chosen to avoid that the rating should be affected by time. Third, there are relatively few subjects in the study sample. In particular, some of the subgroups in the secondary analyses are very small, giving uncertain estimates. Fourth, the CPRS-M is quite brief, and has not been developed specifically for the ECT setting. Fifth, the consistency between subjective memory quantification using CPRS-M-S and objective measures of memory has to our knowledge not been studied. This information is of interest for the interpretation of CPRS-M-S, and should be done in a separate subject sample.

In summary, the results show that the self-rating scale CPRS-M-S can be used instead of the clinician’s rating CPRS-M. Inter-rater differences, and the fact that feasibility has an impact on the clinician’s decision to use standardized assessment tools at all [22], favors the use of the self-rating scale.

Disclosure statement

The authors report no conflict of interest.

Additional information

Funding

This study was supported by Stockholm Health Care Services Development Project Grant HSN 1308-0925.