Toward an online cognitive and emotional battery to predict treatment remission in depression

Figures & data

Table 1 Cognitive and emotion tasks

Task name	Assesses	Test duration	Measures of task performance
General cognition tasks
Switching of attention	Information processing speed	3 minutes	Completion time
Verbal interference	Cognitive flexibility	2 minutes	Reaction time – incongruent trials
Continuous performance test	Attention	6 minutes	Reaction time
Go/no-go	Response inhibition	4 minutes	Reaction time
Digit span	Working memory	6 minutes	Total number of trials correct
Motor tapping	Motor coordination	1 minute	Number of taps – dominant hand Number of taps – non-dominant hand
Maze	Executive function	10 minutes	Completion time Number of errors Number of overrun errors
Emotional cognition tasks
Emotion identification	Emotion identification accuracy Speed of emotion identification	4 minutes	Number of correct responses Reaction time for correct responses

Figure 1 Consort chart.

Notes: Of the patients who met inclusion criteria for the iSPOT-D study, additional criteria were used for this specific analysis. *Minimum criteria for assessment at week 8 is either completion of the HRSD₁₇ or SOFAS assessments at the in-person visit, or completion of the week 8 QIDS-SR₁₆ in the online questionnaire at home.
Abbreviations: QIDS-SR₁₆, 16-item Quick Inventory of Depressive Symptomatology Self-Report; iSPOT-D, International Study to Predict Treatment Response in Depression; XR, extended release; HRSD₁₇, 17-item Hamilton Rating Scale for Depression.

Table 2 Proportion of patients by treatment for whom a test battery recommendation could be made

Medication	Recommendation		No recommendation
	Likely to remit	Not likely to remit
Escitalopram	21%	22%	57%
Sertraline	24%	39%	37%
Venlafaxine-XR	21%	27%	52%

Abbreviation: XR, extended release.

Table 3 Sensitivity and specificity of the depression treatment test

	Sensitivity	Specificity	PPV	NPV	Odds ratio	NNT
	DT value [chance rate]	DT value [chance rate]	DT value [observed remission rate]	DT value [observed non-remission rate]
Full sample
Escitalopram	75 [50]	70 [50]	64 [41]	80 [59]	7.0	2.3
Sertraline	71 [50]	73 [50]	48 [25]	88 [75]	6.9	2.7
Venlafaxine-XR	67 [50]	67 [50]	50 [33]	81 [67]	4.1	3.3
Cross-validated
Escitalopram	72 [50]	67 [50]	61 [41]	77 [59]	4.9	2.7
Sertraline	69 [50]	70 [50]	43 [25]	87 [75]	4.4	3.5
Venlafaxine-XR	67 [50]	62 [50]	46 [33]	81 [67]	2.7	4.6

Notes: All statistics reported here are calculated only for the intended population of the prediction model (better than average emotion identification in escitalopram, >30 years of age in sertraline, and better than average general cognition performance in venlafaxine-XR). Full-sample statistics report the accuracy statistics calculated from the same sample used to develop the regression model. Cross-validated statistics are the averaged accuracy statistics of 300 cross-validation iterations. Observed remission rates are based upon the specific cohort used for the depression test (n=68, n=111, and n=64 for escitalopram, sertraline and venlafaxine-XR, respectively).

Abbreviations: DT, depression treatment test; NNT, number needed to treat; NPV, negative predictive values; PPV, positive predictive values; XR, extended release.

Figure 2 Final regression model to predict remission in each treatment arm and applicable subgroup.

Notes: Each treatment arm was stratified into two groups. Remission could be predicted 10 percentage points above chance. The specific subgroup and threshold is reported. The specific regression equation for each treatment arm is reported. Additionally, the number of patients in each treatment arm and each subgroup is also reported. DT result is the Depression Treatment Test predictive outcome. *Defined as average of z-scores for emotion identification number of correct responses and reaction time for correct responses. **Defined as average of z-scores for maze completion time, maze total number of errors and number of overrun errors, switching of attention completion time, verbal interference reaction time incongruent trials, continuous performance test reaction time, digit span total number of trials correct, and emotion identification number of correct response and reaction time for correct responses.
Abbreviations: DT, depression treatment test; QIDS-SR₁₆, 16-item Quick Inventory of Depressive Symptomatology Self-Report; XR, extended release.

Figure 3 ROC curves comparing cognition with age and sex as well as depression severity.

Notes: This figure illustrates that regression models using cognition and emotion predictors outperform those that use only age and sex (A) or depression severity (B) (baseline QIDS-SR₁₆). The ROC curves were generated using the probability of remission obtained from each regression equation. Only the patients who met criteria for the logistic regression model were used to generate the ROC curves and calculate the AUC.
Abbreviations: AUC, area under the curve; QIDS-SR₁₆, 16-item Quick Inventory of Depressive Symptomatology Self-Report; ROC, receiver operating characteristics; XR, extended release.

Table 4 Proportion of patients identified with greater than 80% certainty to remit

Medication	Total number	Total remitters	Number of participants >80% PPV	% of total treatment arm	% of remitters
Escitalopram	157	64	22	14.01%	34.38%
Sertraline	175	53	–	–	–
Venlafaxine-XR	135	42	–	–	–

Notes: The proportion of participants identified by the regression models to be highly certain to remit (PPV >80%). The number of participants reported represents the total number of participants identified by the regression model to be highly certain to remit and hence will reflect both true and false positives. Percentages of remitters identified include false positives.

Abbreviations: PPV, positive predictive values; XR, extended release.

Table 5 Proportion of patients identified with greater than 80% certainty to not remit

Medication	Total number	Total non-remitters	Number of participants >80% NPV	% of total treatment arm	% of non-remitters
Escitalopram	157	93	38	24.20%	40.86%
Sertraline	175	122	94	53.71%	77.05%
Venlafaxine-XR	135	93	36	26.67%	38.71%

Notes: The proportion of participants identified by the regression models to be highly certain to not remit (NPV >80%). The number of participants reported represents the total number of participants identified by the regression model to be highly certain to remit and hence will reflect both true and false negatives. Percentages of non-remitters identified include false positives.

Abbreviations: NPV, negative predictive values; XR, extended release.

Figure 4 Percent of the sample receiving a recommendation to remit or to not remit, and relative risks for each type of prediction.

Notes: Relative risks were calculated using the remission rate for the specific patients for whom a prediction was made. A horizontal bar is drawn at a relative risk of 1 to allow for chance comparisons. Patients predicted to not remit, with a relative risk of 0.5–0.6, have one half the remission rate for the subgroup used to develop the specific regression model. Patients predicted to remit, with a relative risk of 1.5–1.9, have one and a half to two times the remission rate for the subgroup used to develop the specific regression model.
Abbreviation: XR, extended release.

Table S1 Inclusion/exclusion criteria

Inclusion criteria

• Age 18–65

• Fluent and literate in English or Dutch

• Provide written informed consent

• Total HRSD17 ≥16

• Meets DSM-IV criteria for single or recurrent nonpsychotic MDD established by MINI Plus

Exclusion criteria

• Suicidal ideation and/or tendencies, defined by a score ≥8 on section C of the MINI Plus

• History of bipolar disorder (I, II, not otherwise specified) (lifetime)

• History of schizophrenia, schizoaffective disorder, or psychosis not otherwise specified (lifetime)

• Current primary diagnosis of anorexia or bulimia, obsessive-compulsive disorder, or primary post-traumatic stress disorder

• Known contraindication for escitalopram, sertraline and/or venlafaxine-XR, or previous treatment failure at the highest recommended dose

• Taking any medication that is contraindicated with escitalopram, sertraline, or venlafaxine-XR

• Taking escitalopram, sertraline, or venlafaxine-XR in the current episode of MDD

• Use of any non-protocol antidepressant drug or CNS drug (antipsychotic, anticonvulsant anxiolytic, clonidine) that cannot be washed out prior to participation

• Has general medical condition that contraindicates protocol antidepressant treatments or interferes with protocol measurements (such as epileptic condition for EEG recording)

• Substance dependence (including alcohol intake equaling 29 standard alcoholic drinks per week for males; >15 for females) in the past 6 months

• History of brain injury or blow to the head that resulted in loss of consciousness for greater than 5 minutes

• Severe impediment to vision, hearing and/or hand movement that is likely to interfere with completion of assessments, or with comprehension of instructions or study requirements

• Participation in an investigational study within 4 months prior to baseline that could affect symptoms of MDD

• Is pregnant or breast-feeding

Abbreviations: HRSD₁₇, 17-item Hamilton Rating Scale for Depression; MDD, major depressive disorder; XR, extended release; EEG, electroencephalogram; DSM, Diagnostic and Statistical Manual of Mental Disorders; MINI, Mini-International Neuropsychiatric Interview; CNS, central nervous system.

Table S2 Comparison of included and excluded samples (categorical measures)

	Excluded from sample (N=541)			Included in sample (N=467)			χ^2	P-value
	Count	%	Total N	Count	%	Total N
Sex (female)	240	44.36%	541	197	42.18%	467	3.498	0.174
Family history of MDD	114	21.07%	541	119	25.48%	467	0.313	0.855
MDD diagnosis prior to 18	239	45.01%	531	239	51.96%	460	1.992	0.369
Race							5.146	0.161
Black	93	17.19%	541	74	15.85%	467
Other	125	23.11%	541	87	18.63%	467
Unknown	1	0.18%	541	3	0.64%	467
White	322	59.52%	541	303	64.88%	467
Employment							6.848	0.232
Employed	267	49.35%	541	239	51.18%	467
Other	37	6.84%	541	36	7.71%	467
Retired	28	5.18%	541	15	3.21%	467
Student	100	18.48%	541	89	19.06%	467
Unemployed	30	5.55%	541	36	7.71%	467
Unknown	79	14.60%	541	52	11.13%	467
Marital status							6.391	0.172
Divorced/separated	68	12.57%	541	74	15.85%	467
Married/cohabiting	98	18.11%	541	95	20.34%	467
Single	341	63.03%	541	273	58.46%	467
Unknown	27	4.99%	541	15	3.21%	467
Widowed	7	1.29%	541	10	2.14%	467
Recruitment source							1.261	0.738
Ad	424	78.37%	541	371	79.44%	467
GP	12	2.22%	541	14	3.00%	467
Other	48	8.87%	541	40	8.57%	467
Psych	57	10.54%	541	42	8.99%	467
Number of episodes							6.298	0.278
0	2	0.38%	528	0	0.00%	457
1	57	10.80%	528	46	10.07%	457
2	53	10.04%	528	32	7.00%	457
3	60	11.36%	528	45	9.85%	457
4	64	12.12%	528	56	12.25%	457
5 or more	292	55.30%	528	278	60.83%	457
Dysthymia	101	18.67%	541	118	25.27%	467	2.660	0.264
Panic disorder	43	7.95%	541	42	8.99%	467	0.488	0.783
Agoraphobia	37	6.85%	540	37	7.92%	467	1.237	0.539
Social phobia	37	6.84%	541	56	11.99%	467	4.549	0.103
Specific phobia	24	4.44%	541	31	6.65%	466	0.607	0.738
Generalized anxiety disorder	33	6.10%	541	36	7.71%	467	3.360	0.186
Previous suicide attempt	64	11.83%	541	53	11.35%	467	0.406	0.816
Number of comorbid medication conditions							15.371	0.009
0	333	61.55%	541	236	50.54%	467
1	111	20.52%	541	122	26.12%	467
2	51	9.43%	541	45	9.64%	467
3	22	4.07%	541	27	5.78%	467
4	12	2.22%	541	19	4.07%	467
5 or more	12	2.22%	541	18	3.85%	467
MDD recurrence							0.393	0.822
Not recurrent	59	10.91%	541	46	9.85%	467
Recurrent	469	86.69%	541	411	88.01%	467
Unknown	13	2.40%	541	10	2.14%	467

Abbreviations: MDD, major depressive disorder; GP, general practitioner; Psych, psychiatrist/psychologist.

Table S3 Comparison of included and excluded samples (continuous measures)

	Excluded from sample (N=541)			Included in sample (N=467)			t	P-value
	Mean	SD	Total N	Mean	SD	Total N
Age (years)	37.30	12.56	541	38.47	12.57	467	−1.468	0.142
Education (years)	14.56	2.73	541	14.52	2.88	467	0.189	0.850
Age at first episode (years)	23.21	12.20	531	22.54	11.84	460	0.873	0.383
Duration of MDD (years)	14.02	11.64	531	15.83	12.72	460	−2.323	0.020
HRSD17 score	21.58	4.11	541	22.24	4.12	467	−2.521	0.012
QIDS-SR16 score	13.15	4.22	496	15.86	2.72	467	−11.920	0.000
DASS (depression)	19.73	9.45	469	24.75	8.97	451	−8.255	0.000
DASS (anxiety)	8.08	6.71	469	9.49	6.63	451	−3.199	0.001
DASS (stress)	17.00	8.15	469	19.47	8.45	451	−4.498	0.000
SOFAS	56.34	9.36	540	55.40	8.79	466	1.633	0.103
SWLS	12.10	5.51	506	11.01	5.17	461	3.184	0.002
WHOQoL-physical	54.44	14.04	485	49.02	14.35	453	5.850	0.000
WHOQoL-psychological	37.19	14.29	484	31.93	12.79	453	5.948	0.000
WHOQoL-social	40.96	20.21	485	36.09	19.22	454	3.784	0.000
WHOQoL-environmental	52.17	15.64	483	51.18	15.93	451	0.958	0.338
WHOQoL-overall	2.84	0.92	486	2.65	0.93	457	3.299	0.001
ERQ (reappraisal)	4.39	1.24	506	4.26	1.22	465	1.662	0.097
ERQ (suppression)	4.09	1.34	506	4.22	1.35	465	−1.557	0.120

Abbreviations: DASS, depression anxiety stress scale; ERQ, emotion regulation questionnaire; HRSD₁₇, 17-item Hamilton Rating Scale for Depression; MDD, major depressive disorder; QIDS-SR₁₆, 16-item Quick Inventory of Depressive Symptomatology Self-Report; SOFAS, social and occupational functioning assessment scale; SWLS, satisfaction with life scale; WHOQoL, World Health Organization quality of life; SD, standard deviation.