Psychiatric caseness is a marker of major depressive episode in general practice

Abstract

Objective. Screening for a major depressive episode (MDE) in high-risk groups of patients within the primary care setting has been suggested by several Central Health Organizations. The objective of this study was to investigate whether patients rated as “psychiatric cases” by their general practitioner (GP) were likely to suffer from MDE and therefore qualified for systematic diagnostic screening. Design. Cross-sectional survey of primary care patients assessed through depression screening questionnaires and GP consultations. Setting. A total of 676 general practices in Denmark, Finland, Norway, and Sweden. Subjects. A total of 8879 unselected primary care patients. Main outcome measures. Sensitivity, specificity, and Youden Index of the GPs' diagnoses of depression and psychiatric caseness versus patients' MDE status. Results. The proportion of primary care patients receiving a false-positive diagnosis of depression by their GP ranged from 12.4% to 25.2% depending on country. The corresponding numbers for the false-negative diagnoses were 0.5–2.5%. Among patients with MDE, GPs recognize the disease in 56–75% of cases. However, GPs recognize as many as 79–92% of patients with MDE as “psychiatric cases”. Conclusions. This report confirms that misclassifications of MDE are common in the primary care setting. In addition, it shows that psychiatric caseness is a valid marker for the presence of MDE in primary care patients. This relationship should be considered in future screening recommendations.

Key Words::

• Depression
• diagnosis
• family practice
• mass screening
• questionnaires

Patients with a major depressive episode (MDE) are often overlooked in the primary care setting. Central Health Organizations suggest screening for MDE in high-risk categories of primary care patients. This study investigated whether patients rated as “psychiatric cases” by their GPs were likely to suffer from MDE and therefore qualified for systematic diagnostic screening.

• Misclassifications of MDE were common in the present sample of primary care patients.

• Among primary care patients with MDE, general practitioners (GPs) recognized the disease in 56–75% of the cases.

• Up to 92% of patients with MDE were identified as psychiatric cases by their GP. This relationship should be considered in future screening recommendations.

Major depressive episode (MDE) is the most common mental disorder in primary care [1]. General practitioners (GPs) play a crucial role in detecting and treating mental disorders, including MDE [2]. Diagnosis and treatment of MDE in primary care remains a difficult challenge for GPs and misclassifications are common. There are two types of misclassifications of MDE, namely false negative (patients meeting MDE criteria, but who are not recognized as depressed by their GPs) and false positive (perceived to suffer from MDE by the GPs without fulfilling the MDE criteria) [3–7]. These misclassifications have obvious adverse effects for the patients and procedures to minimize their likelihood of occurrence are required. The purpose of this study was to examine the GPs' diagnostic capability in relation to MDE and to evaluate whether patients rated as “psychiatric cases” by their GP were likely to suffer from MDE.

Material and methods

A total of 3896 GPs (Denmark (n = 3500), Finland (n = 103), Norway (n = 141), and Sweden (n = 152)) were invited to participate in the study. The Danish GPs were invited by letter, whereas the GPs in the other countries were recruited by local representatives from Wyeth Ltd. This difference in recruitment influenced the participation rate as approximately 10% of the Danish GPs participated compared with > 80% of the GPs in Finland, Norway, and Sweden. A more detailed description of the recruitment and pre-study instruction of GPs is reported elsewhere [8].

All patients above 16 years of age (above 18 in Denmark) consulting their GP on one of three pre-selected target days (16–18 September 2001) were invited to participate in the study. A total of 8879 patients (Denmark (n = 4543), Finland (n = 1224), Norway (n = 1764), and Sweden (n = 1348)) completed the Depression Screening Questionnaire (DSQ) [9] prior to their consultation. The GPs completed questionnaires regarding each patient's physical and mental health status. They were specifically asked to evaluate whether the patient suffered from MDE or any other mental disorders.

MDE diagnosis by questionnaire

The patients' MDE status was assessed by the DSQ which consists of 11 items rated on a three-point scale. A research diagnosis of MDE was assigned when at least five of the items were rated as present “most days” by the patient. These criteria for MDE differ from those used in the primary publication of the present data [8], but are in accordance with the DSM-IV diagnostic criteria [9]. Thus, the DSQ result was used as reference standard for patients' diagnostic MDE status.

MDE evaluation by GPs

The GPs rated each patient on the Clinical Global Impression-Severity Scale (CGI-S) [10] in relation to MDE, GAD, and “Other Anxiety Disorders”. The CGI-S posed the question: “In your clinical judgement how severely does the patient suffer from MDE/GAD/Other anxiety disorders?” The rating had six levels: 1 = not ill at all, 2 = a borderline case, 3 = only mildly ill, 4 = moderately ill, 5 = very ill, 6 = extremely ill.

Another part of the GPs' questionnaire contained the “mental disorder” scale with the following question: “Does the patient suffer from one of the following mental disorders? (according to your clinical judgement): MDE, GAD, other anxiety disorders, panic disorder, substance dependence, acute stress disorder/adjustment disorder, psychosomatic disorder, other psychiatric disorders”. Each of these eight conditions were rated on a four-level scale: 1 = not at all, 2 = doubtful, 3 = a borderline case, 4 = definitely.

MDE was defined as present if the GP gave a rating of 3 (only mildly ill) or more on the CGI-S or a rating of 4 (definitely) on the “mental disorder” scale in relation to MDE.

Definition of “psychiatric caseness”

The CGI-S and the “mental disorder” scales were also used to obtain a general index of psychiatric caseness [11]. We considered the patients to be “psychiatric cases” if the GP gave a rating of 3 or more on any of the CGI-S scales for MDE/GAD/Other anxiety disorders or a rating of 4 on any of the disorders included on the “mental disorder” scale. Psychiatric caseness was defined prior to analysis.

Statistical procedures

Based on the comparison of the DSQ results (reference standard) and the GPs' evaluation of mental health, patients were divided into four subgroups: true positives, true negatives, false positives, and false negatives in respect of both MDE and psychiatric caseness as outlined in Table I. Sensitivity and specificity of the GPs' MDE diagnoses was calculated based on the distribution of patients in the four subgroups.

Table I. Distribution of patients in four diagnostic subgroups.

MDE	Denmark	Finland	Sweden	Norway
	% (95% CI)	% (95% CI)	% (95% CI)	% (95% CI)
True positives	3.1 (2.5–3.6)	3.0 (1.8–4.1)	3.3 (2.2–4.3)	2.7 (1.8–3.6)
False positives	12.4 (11.2–13.5)	16.4 (13.8–19.1)	15.3 (13.0–17.6)	25.2 (22.3–28.0)
True negatives	82.1 (80.7–83.5)	79.1 (76.0–82.2)	80.1 (77.6–82.6)	71.3 (68.3–74.3)
False negatives	2.5 (2.0–2.9)	1.5 (0.8–2.3)	1.3 (0.7–2.0)	0.9 (0.4–1.3)
Patients in total	4.271	1.180	1.283	1.720

Notes: The four diagnostic subgroups were defined on the basis of patients' DSQ results and GPs' MDE diagnoses for Denmark, Finland, Sweden, and Norway respectively. The numbers represent proportion (%) of patients classified in each subgroup out of the total number of patients in each country. Parentheses represent 95% confidence intervals. The 95% confidence intervals were adjusted for GP clusters (correlation between the multiple judgements from each GP).

To quantify the quality of the GPs' diagnoses we used the Youden Index (YI) [12]. The YI rates diagnostic performance against a diagnostic standard. The index takes both specificity and sensitivity into account: Youden Index = sensitivity + specificity − 1.

Results

Table I shows the proportion of patients classified in each of the four diagnostic categories. Table II lists the sensitivity, specificity, and Youden Index of the comparisons between GPs' diagnoses of MDE/psychiatric caseness and the patients' MDE status according to the DSQ result. When the psychiatric case status, as evaluated by the GPs, was compared with the patients' DSQ results, the recognition of MDE increased markedly and reached sensitivity between 0.79 and 0.92 depending on country. The sensitivity increased relatively between 13% and 41% compared with the results obtained using the GPs' diagnosis of MDE. The corresponding specificity declined relatively between 11% and 50%, whereas the Youden Indices increased in Denmark, remained unchanged in Norway, and declined in Finland and Sweden.

Table II. Comparison of sensitivity, specificity and diagnostic accuracy.

	Denmark	Finland	Sweden	Norway
MDE
Sensitivity	0.56 (0.49–0.62)	0.66 (0.53–0.79)	0.71 (0.60–0.83)	0.75 (0.65–0.86)
Specificity	0.87 (0.86–0.88)	0.83 (0.81–0.85)	0.84 (0.82–0.86)	0.74 (0.72–0.76)
Youden Index	0.42 (0.36–0.49)	0.49 (0.36–0.62)	0.55 (0.43–0.67)	0.49 (0.38–0.60)
Psychiatric Case
Sensitivity	0.79 (0.74–0.84)	0.92 (0.85–1.00)	0.90 (0.82–0.98)	0.85 (0.76–0.94)
Specificity	0.74 (0.73–0.75)	0.49 (0.46–0.52)	0.42 (0.39–0.44)	0.63 (0.61–0.66)
Youden Index	0.53 (0.48–0.59)	0.42 (0.34–0.49)	0.31 (0.23–0.40)	0.49 (0.39–0.58)

Notes: MDE: Sensitivity and specificity of GPs' MDE diagnoses in Denmark, Finland, Norway, and Sweden. Diagnostic performance is expressed as the Youden Index. Parentheses represent 95% confidence intervals. Psychiatric case: Sensitivity, specificity, and Youden Index of the comparison between patients' psychiatric case status, as evaluated by the GP, and their DSQ status.

In their questionnaire on each patient, the GPs reported whether they had read the patient's answers to the questionnaire. Danish GPs gave an affirmative answer for 38% of their patients. The corresponding values for Finland, Sweden, and Norway were 38%, 50%, and 52% respectively. When the cases in which the GPs had read the patient's questionnaire were compared with the total sample, the sensitivity of the MDE diagnosis was found to be increased in the subgroup (questionnaire read) of all four countries: Denmark (0.60), Finland (0.78), Sweden (0.81), and Norway (0.82). In the same comparison the specificity showed the opposite pattern as it decreased slightly in all countries: Denmark (0.86), Finland (0.81), Sweden (0.82), and Norway (0.72). However, the resulting Youden Index was increased in Denmark (0.46), Finland (0.59), Norway (0.54), and Sweden (0.62).

Discussion

Summary of principal findings

• The recognition rates of MDE observed in the present study were noted in other European and North American countries [3,13]. The same was the case for the proportions of false negatives and false positives [6,14].

• Misclassifications of MDE were common in the present sample of primary care patients.

• The GPs' perception of psychiatric caseness is a sensitive marker for the presence of MDE in primary care patients.

Strengths and weaknesses

The use of a questionnaire as diagnostic gold standard is controversial. Allocating a psychiatric diagnosis is ideally based on a diagnostic interview performed by a trained psychiatrist. However, the psychometric agreement between the DSQ and the Composite International Diagnostic Interview is very good with a kappa value of 0.82 [9]. Furthermore, the use of a questionnaire enabled us obtain a relatively large sample of primary care patients.

Since the recruitment and participation rate of GPs was not identical in the four countries the presence of a significant selection bias among the countries is very likely. Therefore we have not elaborated on the detected differences in sensitivity, specificity, and Youden indices among countries. The focus of this study was to track the changes within countries.

Furthermore, it is likely that the GPs who chose to participate in the present study had a special interest in the diagnosing of mental disorders and may therefore not be representative of the average GP in the four countries.

The fact that GPs only had one consultation to evaluate the patients' mental state might have caused some underestimation of their diagnostic capability. A recent meta-analysis has shown that GPs detect additional cases of depression over time and with repeated consultations [4]. Furthermore the present study does not take depression severity into account. It has been implied that GPs recognize the majority of the more severe cases of depression [5,15], which reduces the clinical significance of the low sensitivity detected in the present study.

Psychiatric caseness is a construct of two different scales, which complicates an accurate translation into clinical practice. However, despite its artificial nature, we believe that the definition is still useful and illustrates that the diagnosing of mental disorders in primary care is a difficult task. It also suggests that differential diagnoses, within the psychiatric spectrum, should ideally be based on some form of psychometric instrument.

When the GPs read the answers to the patients' questionnaires, which contained the DSQ, their diagnosing of MDE was more accurate. This is obviously not a surprise, but indicates that the diagnostic capability reported in this manuscript is likely to be somewhat overrated compared with normal clinical practice.

Implications for future clinical practice

In a recent publication on the management of depression, the British National Institute for Health and Clinical Excellence (NIHCE) suggests screening for depression in high-risk patient categories within primary care [16]. These categories comprise patients with significant physical illnesses and also include patients with mental disorders. When considering the many misclassified patients in our study, we can only support screening initiatives but we suggest defining another high-risk category, namely that consisting of “psychiatric cases”.

Our results show that the intuitive perception of psychiatric disease is fine-tuned in GPs: When we applied the “psychiatric case” definition in the present study, the GPs' sensitivity in identifying MDE (not necessarily as MDE, but as “psychiatric case”) increased markedly. This result casts light on an important aspect in the diagnosing of MDE in primary care, namely that it is indeed a difficult task. However, the fact that as many as 92% of the patients with MDE were covered by the psychiatric case criteria indicates that diagnosing of MDE in primary care could be improved by screening for MDE in patients displaying psychiatric or medically unexplained symptoms [17]. Therefore, until the screening initiative suggested by NIHCE has been tested in a prospective trial, we suggest that GPs recommend patient self-administered screening for MDE [18] whenever they get a “hunch” that their patient may suffer from “something” within the mental disorder spectrum. This approach is likely to decrease the extent of both false-negative and false-positive diagnosing of MDE in primary care to the benefit of both patients and doctors.

Role of funding source

Wyeth AB Denmark funded the project monitors instructing the GPs. The funding was given as a non-conditional education grant. The project monitors were not allowed to promote the interests of the company during the study. Wyeth AB had no further role in the study design, in the analysis/interpretation of data, in the writing process or in the decision to submit the manuscript for publication.

Ethics committee

The National Ethics Committees of Denmark, Finland, Norway, and Sweden approved the study.

ERRATUM

[ePub ahead of print] 13 July 2010, DOI: 10.3109/02813432.2010.501235. The Early Online version of this article published ahead of print on 13 July 2010 contained an error in the abstract. The sentence “The corresponding numbers for the false-negative diagnoses were 0.5-2.5%.” should have read: “The corresponding numbers for the false-negative diagnoses were 0.9-2.5%.”

Acknowledgements

The authors would like to thank participating GPs and patients, and also Donald Smith (Centre for Psychiatric Research, Aarhus University Hospital, Risskov, Denmark) and Ivana Konvalinka (Center of Functionally Integrative Neuroscience, Aarhus University Hospital, Aarhus, Denmark) for linguistic advice.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.