Inter-rater reliability among psychiatrists when assessing depression according to the Montgomery–Åsberg Depression Rating Scale

Abstract

Introduction

Montgomery–Åsberg Depression Rating Scale (MADRS) is a validated tool for rating the depth of depression. The structured interview guide for the MADRS (SIGMA) is an interview guide that has been developed in order to increase the inter-rater reliability. Patients often meet more than one psychiatrists during their hospitalization for depression. A divergent rating of depression between psychiatrists could affect both the treatment and the outcome. This makes knowledge of the inter-rater reliability among psychiatrists important.

Aim

The primary aim of this study was to measure the inter-rater reliability between psychiatrists when rating depression using the MADRS.

Methods

Ten in-patients, who were diagnosed with depression, were filmed while being interviewed using the SIGMA. The patients were after that instructed to rate themselves using the self-rating version of the MADRS. Ten psychiatrists rated the pre-recorded interviews according to the MADRS. The inter-rater reliability was measured using intra-class correlation (ICC).

Results

The mean ICC for the total MADRS score was 0.952 (95% CI 0.891–0.986; p<.001) and Cronbach’s alpha 0.961. ICC values for each item ranged between 0.866 and 0.978 (p<.001). Cronbach’s alpha ranged between 0.905 and 0.984. The ICC values, when comparing the psychiatrists rating to the patients rating, ranged between 0.307 and 0.809 (p<.001).

Conclusion

All of the ICC values in the study, except when comparing the psychiatrists rating to the patients self-rating, were considered to be excellent. This study confirms the findings of reliability found in similar studies which involved fewer raters and not exclusively psychiatrists.

Keywords:

• Psychometry
• depression
• reliability
• psychiatry
• MADRS

Acknowledgements

We would first and foremost like to thank all the patients who participated. Thank you for helping us to better understand this important aspect of the clinical practice and hopefully improve the quality of care for all psychiatric patients. We would also very much like to thank the psychiatrists who took time from their daily practice to help us make this study possible. Thank you Denada Aiff, Anna Hagstad, Lennart Meyer, Pouya Movahed, Niklas Nygren, Magnus Olander, Jarmila Pettersson, Lena Söderby and Olga Tsygankova. We would also like to thank the Psychiatric Department of Halland, Sweden, for lending its resources to make this study possible. Finally, we would like to thank The Royal College of Psychiatrists for permitting us to use the SIGMA in our study. For copyright information, see below.

MADRS © 1979 The Royal College of Psychiatrists. Montgomery, S.A. & Åsberg, M. (1979). A new depression scale designed to be sensitive to change. British Journal of Psychiatry, 134, 382–389. Written permission must be obtained from the Royal College of Psychiatrists for copying and distribution to others or for republication (in print, online or by any other medium).

SIGMA © 2008, 2011 The Royal College of Psychiatrists. Williams JBW, Kobak KA. Development and reliability of a structured interview guide for the Montgomery-Asberg Depression Rating Scale (SIGMA). Br J Psychiatry 2008;192:52–58. Brianne Brown, PsyD, contributed to this revision. Written permission must be obtained from the Royal College of Psychiatrists for copying, distribution to others, for replication (in print, online or by any other medium).

Disclosure statement

The authors declare that there are no conflicts of interest.

Additional information

Funding

This work was supported by the Region Halland.

Notes on contributors

Jonas Geijer

Jonas Geijer is a psychiatrist.

Amir Baigi

Amir Baigi has 28 years experiences in health surveys, evaluation and expertise in improving the health and wellbeing. His scientific network extend from Gothenburg University in west to Lund University in south of Sweden.

Harald Aiff

Harald Aiff is a psychiatrist with affective disorder and lithium treatment as fields of expertise. Chairman of a national task force for depression and anxiety disorders. He is the medical director of an affective inpatient care ward in Varberg, Sweden.