917
Views
84
CrossRef citations to date
0
Altmetric
Review

Confusion assessment method: a systematic review and meta-analysis of diagnostic accuracy

, , &
Pages 1359-1370 | Published online: 19 Sep 2013

Abstract

Background

Delirium is common in the early stages of hospitalization for a variety of acute and chronic diseases.

Objectives

To evaluate the diagnostic accuracy of two delirium screening tools, the Confusion Assessment Method (CAM) and the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU).

Methods

We searched MEDLINE, EMBASE, and PsychInfo for relevant articles published in English up to March 2013. We compared two screening tools to Diagnostic and Statistical Manual of Mental Disorders IV criteria. Two reviewers independently assessed studies to determine their eligibility, validity, and quality. Sensitivity and specificity were calculated using a bivariate model.

Results

Twenty-two studies (n = 2,442 patients) met the inclusion criteria. All studies demonstrated that these two scales can be administered within ten minutes, by trained clinical or research staff. The pooled sensitivities and specificity for CAM were 82% (95% confidence interval [CI]: 69%–91%) and 99% (95% CI: 87%–100%), and 81% (95% CI: 57%–93%) and 98% (95% CI: 86%–100%) for CAM-ICU, respectively.

Conclusion

Both CAM and CAM-ICU are validated instruments for the diagnosis of delirium in a variety of medical settings. However, CAM and CAM-ICU both present higher specificity than sensitivity. Therefore, the use of these tools should not replace clinical judgment.

Introduction

Delirium is characterized by acute onset of an altered level of consciousness with fluctuating levels of orientation, memory, thought, and/or behavior.Citation1 It is commonly observed among patients with an acute medical condition, especially patients in the internal medicine, neurology, psychiatry, and geriatric wards. Delirium has been associated with unfavorable outcomes including higher mortality, longer hospitalization, and a greater degree of dependence after discharge.Citation2 Therefore, early recognition and prevention of delirium may improve outcomes in hospitalized patients.

Currently, two standard diagnostic criteria on delirium are the Diagnostic and Statistical Manual of Mental Disorders (DSM) IV criteriaCitation3 and International Classification of Diseases-10.Citation4 However, neither of these diagnostic tools can be easily applied to daily bedside practice. Therefore, a variety of screening tools such as the Confusion Assessment Method (CAM),Citation5 the Delirium Rating Scale,Citation6 the Intensive Care Delirium Screening Checklist,Citation7 and the Nursing Delirium Screening ScaleCitation8 have been developed. Among them, CAM is one of the most widely used diagnostic instruments for clinical and research purposes with proven psychometric properties.Citation9 It was developed by Inouye et al based on DSM III for the purpose of enabling nonpsychiatric trained clinicians to identify delirium.Citation5,Citation10 Ely et al developed the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU),Citation11,Citation12 an instrument specifically designed to assess nonverbal patients (ie, mechanically ventilated) based on the CAM algorithm. The objective of this study was to investigate whether a nonlanguage based method has impact on diagnostic accuracy of the scale and CAM-ICU performance in identifying delirium in a different spectrum of patients in comparison to CAM. Therefore, in this review, we compared CAM and CAM-ICU to DSM I V, a golden standard for delirium diagnosis applied to verbal versus nonverbal patients.

Materials and methods

Search sources and searches

Two reviewers (QS and LW) conducted a literature search of MEDLINE, EMBASE, and PsychInfo in March 2013. Computer searches based on keywords were conducted. References from previously retrieved articles were also searched. Details of search strategy can be found in Appendix 1.

Study selection

Research articles examining either CAM or CAM-ICU were included for review if they met the following inclusion criteria: (1) the study was designed as an observational, cross-sectional or case series study; (2) the reference test was DSM IV for delirium diagnosis; (3) diagnostic accuracy estimates were reported in the paper, including sensitivity, specificity, true positive, false positive, true negative, and false negative, or sufficient detail to derive these numbers; and (4) the study was written in English.

We excluded papers on prevalence of delirium without diagnostic accuracy data, reference test other than DSM IV, or if the definition of delirium was unclear in the original paper. shows a flow chart of the search.

Figure 1 Result of literature search.

Abbreviation: DSM-IV, Diagnostic and Statistical Manual of Mental Disorders IV.
Figure 1 Result of literature search.

Data extraction and quality assessment

Data were extracted to a form which included the following information: first author, year of publication, study population characteristics, name of tool, assessor of screening and reference test, diagnostic cut point used, and length of administration.

Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) guidelinesCitation13 were employed for this systematic review to assess the study quality. For the purposes of this systematic review, we decided that a low risk of bias was assumed if all the questions were answered “yes.” If an answer was either “no” or “unclear,” a high risk of bias or “unclear” was assigned to the corresponding domain. In the patient selection domain, we considered the studies which included conditions related to mental disorders (ie, dementia, psychosis) which mimic delirium as an “appropriate inclusion” because we believe a good scale can discriminate between people who have delirium and people who have other mental disorders. For the index domain, we considered that a threshold was prespecified in translation versions. In domain 4, flow and timing, we considered 3 hours between the index test and reference standard as a reasonable time interval due to the fluctuations of delirium presentation. We appraised other items following guidelines.Citation13 For the applicability domain, studies were considered to be “high risk of bias” if no explicit description of inclusion and exclusion criteria were reported.

All data extraction and quality assessment were conducted by two reviewers (QS and LW). Discrepancies were resolved by discussion. A third reviewer (GS) was consulted if discrepancies remained.

Data synthesis and analysis

The estimates for sensitivity, specificity, and negative and positive likelihood ratios were computed using bivariate modelsCitation14 and compared to the Moses-Littenberg method.Citation15 We used a bivariate model which preserves the two-dimensional nature of sensitivity and specificity and treats these two estimates as a paired index, thus accounting for the possibility of correlation which other methods do not address. In contrast to a traditional funnel plot, which uses straight lines for pooled estimates, the bivariate model produces an ellipse with a pooled mean sensitivity/specificity, 95% confidence interval, and 95% prediction ellipse. As both the bivariate model and the Moses-Littenberg method require a 2 × 2 data table, in this meta-analysis, we either retrieved the data directly from the study or generated numbersCitation16 based on the information presented in the original paper. Bivariate analyses were performed with PROC NLMixed with SAS (version 9.2, SAS Institute Inc, Cary, NC, USA) and figures were plotted using R (version 2.14, Lucent Technologies, Murray Hill, NJ, USA). We also built a summary receiver operating characteristic (SROC) curve using RevMan.Citation17 Statistical significance was set a priori at an alpha level of 0.05.

Results

Characteristics of studies

A total of 22 studiesCitation8,Citation11,Citation12,Citation18Citation36 were identified for this systematic review (). Nine studies examined CAM (n = 1,033) while 13 assessed for CAM-ICU (n = 1,409). Overall, the mean age of patients included in these studies ranged from 54 to 85 years old. The percentage of patients diagnosed with delirium varied from 14%Citation25 to 87%.Citation12 The most common study populations were ventilated intensive care unit patients,Citation11,Citation12 geriatric inpatients,Citation19,Citation21,Citation22 or postoperative patients.Citation8,Citation25,Citation26 Most studies examined the accuracy of delirium diagnoses made by nondelirium health professionals such as general practitioners, nurses, or trained research assistants compared to delirium experts such as psychiatrists or geriatricians. In these studies, both CAM and CAM-ICU could be administered as a quick questionnaire. A number of studiesCitation18,Citation21,Citation27,Citation29 focused on delirium which developed in the early stage of an acute disease while others monitored the patients from admission to discharge.Citation11,Citation12,Citation24

Table 1 Characteristics of studies included in systematic review

Study quality

There were no disagreements in quality assessment between the reviewers that affected the categorization of studies as high or low risk of bias (). All 22 studies were considered to have a low risk of bias with respect to the reference test part. However, 13 studies had a high risk of bias for the patient selection criteria due to the inappropriate exclusion of demented or psychotic patients who were easily confused with delirium patients. One quarter of studies did not explain whether the assessor was blinded either to the reference test or index test. Therefore, these potentially unblinded studies were assigned as “high risk” for bias. Approximately 20% of studies scored as unclear or a high risk of bias because the interval time between the two tests exceeded 3 hours. Most studies had high applicability. Only three studies had a high risk of bias for the patient selection criteria as a result of vague descriptions of inclusion and exclusion criteria.

Table 3 Quality assessment of included papers

Test accuracy of CAM and CAM-ICU

The psychometric properties of delirium screening tests are summarized in . Overall, CAM and CAM-ICU demonstrated similar performance characteristics when diagnosing delirium in both ventilated and nonventilated patients. The sensitivity of pooled CAM and CAM-ICU were 82% (95% confidence interval [CI]: 69%–91%) and 81% (95% CI: 57%–93%), respectively. The specificity of the two scales were 99% (95% CI: 87%–100%) and 98% (95% CI: 86%–100%), respectively. shows the bivariate summary estimates of sensitivity and specificity for CAM and CAM-ICU diagnostic accuracy. It also contains the 95% CI and prediction ellipses. The red dot shows the mean estimate of CAM-ICU, which was very close to CAM although the sensitivity was slightly lower. The solid line represents the 95% CI. We observed that CAM-ICU had a similar variance in comparison to CAM. The dotted lines represent the 95% prediction ellipses. As expected, CAM-ICU had a wider prediction range than CAM.

Figure 2 Bivariate estimate of sensitivity and specificity.

Notes: Bivariate summary estimates of sensitivity and specificity for delirium screening test with 95% confidence and prediction ellipses. Upper left dots represent sensitivity and specificity of the Confusion Assessment Method and the Confusion Assessment Method for the intensive Care Unit. Smaller ellipses represent 95% confidence interval of sensitivity and specificity. Larger ellipses represent 95% confidence interval of prediction sensitivity and specificity. Red color represents the Confusion Assessment Method for the intensive Care Unit and the black color represents the Confusion Assessment Method.
Abbreviations: CAM, Confusion Assessment Method; CAM-ICU, Confusion Assessment Method for the intensive Care Unit.
Figure 2 Bivariate estimate of sensitivity and specificity.

Table 2 Diagnostic accuracy of included screening test

shows the SROC curves for all 22 studies. Six studies were identified to deviate from the average ROC curves. Three studies had high specificity but low sensitivity while another three had moderate to high sensitivity and specificity. Otherwise, all other studies showed high values for both sensitivity and specificity. The funnel plot () shows the distribution of sensitivity and specificity across studies. A greater degree of homogeneity in specificity was observed across studies in comparison to sensitivity.

Figure 3 Summary receiver operating characteristic curve of delirium screen scales.

Figure 3 Summary receiver operating characteristic curve of delirium screen scales.

Figure 4 Funnel plot of delirium screen scales.

Abbreviations: TP, true positive; FP, false positive; FN, false negative; TN, true negative.
Figure 4 Funnel plot of delirium screen scales.

Due to a lack of an adequate number of studies, we could not perform subgroup analysis to investigate potential factors, such as patient characteristics, type of disease, and difference in assessors influence on accuracy. On the other hand, CAM shows a good reliability across the studies, ranging from 86% to 94%. CAM-ICU demonstrated more variance in reliability, ranging from 20% to 96% ().

Discussion

The diagnosis of delirium in different settings constitutes a challenge. Despite a variety of existing tools, limited information is available on the performance and psychometric properties of these tools. In the present systematic review and meta-analysis, we identified 22 papers with 2,449 patients which studied the accuracy of the two most commonly used tools (CAM and CAM-ICU) in the diagnosis of delirium over the past decade against the standard delirium diagnostic test (DSM-IV).

CAM and CAM-ICU are tools which can be administrated quickly, and have high sensitivity and specificity for early identification of delirium in a variety of hospitalized populations.

This review builds on previous work by Wei et alCitation37 in 2006. In addition to the incorporation of more recent studies, we also employed QUADAS-2 for quality assessment and compared two statistical analysis methods when examining the accuracy performance of CAM and CAM-ICU, which had not previously been done. Overall, CAM and CAM-ICU showed moderate to high sensitivity, high specificity, and moderate to high reliability. Both tests can be administrated by health professionals with appropriate training to obtain the reliable results. In contrast to Wei et al’s finding,Citation37 we observed higher specificity for both CAM and CAM-ICU in comparison to the sensitivity. This may be a result of differences in statistical methodology. However, the results shown in this paper support Wei et al’s recommendation that given the relatively low values of sensitivity (approximately 80%–85%), clinicians should not base delirium diagnoses solely on either CAM or CAM-ICU assessments; rather they should use the tests in addition to their clinical judgment.Citation37

It is not surprising that CAM and CAM-ICU demonstrated similar diagnostic accuracy as they are derived from the same algorithm. However, it is worth noticing that there are wider variances of sensitivity than specificity when applying these two screening scales. The CAM diagnostic algorithm is comprised of four components: (1) an acute onset of mental status changes of fluctuating course; (2) inattention; (3) disorganized thinking; and (4) an altered level of consciousness. The diagnosis of delirium is based on the presence of both component 1 and 2, and either 3 or 4. Missing any of these diagnostic criteria due to inadequate training in assessment may underestimate the percentage of delirium cases, especially hypoactive patients. Another possible reason for the variance in sensitivity is the difference in the rate of sedative drugs included in the studies which affects the diagnostic accuracy of delirium.Citation38

We also observed a wide range of reliability across studies, especially for CAM-ICU. It is plausible that sedativesCitation39,Citation40 widely used in critical care units were causing the fuctuation in delirium.Citation11,Citation12,Citation41 Therefore, it is important to adhere closely to the screening protocol with regard to the point of time assessment, observation period, as well as to provide sufficient training to medical or research staff when conducting delirium assessments.Citation42,Citation43

To our knowledge, this is the first systematic review and meta-analysis to evaluate CAM and CAM-ICU in comparison to DSM-IV using the bivariate model and Moses-Littenberg method. We appraised all studies with recently revised QUADAS-2 to assess quality. However, in the current review, we are only able to compare two of the most popular delirium screening tools. Further studies may be warranted to expand these findings.

Conclusion

Both CAM and CAM-ICU are validated instruments in the diagnosis of delirium in a variety of medical settings, including the emergency department, the postoperative recovery room, in palliative care, the stroke unit, and the rehabilitation unit. Health professionals with appropriate training can achieve similar accuracy to experts specializing in psychiatric evaluation. CAM and CAM-ICU both present higher specificity than sensitivity. Health professionals should be cautiously interpreting these results as superior sensitivity is expected to a screening scale. The incidence of delirium may be underestimated due to the relatively high rate of false negatives in a low sensitivity scale. Therefore, CAM and CAM-ICU instruments should not be relied on alone for diagnosis, but that application of clinical judgment (presumably based on application of DSM) is essential to diagnose the presence and severity of delirium.

Authors’ contributions

QS and JCM conceived the study. QS and LW performed the database searches. QS undertook the statistical analysis. QS contributed to the writing of the first draft of the manuscript. All of the authors contributed to and have approved the final manuscript. GS and JCM both contributed as senior authors.

Acknowledgments

The authors thank Michael Manno for assistance of figures preparation.

Disclosure

The authors report no conflict of interest in this work.

References

  • LipowskiZDelirium in geriatric patientsDelirium: Acute Confusional StatesNew YorkOxford University Press1990
  • ShiQPresuttiRSelchenDSaposnikGDelirium in acute stroke: a systematic review and meta-analysisStroke201243364564922267831
  • American Psychiatric AssociationDiagnostic and Statistical Manual of Mental Disorders Fourth Edition: DSM-IV-TRWashington. DCAmerican Psychiatric Association1994
  • World Health OrganizationThe ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic GuidelinesGenevaWorld Health Organization1992
  • InouyeSKvan DyckCHAlessiCABalkinSSiegalAPHorwitzRIClarifying confusion: the confusion assessment method. A new method for detection of deliriumAnn Intern Med1990113129419482240918
  • TrzepaczPTBakerRWGreenhouseJA symptom rating scale for deliriumPsychiatry Res198823189973363018
  • RadtkeFMFranckMOppermannSThe Intensive Care Delirium Screening Checklist (ICDSC) – translation and validation of intensive care delirium checklist in accordance with guidelinesAnasthesiol Intensivmed Notfallmed Schmerzther20094428086 German19199171
  • LuetzAHeymannARadtkeFMDifferent assessment tools for intensive care unit delirium: which score to use?Crit Care Med201038240941820029345
  • HigginsJPAltmanDGGotzschePCCochrane Bias Methods GroupCochrane Statistical Methods GroupThe Cochrane Collaboration’s tool for assessing risk of bias in randomised trialsBMJ2011343d592822008217
  • InouyeSKForemanMDMionLCKatzKHCooneyLMJrNurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratingsArch Intern Med2001161202467247311700159
  • ElyEWInouyeSKBernardGRDelirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU)JAMA2001286212703271011730446
  • ElyEWMargolinRFrancisJEvaluation of delirium in critically ill patients: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU)Crit Care Med20012971370137911445689
  • WhitingPFRutjesAWWestwoodMEQUADAS-2 GroupQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studiesAnn Intern Med2011155852953622007046
  • ReitsmaJBGlasASRutjesAWScholtenRJBossuytPMZwindermanAHBivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviewsJ Clin Epidemiol2005581098299016168343
  • MosesLEShapiroDLittenbergBCombining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerationsStat Med19931214129313168210827
  • SackettDLHaynesRBGuyattCHTugwellPThe selection of diagnostic testsSackettDLHaynesRBGuyattGHTugwellPClinical Epidemiology. A Basic Science for Clinical Medicine2nd edBostonLittle, Brown and Company1991
  • Review Manager (RevMan) [computer program] Version 5.0CopenhagenThe Nordic Cochrane Centre, The Cochrane Collaboration2008
  • FabbriRMMoreiraMAGarridoRAlmeidaOPValidity and reliability of the Portuguese version of the Confusion Assessment Method (CAM) for the detection of delirium in the elderlyArq Neuropsiquiatr2001592-A17517911400020
  • GonzalezMde PabloJFuenteEInstrument for detection of delirium in general hospitals: adaptation of the confusion assessment methodPsychosomatics200445542643115345788
  • HeoEYLeeBJHahmBJTranslation and validation of the Korean Confusion Assessment Method for the Intensive Care UnitBMC Psychiatry2011119421605375
  • LaurilaJVPitkalaKHStrandbergTETilvisRSConfusion assessment method in the diagnostics of delirium among aged hospital patients: would it serve better in screening than as a diagnostic instrument?Int J Geriatr Psychiatry200217121112111912461759
  • LeungJMLeungVWLeungCMPanPCClinical utility and validation of two instruments (the Confusion Assessment Method Algorithm and the Chinese version of Nursing Delirium Screening Scale) to detect delirium in geriatric inpatientsGen Hosp Psychiatry200830217117618291299
  • LinSMLiuCYWangCHThe impact of delirium on the survival of mechanically ventilated patientsCrit Care Med200432112254225915640638
  • MitasovaAKostalovaMBednarikJPoststroke delirium incidence and outcomes: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU)Crit Care Med201240248449022001583
  • RadtkeFMFranckMSchneiderMComparison of three scores to screen for delirium in the recovery roomBr J Anaesth2008101333834318603528
  • RadtkeFMFranckMSchustSA comparison of three scores to screen for delirium on the surgical wardWorld J Surg201034348749420066416
  • RyanKLeonardMGuerinSDonnellySConroyMMeagherDValidation of the confusion assessment method in the palliative care settingPalliat Med2009231404519010967
  • van EijkMMvan den BoogaardMvan MarumRJRoutine use of the confusion assessment method for the intensive care unit: a multicenter studyAm J Respir Crit Care Med2011184334034421562131
  • WongpakaranNWongpakaranTBookamanaPDiagnosing delirium in elderly Thai patients: utilization of the CAM algorithmBMC Fam Prac20111265
  • Gusmao-FloresDSalluhJIDal-PizzolFThe validity and reliability of the Portuguese versions of three tools used to diagnose delirium in critically ill patientsClinics (Sao Paulo)201166111917192222086522
  • ThomasCKreiselSHOsterPDriessenMAroltVInouyeSKDiagnosing delirium in older hospitalized adults with dementia: adapting the confusion assessment method to international classifcation of diseases, tenth revision, diagnostic criteriaJ Am Geriatr Soc20126081471147722881707
  • GuentherUPoppJKoecherLValidity and reliability of the CAM-ICU Flowsheet to diagnose delirium in surgical ICU patientsJ Crit Care201025114415119828283
  • AdamisDDimitriouCAnifantakiSValidation of the Greek version of confusion assessment method for the intensive care unit (CAM-ICU)Intensive Crit Care Nurse2012286337343
  • NeufeldKJHayatMJCoughlinJMEvaluation of two intensive care delirium screening tools for non-critically ill hospitalized patientsPsychosomatics201152213314021397105
  • RyanDJO’ReganNACaoimhRODelirium in an adult acute hospital population: predictors, prevalence and detectionBMJ Open201331
  • WangCWuYYuePDelirium assessment using Confusion Assessment Method for the Intensive Care Unit in Chinese critically ill patientsJ Crit Care201328322322923159138
  • WeiLAFearingMASternbergEJInouyeSKThe Confusion Assessment Method: a systematic review of current usageJ Am Geriatr Soc200856582383018384586
  • Gusmao-FloresDFigueira SalluhJIChalhubRAQuarantiniLCThe confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studiesCrit Care2012164R11522759376
  • RamsayMSavegeTSimpsonBGoodwinRControlled sedation with alphaxalone-alphadoloneBr Med J1974259206566594835444
  • RikerRRPicardJTFraserGLProspective evaluation of the Sedation-Agitation Scale for adult critically ill patientsCrit Care Med19992771325132910446827
  • BergeronNDuboisM-JDumontMDialSSkrobikYIntensive Care Delirium Screening Checklist: evaluation of a new screening toolIntensive Care Med200127585986411430542
  • VasilevskisEEPandharipandePPGirardTDElyEWA screening, prevention, and restoration model for saving the injured brain in intensive care unit survivorsCrit Care Med20103810S683S69121164415
  • SpronkPERiekerkBHofhuisJRommesJHOccurrence of delirium is severely underestimated in the ICU during daily careIntensive Care Med20093571276128019350214

Appendix 1: Search stategy

Database: Ovid MEDLINE

Searched March 11, 2013

1946 to March week 2 2013

Embase Classic+Embase <1947 to 2013 week 11>

Search Strategy:

PsycINFO 1806 to March week 2 2013

Search Strategy: