144
Views
9
CrossRef citations to date
0
Altmetric
Original Research

Discrepancies in Cornell Scale for Depression in Dementia (CSDD) items between residents and caregivers, and the CSDD’s factor structure

, &
Pages 641-648 | Published online: 04 Jun 2013

Abstract

Purpose

This validation study aims to examine Cornell Scale for Depression in Dementia (CSDD) items in terms of the agreement found between residents and caregivers, and also to compare alternative models of the Thai version of the CSDD.

Patients and methods

A cross-sectional study was conducted of 84 elderly residents (46 women, 38 men, age range 60–94 years) in a long-term residential home setting in Thailand between March and June 2011. The selected residents went through a comprehensive geriatric assessment that included use of the Mini-Mental State Examination, Mini-International Neuropsychiatric Interview, and CSDD instruments. Intraclass correlation (ICC) was calculated in order to establish the level of agreement between the residents and caregivers, in light of the residents’ cognitive status. Confirmatory factor analysis (CFA) was adopted to evaluate the alternative CSDD models.

Results

The CSDD yielded a high internal consistency (Cronbach’s alpha = 0.87) and moderate agreement between residents and caregivers (ICC = 0.55); however, it was stronger in cognitively impaired subjects (ICC = 0.71). CFA revealed that there was no difference between the four-factor model, in which factors A (mood-related signs) and E (ideational disturbance) were collapsed into a single factor, and the five-factor model as per the original theoretical construct. Both models were found to be similar, and displayed a poor fit.

Conclusion

The CSDD demonstrated a moderate level of interrater agreement between residents and caregivers, and was more reliable when used with cognitively impaired residents. CFA indicated a poorly fitting model in this sample.

Introduction

Elderly depressed people who experience cognitive impairment or dementia need to be assessed using special instruments. The Cornell Scale for Depression in Dementia (CSDD) is considered one of the best tools to use with cognitively impaired residents, and is thus widely used in nursing home and long-term care (LTC) facilities.Citation1 The validity of the CSDD has been investigated and substantiated, including cross-culturally.Citation1Citation5 One of the important features of the CSDD is that informants other than the residents themselves can be used to provide the data necessary for diagnosis; having agreement between sources of information is important, though only a few studies have addressed this issue.Citation6,Citation7 The CSDD comes in two formats: one for residents, and the other for caregivers, proxies, or nurses. Each has the same set of questions. The greater the level of agreement between items, the greater the potential level of overall interrater reliability and in turn increased likelihood of validity. Disagreement between items may lead to the need to consider revision of the scale. A number of studies have focused on the level of agreement between residents and caregivers; for example, Schreiner and MorimotoCitation3 and Amuk et alCitation8 found little agreement between residents and caregivers for the items “retardation” and “mood-congruent delusion” in either depressed or nondepressed residents. With regard to the level of agreement as it might vary with the cognitive status of the individual, Towsley et alCitation9 found a significant score discrepancy between residents and nurse proxies in that nurses evaluated residents as less depressed than residents evaluated themselves. They did find that the overall level of agreement between the nurses’ and residents’ CSDD scores was poor (r = 0.16). This is similar to the findings of Burrows et al.Citation7 These authors felt that the lack of strong agreement was, in part at least, a function of underreporting of symptoms in individuals with dementia; relying on nurse-proxy judgments may further worsen the problem of underdetection of depressive symptoms.Citation7,Citation9

The CSDD’s interitem correlation is also of interest. CSDD factors are drawn from consensus based on clinically relevant syndromes, rather than from statistical reasoning.Citation1 Harwood et al found that a four-factor solution accounted for 43.1% of the overall variance: general depression (lack of reactivity to pleasant events, poor self-esteem, pessimism, loss of interest, physical complaints, psychomotor retardation, and sadness), rhythm disturbances (difficulty falling asleep, multiple night awakenings, early morning awakenings, weight loss, and diurnal variations in mood), agitation/psychosis (agitation, mood-congruent delusions, and suicide), and negative symptoms (such as appetite loss, weight loss, a lack of energy, a loss of interest, and a minimal reaction to pleasant events).Citation10 Ownby et al found that the CSDD revealed the same four-factor structure, but also found differences in factor content for several items across Hispanic and Anglo-Saxon cultures.Citation4 In Asia, Schreiner and Morimoto examined use of the CSDD with Japanese poststroke residents and found a four-factor solution with some items loaded on to an undesignated factor, with the most prevalent symptoms being anxiety, irritability, and sadness, and the least prevalent being multiple physical complaints, appetite loss, and mood-congruent delusions.Citation3 Lin and Wang, studying 147 residents from institutionalized care facilities in Taiwan, found anxiety, sadness, and a subscale of ideational disturbance loaded onto the same factor, while a lack of some items linked to mood-related signs and behavioral disturbance were not loaded onto the expected factors.Citation5 Barca et al examined a large sample of 1159 residents and found a five-factor solution, as hypothesized; this is in contrast to the results reviewed above.Citation11 It should be noted that several items did not load onto their designated factors.Citation11

What does all of this lead to? As we know, the greater the CSDD score, the more severe the depressive condition; if some items are not relevant to the scale’s results (low interitem correlation), this may increase error and subsequently affect the score. This will in turn potentially adversely affect both the reliability of the scale and its accuracy (assessing by the area under the curve).Citation12 Since the CSDD concept uses both interrater and interitem agreement to develop a final score, as given by a clinician, both procedures are deemed important. The aims of our study were: (1) to explore the level of interrater agreement between the CSDD items, as rated by residents and caregivers, and (2) to compare the theoretical CSDD model with alternative models using this sample.

Materials and methods

In this study, we carried out a secondary analysis of data derived from a cross-sectional study into the prevalence of major depressive disorder (MDD) in an LTC facility in Chiang Mai, Thailand.Citation13 Data were collected between March and June of 2011. The study was approved by the ethics committee of the Faculty of Medicine, Chiang Mai University. In the study, a trained research nurse performed a diagnostic interview in order to evaluate for psychiatric and cognitive function disorders using the Thai version of the Mini-International Neuropsychiatric Interview (MINI) version 5.0, and the MiniMental State Examination Thai 2002 instrument (MMSE-Thai 2002).Citation14,Citation15 The interviews were conducted separately for each resident and caregiver; therefore, the interviewees were not aware of each other’s responses.

Sample

Eighty-four of the 111 residents in the LTC during the study period agreed to participate in the study, and all gave informed consent. Seven refused to participate, eight had active physical or psychiatric diseases that became an obstacle to interviewing, seven were unable to give intelligible answers, and five provided incomplete data. The final set of participants included 84 residents aged 60 years and over, and their five nonprofessional caregivers (LTC facility staff members).

Instruments

Mini-international neuropsychiatric interview

The MINI instrument, as developed by Sheehan et al,Citation16 was used here as the standard for diagnosing Diagnostic and Statistical Manual of Mental Disorders (fourth edition) major depression, while the Thai version was translated and validated by Kittiratanapaiboon and Khamwongpin.Citation17 It has kappa ranges of between 0.27 and 0.87 to suggest the presence of depressive disorders. The research nurse administered the MINI across all participants and was not aware of the results of the CSDD carried out by each patient. The MINI and CSDD assessments were kept blind and independent of each other.

Cornell scale for depression in dementia

The CSDD is divided into five subscales: (1) mood-related signs (anxiety, sadness, a minimal reaction to pleasant events, and irritability), (2) behavioral disturbance (agitation, psychomotor retardation, multiple physical complaints, and a loss of interest), (3) physical signs (a loss of appetite, weight loss, and a lack of energy), (4) cyclic functions (diurnal variation in mood, difficulty falling asleep, multiple awakenings during the night, and early morning awakening), and (5) ideational disturbance (suicidal tendencies, low self-esteem, pessimism, and mood-congruent delusions).Citation1 It contains nineteen questions, each of which can be given a score ranging from 0 (absent) to 2 (severe), or a “score” (result) of “symptoms not possible to evaluate.” The total score ranges from 0 to 38; a higher score denotes greater levels of depression. The CSDD scale was carried out by a trained psychogeriatric research nurse, who interviewed each resident and each caregiver within the same day. The research nurse also administered the MINI (diagnosing for depression).

The Thai version of the CSDD was developed following the translation and cultural adaptation method, a process that included an initial translation into Thai by a geriatric psychiatrist (NW), followed by a backward translation into English by a bilingual translator (in this case, a university professor) who had no prior knowledge of the questionnaire.Citation18 The two versions produced were then assessed and compared item by item by the authors and the bilingual translator. The resulting final draft was then field-tested with depressed elderly residents in a geriatric outpatient clinic (this group was not involved with the current study).

MMSE-Thai 2002

The MMSE-Thai 2002 was modified from the original version by Folstein et al to measure cognitive impairment in Thai people.Citation15,Citation19 Information regarding the participants’ level of education is required for the interpretation of cognitive impairment or dementia of this version of MMSE. The total score for participants who had at least elementary schooling is 30. The cutoff score for cognitive impairment or dementia is 22. For those who did not complete elementary schooling, the cutoff score is 17. The total score for participants who are illiterate is 23, and for these participants the cutoff score is 14.

Data analysis

Means and standard deviations were used, and t-tests were employed when comparing the two groups. Intraclass correlation coefficients (ICCs) were calculated in order to determine the level of absolute agreement between the scores for each item given by the residents and by the caregivers, using AgreeStat software version 2011.3 (Advanced Analytics, Gaithersburg, MD, USA).

Confirmatory factor analysis (CFA) was used to compare the hypothesized five-factor model and an alternative model, as follows: (1) a five-factor model as per the original theoretical construct, (2) a four-factor model in which factor A (mood-related signs) and factor E (ideational disturbance) were collapsed into a single factor (as supported by prior studies, reviewed above).

These models were compared in terms of the level of model fit. The fit statistics used included a comparative fit index score of ≥0.95, nonnormed fit index or Tucker–Lewis index values of ≥0.9, a root-mean-square error of approximation of ≤0.06 with values as high as 0.08 indicating a reasonable fit, a standardized root-mean-square residual of ≤0.08, as well as the results of equation χ2/degrees of freedom <3.Citation20Citation23 Amos 18 (IBM, Armonk, NY, USA) was used for the analysis.Citation24

The level of reliability of the model’s internal consistency was assessed using Cronbach’s alpha. The recommended cutoff criterion for the coefficient alpha is 0.70. In this study, an alpha level of 0.05 was required. Missing data were found in the response of only one of the caregivers, for whom three of the CSDD items were missing. These missing items were replaced using the lowest possible score (zero). SPSS version 17.0 (SPSS Inc., Chicago, IL, USA) was used for the analysis.Citation25

Results

shows the demographic data, descriptive clinical data (eg, MDD diagnosis), and the CSDD scores. Most of the respondents were female (55%). Prior to admission to the facility, 34.5% were single, 8.3% were married, 2.4% were separated, 3.6% were divorced, and 63% were widowed. Most had an elementary school-level education, with a mean of 3.5 years in school. The mean and standard deviation of the MMSE scores were 18.2 and 7.0, respectively, and 42% met the criteria for cognitive impairment, according to their level of education. Twenty-five of the 84 participants (29.8%) had MDD (see ), as diagnosed with the MINI.

Table 1 Clinical characteristics of the sample (n = 84)

shows the mean caregiver and resident total CSDD scores. It was found that the mean resident total CSDD scores were not significantly different from those of the caregivers (mean [standard deviation] = 5.1 [4.51] vs. 4.46 [4.33], t = 1.51, P = 0.136). The scores for factors B and C were not significantly different either. Cronbach’s alpha for the entire group was found to be 0.87, while for residents and caregivers only, the scores were 0.84 and 0.86, respectively.

Table 2 Means, SDs, and mean differences of CSDD scores between caregivers and residents (caregiver CSDD score – resident CSDD score): subscales and totals

Level of agreement between residents and caregivers

In order to compare the level of agreement for each CSDD item response given by the residents and caregivers, ICC was calculated for all participants, revealing a fair-to-moderate agreement for 14 of the items (between 0.3 and 0.6). In the cognitively impaired group, eleven items were found to have significant agreement (sadness, irritability, retardation, multiple physical complaints, loss of interest, lack of energy, difficulty falling asleep, multiple awakenings, suicidal ideation, self-deprecation, and mood-congruent delusions) compared to only five items in the cognitively intact group (irritability, retardation, multiple physical complaints, early morning awakening, and self-deprecation; ). In terms of the total CSDD score, there was strong agreement (ICC = 0.71) between caregivers and residents in the cognitively impaired group, but only a fair level of agreement in the cognitively intact group (ICC = 0.32).

Table 3 ICC values for the residents’ and caregivers’ scores, according to the level of cognitive functioning

Factor structure of the CSDD

Before comparing the models, the factor-analytic study results from the present study (using residents and caregivers combined) were compared with five previous findings.Citation3,Citation5,Citation10,Citation11,Citation20 All studies used factor analysis with the principal-components method and a varimax rotation. shows that the results seem to have been consistent among studies. Items 5–8 from factor B (behavioral disturbance) appear to be incorrectly loaded, especially item 7. All the studies found that item 7 was not loaded onto factor B (behavioral disturbance), as expected, and four out of the six studies found item 7 was not loaded onto any factor (see the items and the factors in bold). Disagreement was also found in items 3, 4, 9, 12, 16, and 19. All studies came up with a five-factor solution, except for HarwoodCitation10 and KurlowiczCitation26 (four-factor); however, these five-factor solutions were not the same as that produced by Alexopoulos,Citation1 since factors A (mood-related signs) and E (ideational disturbance) were loaded onto the same factor, while the fifth factor was comprised of the problematic items mentioned above. In fact, the fifth factor was loaded with different items, those that varied from study to study. It should be noted that item 4 (irritability) and item 5 (agitation) were loaded onto the same factor across all the studies (see items highlighted in grey). Item 19, mood-congruent delusions, generally loaded on factor A, while in the present study it did not load on any factor.

Table 4 Comparison of studies regarding how each item loaded on to a particular factor

shows the CFA results comparing model fit between models. Both the five-factor and four-factor models were found to be similar and displayed a poor fit to the data, as they were not able to reach threshold criteria (comparative fit index = 0.76–0.79, Tucker–Lewis index = 0.72–0.75, root-mean-square error of approximation = 0.089–0.094, and standardized root-mean-square residual = 0.090–0.097).

Table 5 Confirmatory factor analysis of the CSDD: overall model fit (without correlated residuals)

Discussion

The CSDD demonstrated a moderate level of interrater agreement between residents and caregivers, and was more reliable when used with cognitively impaired residents. These data support the use of proxy informants (eg, staff members) in the assessment of depression utilizing the CSDD in elderly populations, particularly those with cognitive impairment. The CFA indicated a poorly fitting model, suggesting that the hypothesized model was not a good fit with the observed data.

We found a few studies that compared caregiver and resident ratings using the CSDD in a nursing home setting.Citation3,Citation9,Citation27 Compared to Towsley et al,Citation9 we found differences in three out of the five subscale scores and between the two groups of informants, even though there was no difference in the mean total scores. It may be that this is attributable to caregivers’ (or nurses’) limitations in observing the residents’ signs and symptoms. Our findings yielded a fair level of agreement in the mean total scores (between the residents and caregivers), which were consistent with the results of Schreiner and Morimoto.Citation3

The outstanding feature of the CSDD was a strong level of agreement between the answers given by cognitively impaired residents and those given by caregivers, denoting that information given by the caregivers was similar to that given by the group of residents. This finding supports the use of a proxy, as suggested by Leontjevas et al.Citation6 This may be explained by the fact that in such settings, caregivers may pay more attention to those residents who are obviously vulnerable, such as those who have physical disabilities or cognitive problems (regardless of whether these conditions are caused by dementia or depression). Likewise, stroke residents suffering from cognitive disability may tend to be more closely observed, leading to a high number of significant correlations and a high level of agreement between the caregivers and the residents in terms of either their behavioral (eg, retardation) or psychological symptoms (eg, sadness, self-deprecation, and suicidal ideation).Citation3 This clearly demonstrates the distinctive benefit of the CSDD in detecting depression in residents or patients with cognitive impairment.

The CFA also underscored how poor some factors are in relation to the whole set of items, and this confirms the previous findings of Leontjevas et alCitation6 and Schreiner and Morimoto.Citation3 The item “multiple physical complaints” does not distinguish well between depressed and nondepressed residents; in fact, it was shown either not to load to any factor at all, or to load to an unintended factor.Citation3,Citation28 Harwood et al, meanwhile, found it was loaded on to “general depression,” which comprises a mix of items: lack of reactivity to pleasant events, poor self-esteem, pessimism, loss of interest, physical complaints, psychomotor retardation, and sadness.Citation10 This finding was also supported by a study carried out in Asia by Lin and Wang, in which cross-loadings for the items of irritability, multiple physical complaint, and loss of interest were evident.Citation5

Even though multiple physical complaints are a common symptom among elderly depressed Thais, this item may not be helpful in differentiating depression from nondepression, and in fact may contribute to poor construct validity within the CSDD.Citation29,Citation30 Even in a larger sample size, as studied by Barca et al and Kurlowicz et al, the same problem with this item (multiple physical complaints) was found.Citation11,Citation26 It is interesting to note that in addition to multiple physical complaints, both studies also found evidence of cross-loading for loss of interest, and for diurnal variation in mood. These latter findings were also found in studies using a smaller sample size.Citation3,Citation10

All studies found it difficult to differentiate between the items under factor E (ideational disturbance) and those under factor A (mood-related signs), as both these sets of symptoms reflect the presence of core depression, as found in Ownby et al’s and Schreiner and Morimoto’s studies.Citation3,Citation4 Mood-congruent delusion was found to be loaded in an inconsistent way depending on the nature of the sample. The level of depression in our sample may not have been so severe as to make this item successfully exhibit discriminatory performance, and it tends to perform better with a larger sample or with a sample with different characteristics, as found in other studies.

The use of a small sample with low levels of depression impacts on the generalizability of the results of this study; however, there is some evidence from a number of findings () to suggest that the CSDD may have better construct validity and model fit with minor revising, ie, grouping irritability and agitation together (regardless of what factor they are in), and taking out multiple physical complaints and diurnal variation in mood, since they are less likely to be able to measure actual depression.

Multiple physical complaints, diurnal variation in mood, and mood-congruent delusion are important and relevant in specific clinical cases. If they are retained within the CSDD scale for this reason, we suggest excluding these items from any calculation of the total CSDD score during the screening or measuring for outcome of depression (due to the finding of lack of discriminatory power and potential for reduction in accuracy).

The strength of this study is that we used only trained geriatric nurses to conduct the interviews with both the residents and caregivers. Using this method helped shed light on differences between informants when interviewed; however, this method may also be viewed as representing bias, as it may have inflated the level of agreement shown.

Limitation

This study did have some limitations. First, the sample size was relatively small, which precluded precision in terms of the point estimates and limited the study’s power, which may ultimately have impacted upon the factor structure. In addition, a test–retest analysis was not performed to ensure this type of reliability.

Conclusion

The CSDD demonstrated moderate interrater agreement between residents and caregivers, and was more reliable when used in cognitively impaired residents. Neither the five-factor model, as per the original theoretical construct, nor the four-factor model, in which factor A (mood-related signs) and factor E (ideational disturbance) were collapsed into a single factor, demonstrated a good fit.

Acknowledgments

NW and TW played an equal part in conceiving and designing this study and writing the manuscript. Statistical analysis was performed by TW. RvR assisted with methodology, critical analysis, and writing. All authors approved the final manuscript. The authors owe a great debt of gratitude to Ms Kanlaya Kaewsom, our former research nurse, and all the residents and caregivers at TWDCOP for their cooperation during the data-collection activities.

Disclosure

The authors report no conflicts of interest in this work.

References

  • AlexopoulosGSAbramsRCYoungRCShamoianCACornell Scale for Depression in DementiaBiol Psychiatry19882332712843337862
  • SnowdonJRosengrenDDanielFSuyasaMAustralia’s use of the Cornell scale to screen for depression in nursing homesAustralas J Ageing2011301333621395938
  • SchreinerASMorimotoTFactor structure of the Cornell Scale for Depression in Dementia among Japanese poststroke patientsInt J Geriatr Psychiatry200217871572212211120
  • OwnbyRLHarwoodDGAcevedoABarkerWDuaraRFactor structure of the Cornell Scale for Depression in Dementia for Anglo and Hispanic patients with dementiaAm J Geriatr Psychiatry20019321722411481129
  • LinJNWangJJPsychometric evaluation of the Chinese version of the Cornell Scale for Depression in DementiaJ Nurs Res200816320221018792890
  • LeontjevasRGerritsenDLVernooij-DassenMJSmalbruggeMKoopmansRTComparative validation of proxy-based Montgomery-Åsberg Depression Rating Scale and Cornell Scale for Depression in Dementia in nursing home residents with dementiaAm J Geriatr Psychiatry2012201198599321989316
  • BurrowsABSatlinASalzmanCNobelKLipsitzLADepression in a long-term care facility: clinical features and discordance between nursing assessment and patient interviewsJ Am Geriatr Soc19954310111811227560702
  • AmukTKaradağFOğuzhanoğluNOğuzhanoğluAReliability and validity of the Cornell Scale for Depression in Dementia in an elderly Turkish populationTurk Psikiyatri Derg2003144263271 Turkish14704928
  • TowsleyGNeradilekMBSnowALErsekMEvaluating the Cornell Scale for Depression in Dementia as a proxy measure in nursing home residents with and without dementiaAging Ment Health201216789290122486638
  • HarwoodDGOwnbyRLBarkerWWDuaraRThe factor structure of the Cornell Scale for Depression in Dementia among probable Alzheimer’s disease patientsAm J Geriatr Psychiatry1998632122209659954
  • BarcaMLSelbaekGLaksJEngedalKThe pattern of depressive symptoms and factor analysis of the Cornell Scale among patients in Norwegian nursing homesInt J Geriatr Psychiatry200823101058106518457344
  • WongpakaranNWongpakaranTReliability analysis – application in clinical practiceVaccines: Benefits and Riskshttp://iconceptpress.com/download/paper/11120101021646.pdfAccessed Mar 12, 2013
  • WongpakaranNWongpakaranTPrevalence of major depressive disorders and suicide in long-term care facilities: a report from northern ThailandPsychogeriatrics2012121111722416824
  • UdomratnPKittirattanapaiboonPManual for MINI (Thai version)Chiang MaiI Am Organizer and Advertizing2004
  • Thai Cognitive Test Development Committee 1999Mini-Mental State Examination-Thai 2002BangkokInstitute of Geriatric Medicine, Department of Medical Services, Ministry of Public Health2002
  • SheehanDLecrubierYSheehanKThe Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10J Clin Psychiatry199859Suppl 202233 quiz 34–579881538
  • KittirattanapaiboonPKhamwongpinMThe validity of the Mini International Neuropsychiatric Interview (M.I.N.I.)-Thai VersionUdomratnPKittirattanapaiboonPManual for MINI (Thai version)Chiang MaiI Am Organizer and Advertizing20041321
  • HarknessJVillarAEdwardsBTranslation, adaptation, and designHarknessJBraunMEdwardsBJohnsonTLarsESurvey Methods in Multicultural, Multinational, and Multiregional ContextsHyattsville (VA)Erlbaum2010117140
  • FolsteinMFFolsteinSEMcHughPR“Mini-mental state.” A practical method for grading the cognitive state of patients for the clinicianJ Psychiatr Res19751231891981202204
  • HuLBentlerPMEvaluating model fitHoyleRHStructural Equation Modeling: Concepts, Issues and ApplicationsThousand Oaks (CA)Sage19957699
  • HuLBentlerPMFit indices in covariance structure modeling: sensitivity to underparameterized model misspecificationPsychol Methods199834424453
  • HuLBentlerPMCut off criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternativesStruct Equ Modeling199961155
  • KlineRBPrinciples and Practice of Structural Equation ModelingNew YorkGuilford1998
  • ArbuckleJAmos (version 18.0) [computer program]Armonk (NY)IBM2009
  • SPSSIncSPSS Statistics for Windows, Version 170 [computer program]Chicago (IL)SPSS Inc2008
  • KurlowiczLHEvansLKStrumpfNEMaislinGA psychometric evaluation of the Cornell Scale for Depression in Dementia in a frail, nursing home populationAm J Geriatr Psychiatry200210560060812213695
  • WatsonLCZimmermanSCohenLWDominikRPractical depression screening in residential care/assisted living: five methods compared with gold standard diagnosesAm J Geriatr Psychiatry200917755656419554670
  • LeontjevasRvan HoorenSMuldersAThe Montgomery-Asberg Depression Rating Scale and the Cornell Scale for Depression in Dementia: a validation study with patients exhibiting early-onset dementiaAm J Geriatr Psychiatry2009171566419092312
  • WongpakaranTWongpakaranNBoripuntakulTSymptom checklist-90 (SCL-90) in a Thai sampleJ Med Assoc Thai20119491141114921970206
  • WongpakaranNWongpakaranTvan ReekumRSocial inhibition as a mediator of neuroticism and depression in the elderlyBMC Geriatr20121214122856615