The psychiatric assessment schedule for adults with developmental disability checklist: reliability and validity of Turkish version

ABSTRACT

Aim/background: There is a lack of psychometric instruments to measure psychopathology in people with intellectual disabilities (ID). This may lead to underdiagnosis of treatable psychiatric comorbidities in this population. Psychiatric assessment schedule for adults with developmental disabilities (PAS-ADD) Checklist was developed as a screening tool designed for lay people. The PAS-ADD Checklist was created in English and was later on validated for other languages, including French and German. There is no psychometric measure available in Turkish to screen for or detect psychiatric symptoms in adults with ID. The aim of the present study was to present a psychometric evidence of the Turkish language version of the PAS-ADD Checklist. This is the first study aiming to present and validate a psychiatric screening instrument for adults with ID in Turkey.

Methods: The Turkish version was developed by translation and back translation. The checklist and a structured purpose-designed socio-demographic form were administered to 151 adult individuals with ID of whom 71 were living in care homes, 80 at family home. They all underwent face-to-face psychiatric assessment through clinical examination. The Turkish version of the Checklist was compared with the original English and other language versions, and the psychometric properties were presented. Internal consistency, test–retest reliability, inter-rater reliability, factorial analysis and sensitivity/specificity were calculated.

Results: The validity and reliability of the Turkish version seem acceptable with good psychometric properties. The item analysis of the total scale showed strong consistency with Cronbach’s alpha of 0.75. Test–retest reliability for different items (rs: between 0.50 and 0.69) was better than inter-rater reliability (rs: between −0.15 and 0.75) which could be expected given that raters are non-specialist people. Sensitivity and specificity were calculated for the number of participants who did and did not cross threshold and for whom a psychiatric disorder was or was not present. The sensitivity was 78.7% and specificity was 78.8%. An explanatory principal component factor analysis was conducted on the Turkish version of the Checklist revealed six factors. This six-factor solution explained 54% of variance. The likely reasons for the findings are discussed.

Conclusion: Overall, the Turkish version of the PAS-ADD Checklist is an acceptable generic screening tool considering the difficulty of detecting mental health problems in people with ID and the lack of Turkish screening instrument. The Turkish version of the PAS-ADD Checklist can be recommended as a general screening tool for psychiatric disorders in adults with ID. It will contribute to early diagnosis and management of mental health problems and therefore improve the quality of life of those with ID and their family/care givers.

KEYWORDS:

• Adults
• intellectual disabilities
• psychiatric symptoms
• mental disorders
• tests
• screening

View correction statement:

Corrigendum

Introduction

Studies report that people with intellectual disabilities (ID) develop mental illness at rates similar to or higher than the general population [1]. This is not surprising considering the risk factors they possess for mental illnesses. They have got predisposing biological factors, communications problems and increased exposure to psychosocial stressors, such as stigmatization, social deprivation, financial problems, physical comorbidities/disabilities and trauma experience.

Epidemiological studies of mental health problems among adults with ID report a wide range of prevalence figures from 13.9% to 75.2% [1]. There are several possible explanations for this; studies using different methodologies, definition of mental disorders and biased samples, among others.

Mental illness is not easy to diagnose in people with ID because of inherent cognitive and communication deficits and the clinical presentation of symptoms can be atypical in this population. It is possible that many individuals with ID have psychiatric problems that are not identified or treated. Another issue is the fact that psychiatric classification systems are developed for general population and may not be easily applicable for people with ID. Hence, it is of paramount importance to detect potential cases for further assessment and treatment which is a crucial step in meeting the needs of people with ID [2].

Psychiatric assessment schedule for adults with developmental disorders (PAS-ADD) Checklist was designed as a screening tool for use with adults with ID to help with the detection of possible cases. It was developed for non-specialists such as families and care givers. Items are designed using simple language so the checklist can be completed by non-specialists who know the person with ID but have no training in psychopathology. The checklist was developed as a 29-item version [3] and revised to a 25-item version [4]. It is scored on a four-point scale about psychiatric symptoms observed in the past four weeks. The 25 items of the PAS-ADD Checklist result in five scores (A, B, C, D, E) which are combined into three final subscales – (1) possible organic condition (A + B), (2) possible affective or neurotic disorder (B + C + D) and (3) possible psychotic disorder (E). Each diagnostic category has a threshold value at or over which detailed clinical or formal mental health assessment is needed.

The PAS-ADD Checklist was designed and validated for English-speaking countries. It was recently validated for German- and French-speaking populations by studies carried out in those countries [5,6]. The present study aimed to translate the Checklist and validate it for Turkish-speaking populations.

Methods

Ethical approval was obtained. Pavilion, the publisher of the PAS-ADD Checklist since 2002, was contacted which accepted the Turkish translation for the purpose of this study.

Translation

The English version of the PAS-ADD Checklist was translated into Turkish separately by two clinicians, one of whom has experience of working with people with ID in the UK and the other with experience of working in public health in the UK. Both translators were blind to the other clinician’s translation. The consensus forward translation was back translated into English by a native English speaker who is fluent in Turkish. Additionally, the original author of the PAS-ADD Checklist was contacted for items needing some clarification during this process.

Data collection

A total of 151 adults with ID were identified over a period of March 2015 to February 2016. Twenty-five of those were the patients who came to neurodevelopmental psychiatry outpatients for the first time for diagnosis and treatment. Seventy-one participants were identified from care homes and 55 from a community rehabilitation center for people with ID.

Every informant (families/care givers) was given instructions on how to fill out the checklist.

Participants

Participants in this study included 151 adults aged between 18 and 88 years (mean = 29.3). Of the 151 participants, 45 (30%) were women with a mean age of 29.8 and 106 (70%) were men with a mean age of 29. Eighty (53%) of them lived with their family and 71 (47%) were placed in a care home.

In Turkey most adults with ID do not have a formal IQ test conducted to specify the level of their disability. The severity of their ID is usually determined on the basis of their communication skills and the amount of support they need. We therefore used only two categories similar to German study to define the level of ID [5]. Participants who were able to express themselves verbally, managed to be interviewed by themselves and showed lower needs of support in the activities of daily living were rated as having a mild/moderate level of ID. Subjects who could not be interviewed or speak for themselves and who showed a higher need of support were rated as having severe/profound ID.

Data analysis

In the present study, informants completed the 25-item Turkish version of PAS-ADD Checklist and a structured, purpose-designed socio-demographic form. The informants had known the participant with ID for at least six months (Appendix).

In order to evaluate internal consistency, Cronbach’s alpha was computed for the five scales and three threshold scores. To test inter-rater reliability, we randomly selected 25 participants from our sample and asked 2 key informants who have known the person well for at least 6 months to fill out the Checklist on the same person. One of these key informants was usually the mother of the person, and the other ones were the other parent, family members, a teacher or a paid care giver. Out of 25, there were 17 forms returned to us. For the test–retest reliability, on another randomly selected 25 participants, we asked the main care giver of the participant to fill out the Checklist twice with a maximum of two-week interval. Out of this 25, 17 care givers filled out the Checklist on the second occasion.

Validity was tested with sensitivity-specificity analysis on 151 participants by comparing the threshold crossed and the presence of current psychiatric diagnoses as suggested by previous studies [7,8]. Diagnoses were established via a comprehensive clinical evaluation conducted by a psychiatrist specialized in psychiatry of ID. Current diagnoses are considered as the gold standard.

A factor analysis was conducted on the whole sample of 151 participants who had completed the questionnaires. One question (weight change) had no effect and therefore was not included in the analysis. An explanatory principal component factor analysis was conducted on the PAS-ADD Checklist.

The data analysis was conducted using the Statistical Package for the Social Sciences version 23.

Results

Internal consistency

It is assumed that, to a certain degree, the items on a scale are measuring a common entity. Therefore, Cronbach’s alpha was calculated for three threshold scores and for the five subscales in order to evaluate internal consistency (Table 1).

Table 1. Internal consistencies of the threshold scales and the total score.

Scale	Cronbach’s alpha	Number of items
A	0.29	2
B	0.64	4
C	0.65	9
D	0.53	6
E	0.37	3
Organic (A + B)	0.58	6
Affective/neurotic (B + C + D)	0.76	19
Psychotic (E)	0.37	3
Total (A + B + C + D + E)	0.75	24

Cronbach’s alpha for the total scale item analysis was 0.75, which indicates a strong consistency. The subscales with a small number of items in the present study revealed a smaller value of alpha, as in subscale A (two items) and subscale E (three items).

As demonstrated in Table 2, our Turkish version has similar alpha values to the German study apart from the psychotic scale.

Table 2. Comparison of Cronbach’s alpha of the Turkish PAS-ADD Checklist with other studies.

PAS-ADD Checklist	Number of items	Present study: Turkish n = 151	Zeilinger, Weber and Haveman (2011) German, n = 269	Gerber and Carminati (2013) French, n = 134	Moss et al. (1998) English, n = 201
Organic	6	0.58	0.55	0.76	0.84
Affective/neurotic	19	0.76	0.81	0.81	0.63
Psychotic	3	0.37	0.57	0.72	0.51
Total scale	24	0.75	0.83	0.84	0.87

Inter-rater reliability

The checklist was completed by 2 raters during the same time frame for 17 out of randomly selected 25 participants. Spearman’s Rank correlations were calculated for 17 pairs of raters for the 3 threshold scores (Table 3).

Table 3. Inter-rater reliability.

			Organic	Neurotic/affective	Psychotic
Spearman’s rho	Organic	rs	0.750
		p	0.001
		n	17
	Affective or neurotic	rs		−0.157
		p		0.547
		n		17
	Psychotic	rs			0.585
		p			0.014
		n			17

Note: rs: Spearman’s Rank correlation coefficient.

Results show that correlation was strong for organic disorders threshold, moderate for psychotic disorders threshold; however, there was weak, statistically non-significant correlation for affective/neurotic disorder scales between two raters (r = −0.157, p = .547) (Table 4).

Table 4. Test–retest reliability.

			Organic	Neurotic/affective	Psychotic
Spearman’s rho	Organic	rs	0.521
		p	0.032
		n	17
	Neurotic/affective	rs		0.507
		P		0.038
		n		17
	Psychotic	rs			0.696
		p			0.002
		n			17

Note: rs: Spearman’s Rank correlation coefficient.

Test–retest reliability

Test–retest reliability was evaluated for another randomly selected 25 participants. We asked the raters to fill out the checklist again on the same person within two-week time. Seventeen forms were returned to us. Spearman’s Rank correlations were calculated for three subscale thresholds. It showed a moderate level of agreement, the lowest being 0.50.

Validity

Because the PAS-ADD Checklist did not cover all the psychiatric disorders, the breakdown of participants who had a diagnosis covered or not covered by the Checklist is presented in Table 1. Some participants had more than one diagnosis (Table 5).

Table 5. Diagnoses based on clinical assessment by psychiatrists.

Diagnostic category	n (total N: 151)	%
Disorders covered by the PAS-ADD checklist
Psychotic disorder	8	5.4
Affective/neurotic	50	33.1
Organic disorder (dementia)	1	0.7
Disorders not covered by the PAS-ADD
Attention deficit hyperactivity disorder	16	10.6
Autism spectrum disorders	2	13.9
Problem behavior	52	34.4

Validity was tested with sensitivity and specificity analysis on 151 participants by comparing the threshold crossed and the presence of current psychiatric diagnoses, as suggested by Moss et al. [7] and Sturmey et al. [8]. Current diagnoses were considered as the gold standard which was established via clinical evaluation by a psychiatrist experienced in ID (Table 6).

Table 6. Crosstab for the presence or absence of psychiatric disorders and PAS-ADD Checklist threshold.

	Psychiatric disorders covered by PAS-ADD	Psychiatric disorders not present or not covered by PAS-ADD
Threshold crossed			Total
Yes	37	22	59
No	10	82	92
Total	47	104	151

Sensitivity, which is the probability to identify positive results, was 78.7%. The specificity which is the probability to identify negative results was 78.8%.

Factor analysis

An explanatory principal component factor analysis was conducted on the PAS-ADD Checklist revealed six factors. This six-factor solution explained 54% of variance and the Kayser–Meyer–Olkin measure of sampling adequacy was 0. 64 which indicate a moderate fit of data to the model. This analysis showed a resemblance to German version of the checklist [5] and to the structure found by Moss et al. [7]. However, it led to a different structure compared to the French version [6] and to the study by Sturmey et al. [8]. There could be several factors leading to these variations, including using different methods in different studies for factor analysis, not considering the level of IQ which can act as an unstable factor (Table 7).

Table 7. Factor analysis.

	Factor
Item	1	2	3	4	5	6
Loss of interest	0.83
Sad or down	0.73
Loss of appetite	0.71
Avoids social contact	0.65
Loss of self-esteem	0.56
Delay in falling asleep	0.43
Less able to concentrate		0.81
More forgetful		0.70
Restless/pacing		0.55
Irritable/bad tempered		0.53
Strange gestures/mannerism		0.30
Suspicious, untrusting			0.82
Loss of confidence with other people			0.65
Increased appetite			0.61
Sudden intense fear				0.68
Startled by sudden sounds				0.55
Fearful/panicky				0.54
Repeated actions				0.46
Waking too early					0.77
Broken sleep					0.47
Strange beliefs						0.72
Too happy						0.58
Strange use of language						0.58
Less able to use self-care skills						0.43

Discussion

Overall, the Turkish version of the PAS-ADD Checklist is an acceptable generic screening tool given the inherent difficulty of identifying mental health problems in people with ID and the lack of Turkish screening instrument.

The value of Cronbach’s alpha for the total scale was close to the previous validation studies. However, Cronbach’s alpha depends on the number of items in the scale as well as their correlations. The small number of items in the A and E subscales makes them inherently more unstable, as it is reflected in their relatively low alphas. On the other hand, as stated by the creator of the Checklist, the lower alpha scores indicating low inter-correlations cannot exclude the possibility that these items could function well as screening items [7]. Because the ultimate aim of screening process is to catch potential cases, rather than to identify specific nature of disorders.

Although inter-rater reliability was good for two scales (psychotic and organic), there was weak, statistically non-significant correlation for affective/neurotic disorder scales between two raters. This could reflect the fact that two raters will not have same amount of information on the person. One of them will be the primary care giver who has the most accurate information on the person. The second rater may have less information compared to the primary care giver despite our efforts to choose the second-best informant. This hypothesis could be supported by the fact test–retest agreements were 0.5 and above, which were filled out by the same rater, who were the primary care givers. There could also be some differences between the raters’ characteristics such as level of education having an impact on their scoring as highlighted by the creator of the checklist as well [7]. Although our number of 17 is not ideal to check for reliability, considering the previous validation studies it seems acceptable. In German version, they used 17 participants for inter-rater reliability and only 8 for test–retest reliability [5]. In French study, they even did not calculate test–retest reliability [6]. As a general suggestion, it could be helpful to provide training to informants who are asked to use such checklists to screen for mental health problems, as the information they provided could potentially have a considerable effect on results.

Sensitivity and specificity were computed to evaluate the validity of the checklist. The sensitivity was 78.7% in Turkish version which is a similar figure calculated from the findings of the developers of the checklist [7] and higher than the replication study of PAS-ADD [8] and also the French version [6]. The strength of a screening tool is directly related to its sensitivity as they are expected to be over-inclusive rather than under-inclusive. In this regard, Turkish version seems to function well for this purpose.

There was only one person with organic disorder (dementia) in this sample which could be due to most participants being young (mean age 29.9 years). Hence, it is difficult to determine how successful the PAS-ADD Checklist was at identifying these disorders.

Due to small group sizes and using only two categories to define the level of ID, the sample could not be evaluated according to the level of disability. Examining the checklist separately for all levels of ID would have been useful.

Conclusion

Turkish version of the PAS-ADD Checklist can be used as a general screening tool for psychiatric disorders in clinical practice. Although its diagnostic validity for specific disorders particularly psychosis cannot be claimed for, it will still function well as a screening instrument and contribute to early detection of mental health problems and therefore improve the quality of life of those with ID and their family/care givers.

Acknowledgements

We would like to thank all individuals and their families/care givers participated in this study. We are also grateful to staff teams in care homes and rehabilitation centers, particularly Bağcılar Başak Rehabilitation Centre, for their help and support with this project.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This study was funded by the Scientific & Technological Research Council of Turkey (TÜBITAK) within BIDEB program with a project number of 115C015. The funding body did not have any involvement with study design, data collection, data interpretation, writing or submission for publication.

Appendix

The appendix for this online article has been removed. Turkish version of the PAS-ADD is copyright protected. In order to obtain permission for research or clinical purposes, please contact the Pavilion Publishing and Media Ltd.