Reliability, validity, and factorial structure of the Turkish version of the Structured Inventory of Malingered Symptomatology (Turkish SIMS)

ABSTRACT

OBJECTIVE

Smith and Burger developed the Structured Inventory of Malingered Symptomatology (SIMS) in 1997 as a self-report measure for malingering of psychiatric symptoms. The SIMS consists of 75 dichotomous (True–False) items that form into five subscales Psychosis (P), Neurologic Impairment (NI), Affective Disorder (AF), Amnestic Disorders (AM), Low Intelligence (LI); each subscale containing 15 items. In this study, we aimed to examine the reliability, validity, and factor structure of the SIMS in a Turkish forensic psychiatry sample.

METHODS

A sample of 103 forensic patients (9 female, 94 male), aged 18–75, undergoing an inpatient forensic evaluations for competency to stand trial (CST) were recruited from a large forensic hospital in Turkey. The study protocol was approved by the local Ethics Committee. Sociodemographic information of the participants was collected and the SIMS, Miller Forensic Assessment of Symptoms Test (M-FAST), the Scales of Psychological Well-being, 36-Item Short Form Survey (SF-36), Beck Depression Inventory (BDI) and Beck Anxiety Inventory (BAI) were administered. All statistical analyses were performed by using SPSS version 23.0 for Windows.

RESULTS

The Cronbach’s alpha coefficients for the Turkish SIMS were ranging from 0.32 to 0.88. The lowest alpha coefficient was observed for the Low Intelligence (0.32). For the whole scale, Cronbach’s alpha coefficient was found to be 0.93. The test–retest (at after 1 week) correlation coefficients for Psychosis (P), Neurologic Impairment (NI), Affective Disorder (AF), Amnestic Disorders (AM), Low Intelligence (LI), and whole scale were found to be 0.97, 0.98, 0.96, 0.67, 0.83, and 0.95, respectively. A positive and statistically significant correlation was found between the Turkish SIMS and BDI (r = 0.620, p < .01), BAI (r = 0. 597, p < .01), M-FAST subscale Reported versus Observed Symptoms (r = 0.675, p < .01), M-FAST subscale Extreme Symptomatology (r = 0.713, p < .01), M-FAST subscale Rare Combinations (r = 0.751, p < .01), M-FAST subscale Unusual Hallucinations (r = 0.710, p < .01), M-FAST subscale Unusual Symptom Course (r = 0.588, p < .01), M-FAST subscale Negative Image (r = 0.528, p < .01), M-FAST subscale Suggestibility (r = 0.440, p < .01), and MFAST Total (r = 0.816, p < .01) scores. Principal axis factor analyses with Promax rotation were performed and four-factor solution that accounted for 39.87% of the variance observed.

CONCLUSIONS

Our preliminary findings suggested that Turkish SIMS was a valid and reliable tool with a robust factorial structure for further use in detecting malingering of forensic psychiatric cases in Turkey.

KEYWORDS:

• Structured Inventory of Malingered Symptomatology
• reliability
• validity
• factor structure

Introduction

Deception is an ordinary and ubiquitous fact of everyday life, as both verbal and non-verbal social behaviour of human nature [1]. Earliest representations of feigned madness appear within texts as old as the Bible [2] and the first reference to malingering from medicine can be found in “On Feigned Disease and the detection of them” by Galen in 2nd century AD [3].

The American Psychiatric Association (1980) officially defined malingering in the Diagnostic and Statistical Manual of Mental Disorders – Third Edition (DSM-III). Prior to that time, the absence of an official definition was due to an earlier debate whether malingering constitutes a distinct psychiatric disorder. The DSM-III stated malingering as not a mental disorder per se but a condition that deserves a focus of attention or treatment. In other words, although malingering is not in and of itself a psychiatric disorder, it does have clear psychological implications. Under the current nosology of the Diagnostic and Statistical Manual of Mental Disorders – Fifth Edition (DSM-5), malingering is defined as the intentional production of false or grossly exaggerated physical or psychological symptoms, motivated by external incentives. DSM-5 instructs clinicians to strongly suspect the presence of malingering when any combination of the following indicators are present: (a) the existence of a medicolegal context, (b) explicit discrepancy between the reported impairment and objective observation, (c) lack of cooperation during the diagnostic evaluation and nonadherence to treatment, and (d) the presence of antisocial personality disorder [4].

When malingering was considered in differential diagnosis, clinicians are expected to ascertain the motivations and level of conscious awareness that accompany symptoms reported by individuals rather than identifying certain diagnostic criteria during presentation [5]. Among the differential conditions to be considered in the malingering cases are factitious disorders, in which the motivation is internal such as assuming a sick role and somatoform disorders, in which the condition and the presentation of symptoms are not viewed as intentional. Different types of external motivations might have an impact on the presentation of malingering, including avoiding military duty or work, efforts to obtain financial compensation, efforts to evade criminal prosecution, and obtaining drugs [5].

The frequency of malingering differs across settings; however, the base rate of malingering psychiatric symptoms in sanity and competence cases has been reported to range between 15.7% and 45% [5]. The evaluation of malingering of psychiatric disorders is crucial in forensic settings. Clinicians are needed to indicate whether a defendant has sufficient understanding and cognition to comprehend the purpose of trial proceedings or to defend him/herself in front of the court [6]. Failure to detect malingering may lead to vital adverse consequences for the administration of justice [7]. The higher frequency of malingering in forensic cases is indisputably related to the fact of the diminished or excluded responsibility of the psychiatric patients in criminal jurisdictions and the temptation of the offenders to evade criminal prosecution.

Detecting simulated or exaggerated symptoms is a major challenge in forensic psychiatry. Rogers suggested that any psychological examination conducted in a compensation-seeking context should include an assessment of the likelihood of malingering [8]. There have been three types of instruments used in the detection of malingering: (a) structured interviews, (b) general psychological or cognitive instruments, and (c) tests specifically designed for the detection of malingering. Although structured interviews contributed to the systematic assessment of psychiatric disorders, this method is time-consuming and requires a trained evaluator. Both personality measures and cognitive/ intellectual assessment instruments may be effective in the malingering detection; however, length of administration, need for specialized administration training and demand for a higher level of literacy would limit their extensive use in clinical settings. Some measures were specifically developed to determine the presence of malingered psychopathology (e.g. the Miller Forensic Assessment of Symptoms Test – M-FAST) [9]. Although the M-FAST has been shown utility in clinical settings function, there remains the need for an instrument that screens more than general psychopathology since malingerers often feign symptoms of more than one condition.

The majority of malingering research incorporates three common design approaches, each having different strengths and limitations: (a) differential prevalence design studies, (b) simulation studies, and (c) known group comparisons.

Researchers take two groups of participants from two populations with clear differences among base rates of malingering in differential prevalence design studies. This eliminates the difficulty of accurately detecting malingering for each participant while posing a substantial threat to internal validity. Thus these studies require stringent selection of participants to narrowly defined criteria, such as those provided by Slick et al. [10].

Simulation studies instruct at least one group to feign psychological symptoms, therefore providing a controlled rate of malingered symptoms while losing the internal motivation of the malingerer which will face consequences if malingering is detected. Thus these studies give a superior internal validity with questionable external validity [11].

On the other hand, known group comparisons overcome this problem by using additional measures, such as previously existing symptom validity tests or an examination by a group of forensic experts, and creating a known group of malingerers for further evaluation. This additional measure constructs high generalizability thus resulting in high scores of external validity but also forces a ceiling of validity defined by the additional measure, which gives relatively low values of internal validity.

Smith and Burger developed the Structured Inventory of Malingered Symptomatology (SIMS) in 1997 as a self-report measure designed to assess symptoms of both feigned psychopathology and cognitive function [9]. In this present study, we aimed to examine the reliability, validity, and factor structure of the SIMS with a known group study design in Turkish forensic psychiatry sample.

Material and methods

Study participants

A sample of 103 forensic patients (9 female, 94 male), aged 18–75, undergoing inpatient forensic evaluations for competency to stand trial (CST) were recruited from the Ministry of Justice Forensic Medicine Institute in Turkey. All participants’ initial likelihood of malingering was determined by the members of the 4th Council at the Ministry of Justice Forensic Medicine Institute (3 forensic psychiatry professors and 1 neurology professor) with elaborate clinical examination of their reported symptoms.

The study protocol was approved by the local Ethics Committee (Approval Number: 2017/570, Approval Date: 18.04.2017) and all of the patients gave written informed consents before participation. Following, the study has been thoroughly explained them, sociodemographic information of the participants was collected with a form. Structured Inventory of Malingered Symptomatology (SIMS), Miller Forensic Assessment of Symptoms Test (M-FAST), Beck Depression Inventory (BDI), and Beck Anxiety Inventory (BAI) were administered to the participants.

Psychometric instruments

Sociodemographic data form

This form includes demographic variables, including age, marital status, occupational status, and education.

Structured Inventory of Malingered Symptomatology (SIMS)

The SIMS is a 75-item, multiaxial, self-administered screening measure used for detection of malingering across a variety of clinical and forensic settings. The SIMS includes dichotomous (True–False) items that form into five subscales namely Psychosis (P), Neurologic Impairment (NI), Amnestic Disorders (AM), Low Intelligence (LI), Affective Disorder (AF); each subscale containing 15 items. The Psychosis (P) subscale assesses the degree to which a respondent endorses bizarre or unusual psychotic symptoms not typically seen in actual psychotic patients. The Neurologic Impairment (NI) subscale assesses the degree to which a respondent endorses illogical or highly atypical neurological symptoms. The Amnestic Disorders (AM) subscale items indicate the degree to which a respondent endorses symptoms of memory impairment that are inconsistent with patterns of impairment seen in brain injuries or dysfunctions. The Low Intelligence (LI) assesses the degree to which a respondent fabricates/ exaggerates intellectual deficits by failing simple general fund of knowledge items. The Affective Disorder (AF) subscale assesses the degree to which a respondent reports atypical symptoms of depression and anxiety.

Although several cut-off scores have been proposed for different purposes in literature, a total score exceeding 16 is considered to be indicative of malingering [12–14]. If SIMS is administered as part of a multi-method symptom validity testing (SVT) battery, for more diagnostic certainty, cut scores of 19 or even 25 are also suggested by different researchers [15,16].

The SIMS has been validated previously many times among various groups [9,12,14,17–19] and has been translated to Dutch [20], German [21], Spanish [22] and Italian [23]. The Turkish SIMS has been translated into Turkish by Samet Kose and back-translated into English by Filiz Kulacaoglu who was blinded to the original items. The content equivalence of SIMS items was examined, and necessary changes were made as some items being irrelevant to Turkish culture (e.g. Item 14 was “The United States has 55 states” and it was adapted as “Türkiye’nin 81 ili vardır.” [Turkey has 81 provinces.])

Miller Forensic Assessment of Symptoms Test (M-FAST)

Miller developed MFAST as an abbreviated version of the Structured Interview for of Reported Symptoms (SIRS) for the screening of malingering, in 2001 [24]. MFAST contains 25 items in seven subscales, reported versus observed symptoms, extreme symptoms, rare combinations, unusual hallucinations, unusual symptom course, negative image and suggestibility. None of these subscales recommended for detection of malingering [25]. Miller suggests a MFAST Total cut score of 6 or more to predict malingering which gives a specify score of 0.83 and sensitivity score of 0.93 [26]. MFAST has been adapted into Turkish, and the reliability and validity have been studied in 2015 [27].

Statistical analysis

Mean, median, standard deviation, and percentages were used to display descriptive statistics. The internal consistencies of the total SIMS and subscales were calculated by Cronbach alpha coefficient. For examining test–retest reliability, the Turkish SIMS was reapplied to the thirty-six patients selected from whole sample one week after the initial administration. Pearson’s correlation test was performed to analyse association between SIMS subscales and SIMS Total, BAI, and BDI. Factorial structure of the Turkish SIMS was analysed with Promax rotation. All statistical analyses were performed by using SPSS version 23.0 for Windows, and a p value less than .05 was considered statistically significant.

Results

Sociodemographic characteristics of the patient group and the healthy controls were presented in Table 1. The average age of 103 participants in the study was 34.89 with a standard deviation of 12.29, and it ranged from 18 to 75. The sample consisted of 9 female (8.7%) and 94 male (91.3%) patients. The majority of the patients participated in the study were single (48.5%) and 27 (26.2%) were married, and 26 patients were divorced or separated. Educational levels of subjects were broken down as follows: Elementary School 17 (16.5%), Middle School 30 (29.1%), High school 33 (32%) University 21 (20.4%). Before administered for evaluation just below one third of patients were unemployed 30 (29.1%), 73 were employed (40 patients work irregularly, 33 patients work regularly.) 39.6% of the participants indicated that they have at least one attempt of self-harming behaviour.

Table 1. Sociodemographic characteristics of the participants (n = 103).

Age		34.89 ± 12.29
		N	%
Educational status	Elementary school	19	18.4
	Middle school	30	29.1
	High school	33	32
	University	21	20.4
Employment	Regular Work order	33	32
	Irregular Work order	40	38.8
	Unemployed	30	29.1
Marital status	Single	50	48.5
	Married	27	26.2
	Divorced	26	25.2
Self-harm	Present	38	39.6
	Absent	65	63.1

Table 2 provides the summary statics for patients’ previous psychiatric treatment history. Closer inspection of the table shows that the majority 90 (87.4%) of those who responded claim having some kind of psychiatric treatment.

Table 2. Psychiatric history of the participants.

	n	%
Psychiatric treatment history
None	13	12.6
Affective disorder	36	35
Psychotic disorder	32	31.1
Bipolar disease	10	9.7
Substance use	12	11.7
Hospitalization
Not treated	12	11.7
Never hospitalized	33	32
Hospitalized once	21	20.4
Multiple hospitalizations	37	35.9
Opinion on having a psychiatric disorder
Claims having a psychiatric disorder	60	58.3
Claims healthy	37	35.9
Not sure	6	5.8
Family psychiatric history
Present	29	28.2
Not present	74	71.8
Substance usage history
Present	40	38.8
Not present	63	61.2

Reliability analysis

The mean scores and standard deviations for the Turkish SIMS scale and its subscales are presented in Table 3. The Cronbach’s alpha coefficients for the Turkish SIMS were ranging from 0.32 to 0.88. The lowest Cronbach’s alpha coefficient was observed for the Low Intelligence (0.32). For the whole scale, Cronbach’s alpha coefficient was found to be 0.93. The test–retest correlation coefficients for all dimensions were relatively high and statistically significant. The test–retest (at after 1 week) correlation coefficients for Psychosis (P), Neurologic Impairment (NI), Amnestic Disorders (AM), Low Intelligence (LI), Affective Disorder (AF), and whole scale were found to be 0.97, 0.98, 0.67, 0.83, 0.96, and 0.95, respectively.

Table 3. SIMS subscale means, standard deviations, Cronbach’s alpha, test–retest Cronbach’s alphas.

SIMS subscales	M	SD	α	Test–retest α
Psychosis (P)	4.94	4.19	0.87	0.97
Neurologic impairment (NI)	5.85	4.50	0.88	0.98
Amnestic disorders (AM)	6.12	4.91	0.71	0.67
Low intelligence (LI)	6.00	2.07	0.32	0.83
Affective disorder (AF)	6.77	3.14	0.69	0.96
TOTAL SIMS	29.70	16.29	0.93	0.95

Note: SIMS: Structured Inventory of Malingered Symptomatology; M: mean; SD: standard deviation.

Validity analysis

Convergent validity was examined by correlations between the Turkish SIMS scores and M-FAST scores. A positive and statistically significant correlation was found between the Turkish SIMS and M-FAST subscale Reported versus Observed Symptoms (r = 0.675, p < .01), M-FAST subscale Extreme Symptomatology (r = 0.713, p < .01), M-FAST subscale Rare Combinations (r = 0.751, p < .01), M-FAST subscale Unusual Hallucinations (r = 0.710, p < .01), M-FAST subscale Unusual Symptom Course (r = 0.588, p < .01), M-FAST subscale Negative Image (r = 0.528, p < .01), M-FAST subscale Suggestibility (r = 0.440, p < .01), and MFAST Total (r = 0.816, p < .01) scores (Table 4).

Table 4. Correlations between the SIMS Total and Subscales Scores and total scores of M-FAST, BAI, and BDI

		M-FAST total	BAI total	BDI total
P	r p	0.792 0.000**	0.543 0.000**	0.539 0.000**
NI	r p	0.777 0.000**	0.547 0.000**	0.591 0.000**
AM	r p	0.687 0.000**	0.523 0.000**	0.524 0.000**
LI	r p	0.448 0.000**	0.185 0.061	0.213 0.031*
AF	r p	0.691 0.000**	0.646 0.000**	0.687 0.000**
TOTAL SIMS	r p	0.816 0.000**	0.597 0.000**	0.620 0.000**

Note: SIMS: Structured Inventory of Malingered Symptomatology; M-FAST: Miller Forensic Assessment of Symptoms Test (M-FAST); BAI: Beck Anxiety Inventory; BDI: Beck Depression Inventory

*Correlation is significant at the .05 level (two-tailed).

**Correlation is significant at the .01 level (two-tailed).

In addition, The BDI was positively correlated with P (r = 0.539, p < .01), NI (r = 0.591, p < .01), AM (r = 0.524, p < .01), AF (r = 0.687, p < .01), SIMS Total (r = 0.620, p < .01). The BAI was positively correlated with P (r = 0.543, p < .01), NI (r = 0.547, p < .01), AM (r = 0.523, p < .01), AF (r = 0.646, p < .01), SIMS Total (r = 0.597, p < .01) (Table 4).

Factor analysis

Before factor analysis scale’s inter-item correlation matrix was assessed with the Kaiser–Meyer–Olkin Measure of Sampling Adequacy. The measure of sampling adequacy was 0.620 and Bartlett’s Test of Sphericity χ² was 5399.7 (p < .001) indicating that the correlation matrix was appropriate for factor analysis.

Principal components analysis with Promax rotation was performed to optimize factor loadings and to facilitate the interpretation of different factors. Both a four- and a five-factor solution were performed following inspection of the plot of Eigenvalues. Only four-factor solution provided clear loadings of the scales and four-factor solution accounted for 39.87% of the variance observed. Table 5 and shows the results obtained from factor analysis.

Table 5. Factor analysis: total variance explained and component matrix.

Explained variance	Total	Factor 1	Factor 2	Factor 3	Factor 4
	39.8%	27.3%	4.9%	4.2%	3.3%
SIMS 1		.608	.564
SIMS 2				−.548
SIMS 3		.437	.595
SIMS 4				−.510
SIMS 5		.577
SIMS 6
SIMS 7
SIMS 8			.593
SIMS 9		.436			.553
SIMS 10					.463
SIMS 11					.650
SIMS 12		.615	.508
SIMS 13		.659	.584
SIMS 14
SIMS 15		.637	.444
SIMS 16				−.601
SIMS 17		.562
SIMS 18		.655	.456
SIMS 19				.464
SIMS 20		.729
SIMS 21			.569
SIMS 22		.719
SIMS 23
SIMS 24
SIMS 25		.640	.592
SIMS 26		.426
SIMS 27		.782	.467
SIMS 28			.561
SIMS 29		.411	.426
SIMS 30		.635			.407
SIMS 31		.701	.508
SIMS 32		.563
SIMS 33		.676	.557
SIMS 34		.444	.537
SIMS 35		.701	.505
SIMS 36		.630	.536
SIMS 37		.790	.463
SIMS 38		.623	.592
SIMS 39		.708	.549
SIMS 40			.562
SIMS 41		.494	.423
SIMS 42		.504	.462
SIMS 43		.480			.415
SIMS 44		.649	.436
SIMS 45		.604	.478
SIMS 46		−.429	−.680
SIMS 47		.703	.524
SIMS 48		.613	.548
SIMS 49		.594	.573
SIMS 50		.760	.401
SIMS 51		.576
SIMS 52		.485
SIMS 53		.466	.409
SIMS 54
SIMS 55				.594
SIMS 56
SIMS 57		.559	.534
SIMS 58
SIMS 59		.683	.448
SIMS 60		.731
SIMS 61
SIMS 62			.738
SIMS 63			.403		.445
SIMS 64		.632			.457
SIMS 65		.533	.448
SIMS 66		.490
SIMS 67			.427
SIMS 68
SIMS 69		.430	.680
SIMS 70		.739
SIMS 71		.565
SIMS 72				.497
SIMS 73			.447
SIMS 74		.693	.502
SIMS 75					.608

Discussion

In this study, we aimed to examine the reliability, validity, and factor structure of the Turkish SIMS in a Turkish forensic psychiatry sample. The main results of the study confirmed that the Turkish SIMS was observed to have stable and reliable psychometric properties. The internal consistency coefficients of the Turkish SIMS scale and subscales showed that the scale was reliable. The Cronbach’s alpha coefficients for the Turkish SIMS were ranging from 0.32 to 0.88. The lowest Cronbach’s alpha coefficient was observed for the Low Intelligence (0.32) and the highest Cronbach’s alpha coefficient was observed for the Neurologic Impairment (0.88).

Additionally, the positive correlation coefficients between the first and the second administration of the Turkish SIMS revealed high test–retest reliability. A limitation of the literature on the SIMS is the lack of test–retest reliability data. Merckelbach and Smith [20] obtained a test–retest correlation coefficient of 0.72, Cima and colleagues [13] found a test–retest correlation of 0.97. In our study correlation coefficients for Psychosis (P), Neurologic Impairment (NI), Affective Disorder (AF), Amnestic Disorders (AM), Low Intelligence (LI), and whole scale were found to be 0.97, 0.98, 0.96, 0.67, 0.83, and 0.95, respectively.

On examination of the relationship between the SIMS scale and other measures of malingering, the SIMS subscales demonstrated moderate to high correlations with and M-FAST subscales and MFAST Total scores.

Primary advantage of the SIMS is that its subscales are intended to assess different facets of feigned psychopathology and cognitive impairment [28]. SIMS, having five individual homogenous subscales, provides forensic clinicians with useful insights into what potential domains of feigning require further evaluation unlike other currently available screening tools [29].

Verbal intelligence affects SIMS scores slightly [30]. Van Impelen et al. [31] suggested that individuals with an intellectual disability may produce heightened SIMS scores because of their diminished capacity to comprehend SIMS items or it might also be that low intelligence predisposes individuals to engage in more transparent forms of feigning as demonstrated by Solomon et al. [32].

The positive correlations between Turkish SIMS and the Beck Depression Inventory (BDI) and Beck Anxiety Inventory (BAI) further gave support to the validity of the scale. These correlations between SIMS, depression and anxiety were also reported previously by Edens et al. [14]. Although these results would seem to suggest that SIMS could be sensitive to real psychopathology when a cutoff score of 16 was applied false positives rates stays relatively low [20].

Our Principal components factor analysis with Promax rotation provided a four-factor solution with clear loadings of the scales, which accounted for 39.87% of the variance observed. This was consistent with the original study [33].

The generalizability of our findings should be considered in light of certain limitations. Our study participants were comprised of patients from the Ministry of Justice Forensic Medicine Institute for whom a specific legal limitation on observation period was present, which resulted in conducting our test–retest assessments in only one week. An additional limitation was the fact that our sample consisted mostly of male participants, which ended up in uneven numbers in terms of gender distribution. And finally, we aimed to include at least 200 participants. However, the literacy level of most patients who were admitted at the Ministry of Justice Forensic Medicine Institute was not sufficient enough to reliably answer the questionnaires used in the study. Almost half of the initial sample was eliminated due to this essential requirement. Therefore; our study sample might not truly represent the general malingering population seen in clinical/ forensic practice in Turkey.

In conclusion, the SIMS is a brief 75-item, self-report screening measure of multiple domains of malingered symptomatology, which includes psychiatric and neurocognitive disorders. In addition, higher SIMS subscale scores might signal a need for further scrutiny for emotional distress, given the possibility of comorbidity between malingering and psychopathology. The Turkish version of the SIMS had sound psychometric properties in our sample of Turkish forensic patients with its satisfactory internal consistency, test–retest reliability, concurrent validity, and factorial structure.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Samet Kose http://orcid.org/0000-0003-0841-004X

Yasin Hasan Balcioglu http://orcid.org/0000-0002-1336-1724