644
Views
1
CrossRef citations to date
0
Altmetric
Research Articles

A follow-up of the PFS-AV, an assessment instrument for hostility

ORCID Icon &

Abstract

This follow-up study presents psychometric data and norms of the Adapted Version of the Picture-Frustration Study (PFS-AV) from 422 male violent forensic psychiatric in- and outpatients, 101 male violent long-term prisoners, and 319 secondary vocational students (160 males and 159 females). The PFS-AV is a production instrument to measure hostility with 12 items. A diagnostician scored the responses on a seven-point scale, running from not at all hostile (0) to extremely hostile (7). Support was found for the PFS-AV construct, concurrent, and discriminant validity in all three samples. Therefore, the present results underline the generalizability and applicability of the PFS-AV in different populations. Furthermore, for clinical diagnosis, norms are presented for the three samples, just as examples for the scoring on the Likert scale. Also, Dutch, English, and German versions of the PFS-AV with scoring instructions are available.

Attributing hostile intentions to others in (alleged) conflict situations is a negative attitude that may play an important role in negative emotions such as anger and aggressive behavior. Over time, various definitions of the concept of hostility have been suggested. For example, Buss (Citation1961) defined hostility as an attitude that entails the disgust and negative evaluation of others, while Berkowitz (Citation1993) defined hostility as “a negative attitude toward one or more people that is manifested in a decidedly unfavorable judgment of the target” (p. 21). Smith (Citation1994) saw hostility as a trait that “means a devaluation of the value and motives of others, an expectation that others are likely sources of wrongdoing, a relational view of opposition to others, and a desire to harm or harm others” (p. 26). A more or less identical description was opted by Eckhardt et al. (Citation2004). They considered cynicism (others are selfishly motivated), mistrust (others will be hurtful and willfully provoke), and denigration (others are dishonest, ugly, mean, and unsocial) as the three central features of the hostility construct.

Hostility and Hostile Attributional Bias

Related to the concept of hostility is the concept of hostile attribution bias (HAB), which was first used by Nasby et al. (Citation1980) in their research on aggressive behavior in children. Hostile attribution bias can be defined as a tendency to interpret the behavior of other people as having hostile intentions, especially when social context cues are ambiguous or unpredictable and difficult to interpret (Milich & Dodge, Citation1984). HAB is mainly seen as part of the social information processing (SIP) model, in which social information leads to behavior through several steps during an interaction with others, namely encoding information, interpreting information, deciding a goal for interaction, generating responses, evaluating responses, and finally, enacting a response (Dodge, Citation1986). Many studies have been carried out on the alleged deviant SIP in children with behavioral problems. In most of these studies, participants were confronted with stories, pictures, or video clips of social interactions with peers that can easily lead to conflict. In some studies, real-life situations are staged. Research has shown a robust association between hostile attribution biases and aggressive behavior in youth (Orobio de Castro et al., Citation2002) and adults (Klein Tuente, Citation2020).

Functional analysis

According to the Integrative Cognitive Model (ICM) by Wilkowski and Robinson (Citation2010), a hostile interpretation of an (alleged) conflict situation can activate trait anger which in turn leads to state anger or reactive aggression. Attributing hostile intentions to others is seen as an automatic cognitive process characterized by biases or deviations during the various steps of Social Information Processing (Dodge, Citation2006; Ostrov & Godleski, Citation2010). Daffern et al. (Citation2007) incorporated the several individual factors of aggressive behavior into functional analysis, a method that correlates antecedents, individual characteristics, emotional responses, and consequences (Haynes & O’Brien, Citation2000). In support of the ICM’s assumed role of hostility in anger, Kraaimaat and Hornsveld (Citation2022) applied this functional analysis method. They found that interpreting the behavior of others as hostile contributed significantly to the state of anger in forensic psychiatric inpatients.

Measuring instruments

Due to the different definitions of hostility, quite a few measuring instruments have been developed for this concept. Miller et al. (Citation1996) found in a meta-analytic study on hostility and physical health 63 different hostility measures, structured interviews, and self-report questionnaires. Even the self-report questionnaires were found to be based on different definitions of hostility.

Eckhardt et al. (Citation2004) discussed the three most frequently used self-report questionnaires that are supposed to measure hostility, namely the Buss-Durkee Hostility Inventory (BDHI; Buss & Durkee, Citation1957), the Cook-Medley Hostility Inventory (Ho; Cook & Medley, Citation1954), and the Hostility and Direction of Hostility Questionnaire (HDHQ; Caine et al., Citation1967). According to Eckhardt et al. (Citation2004), these three questionnaires cannot adequately assess the hostility construct and possess unclear conceptual clarity and weak psychometric support. The BDHI has 75 items in eight subscales that can be traced back to two factors and do not measure hostility unambiguously. The Ho with 50 items appears to possess adequate psychometric properties, but Ho scores have been found to correlate highly with the personality trait neuroticism. Finally, the HDHQ with 51 items has hardly been investigated, and construct validity seems to be lacking.

It should be noted that the three self-report questionnaires mentioned above are choice-response questionnaires in which respondents have to compare the suggested item with their probable responses and rate the item for the intensity or frequency on, for instance, a Likert scale. However, several authors have recommended the use of “production tools” (free-answer tests) instead of multiple-choice “recognition tools” (choice-response tests) for people with a history of offending (Gavaghan et al., Citation1983; Stams et al., Citation2006; Van Vugt et al., Citation2011). For free-answer questionnaires, respondents are asked to write their responses to questions, after which an independent reviewer scores their responses. In choice-answer questionnaires, respondents do not have to think about the content of their responses. In free-answer tests, written responses might give more direct information about respondents’ hostile thought content.

Production instrument for measuring hostility

To address the lack of a production tool for the measurement of hostility, Hornsveld et al. (Citation2007) developed the Adapted Version of Rosenzweig’s (Citation1978) Picture-Frustration Study (PFS-AV). This instrument asks participants to write down their reactions to 12 cartoon-like pictures of conflicting situations. Then, respondents are instructed to examine the situation as shown in the pictures (e.g., to a shopkeeper: “This is the third time that this watch has stopped.”) and to write the first appropriate reply in the blank text box that enters their mind ().

Figure 1. One of the cartoon-like pictures of the PFS-AV.

Figure 1. One of the cartoon-like pictures of the PFS-AV.

When assessing the subjects’ responses to expressing hostility, the hostility definition of Eckhardt et al. (Citation2004) serves as a starting point, with cynicism, distrust, and contempt as central features. Items are scored by a trained diagnostician on a seven-point scale ranging from not at all hostile (1) to extremely hostile (7). Guidelines, instructions, and rating examples are available to prevent scoring drift and support scoring consistency among diagnosticians. In the Hornsveld et al. (Citation2007) study, it was found that the construct reliability, test-retest reliability, inter-rater reliability, and concurrent validity were moderate to good in forensic psychiatric in- and outpatients.

Purpose of the present study

The present study further examines the psychometric properties, generalizability, and clinical applicability of the Adapted Version of the Picture-Frustration Study (PFS-AV). As a starting point, we use the functional analysis of Hornsveld et al. (Citation2019), in which hostility is assumed to be related to certain personality traits (e.g., neuroticism or anger as traits and problem behaviors as social skills). Measures to test the construct, concurrent and discriminant validity of the PFS-AV were taken from a new sample of Dutch violent forensic psychiatric patients and samples of violent long-term prisoners and secondary vocational students. The first two samples were taken for their history of antisocial behavior and aggressiveness, while the students were used as a comparison group. Concerning concurrent validity testing, trait anger, state anger, and aggression were assumed to be positively related to hostility. On the other hand, openness, agreeableness, conscientiousness, and extraversion were hypothesized to be negatively related to hostility, while neuroticism, social anxiety, and social skills were not expected to be related. In addition to psychometric properties, norms were calculated on the three samples for diagnostic purposes.

Method

Participants

The study was performed on three samples: forensic psychiatric in- and outpatients (hereafter referred to as patients), long-term prisoners (hereafter referred to as prisoners), and secondary vocational students (hereafter referred to as students).

Patients

The 422 male patients had an average age of 31.06 years (SD = 11.93; range: 14–70 years). Inpatients (37%) were admitted to six forensic psychiatric hospitals. They had been convicted of serious crimes that are punishable with an imprisonment of more than four years (e.g., murder, manslaughter, aggravated assault, or rape). Their primary diagnosis was an antisocial personality disorder (75%) or a psychotic disorder in remission combined with an antisocial personality disorder (DSM-V: American Psychiatric Association, Citation2013). According to the psychiatrists of the multidisciplinary composed staff, the condition of the psychotic patients had stabilized to such an extent that their antisocial personality disorder was most prominent, and the patients were able to follow the treatment program for aggressive behavior.

Outpatients (63%) were treated at two forensic psychiatric outpatient clinics, to which they have been referred for compulsory treatment by the court. The primary diagnosis of the outpatients was an antisocial personality disorder. In addition, all patients were referred to a treatment program for violent offenders (Hornsveld & Kraaimaat, Citation2019) and completed the present study questionnaires as part of the assessment procedure. Nearly all patients had completed primary school, followed at least by a few years of secondary education.

Prisoners

The sample concerned 101 male prisoners with an average age of 32.35 years (SD = 9.50; range: 19–59 years) who had committed a violent offense punishable with a minimum of four years. The prisoners resided in three penitentiary institutions. Fifty-one prisoners were willing to be interviewed by the first author using the PCL-R. Scores of one or two on items 2, 4, 5, 6, 7, 12, 14, 15, and 17 of the PCL-R were used for the preliminary diagnoses of an antisocial personality disorder. Since 41 of the 51 prisoners met these criteria, the total sample’s percentage of antisocial personality disorder was estimated to be 80%. In addition, almost all prisoners completed primary school, followed at least by a few years of secondary education.

Students

The 319 students (160 males, 159 females) followed lower or upper secondary vocational education. In the Netherlands, secondary vocational education concerns professional training at the lowest level for professionals, such as hairdressers, carpenters, and nursing assistants. The students’ mean age was 18.53 years (SD = 2.11; range 16–27).

Measurement instruments

The Adapted Version of Rosenzweig’s (Citation1978) Picture-Frustration Study (PFS-AV; Hornsveld et al., Citation2007). The PFS-AV measures hostility and asks participants to write down their reactions to 12 cartoon-like pictures of conflicting social situations. Respondents are instructed to examine the situation as shown in the pictures. Answers are scored by an experienced and independent research assistant on a seven-point scale, ranging from not at all hostile (1) to extremely hostile (7). In the example of , that is “Irritating for you. I’ll give you another watch.” to “You have no right to complain. You have, of course, handled it carelessly. Own fault.” In a sample of 231 Dutch violent patients, the internal consistency (α = .76), test-retest reliability (r = .67), and interrater reliability (r = .77) of the PFS-AV were moderate to good. Furthermore, evidence was found for the test’s concurrent validity as scores correlated with trait and state anger, antisocial attitude, and aggression.

To make scoring less time-consuming, prevent rater drift, and increase the reliability of the answers, examples of answers and their scoring on the seven-point Likert scale are given at www.agressiehanteringstherapie.nl/nl/menu/meetinstruments.

The NEO Five-Factor Inventory (NEO-FFI; Costa & McCrae, Citation1992) has 60 items and measures the Big Five personality domains neuroticism, extraversion, openness, agreeableness, and conscientiousness. Participants score items on a five-point Likert scale, ranging from entirely disagree (1) to entirely agree (5). Internal consistency and test-retest reliability of the Dutch NEO-FFI scales were good in nonclinical adults’ samples (Hoekstra et al., Citation1996).

The Trait Anger subscale of Spielberger’s (Citation1980, Citation1988) State-Trait Anger Scale (STAS; Van der Ploeg et al., Citation1982) was used to measure the general disposition to anger. Participants rate each item (e.g., “I am quick-tempered”) how they generally feel using a four-point Likert scale: almost never (1), sometimes (2), often (3), and almost always (4). In a group of 150 Dutch male university students, Van der Ploeg et al. (Citation1982) found that the trait anger scale’s internal consistency (α coefficient) was .78, and test-retest reliability of .78 was documented in a subgroup of 70 students. The convergent validity of the trait anger scale also proved to be satisfactory.

The NAS part A of the Novaco Anger Scale–Provocation Inventory (NAS-PI; Novaco, Citation2003; Dutch version: Hornsveld et al., Citation2011) was used to measure state anger and concerned the self-reported responses relating to cognitive (e.g., “I get angry because I have a good reason to be angry.”), arousal (e.g., “Some people would say that I am a hothead.”), and behavioral (e.g., “When someone yells at me, I yell back at them.”) components of anger in 48 anger-eliciting situations. The items are scored on a three-point Likert-type scale: never true (1), sometimes true (2), and always true (3). In a sample of 194 Dutch violent forensic psychiatric outpatients (all males), for the NAS total score, the internal consistency (Cronbach’s α) was found to be .95, and the test-retest reliability in a subgroup of 90 outpatients was .80 (Hornsveld et al., Citation2011).

The Aggression Questionnaire (AQ; Buss & Perry, Citation1992; Dutch version: Meesters et al., Citation1996) originally had 29 items spread among four subscales, namely Physical Aggression (e.g., “I have threatened people I know”), Verbal Aggression (e.g., “My friends say I am somewhat argumentative”), Anger (e.g., “I have trouble controlling my temper”), and Hostility (e.g., “Other people always seem to get the breaks”). Respondents answered the items on this version of the AQ using a five-point scale ranging from extremely uncharacteristic of me (1) to extremely characteristic of me (5). In a group of 138 Dutch violent forensic psychiatric inpatients (all males), Hornsveld et al. (Citation2009) found for the total AQ an internal consistency of .83 and for the four subscales of the AQ an internal consistency of .72 .34, .57, and .81 successively. The Physical Aggression and Verbal Aggression scores were added together in a combined Aggression subscale. In the present sample of patients, the correlation between both subscales was .61, and the internal consistency of the combined aggression scale was .83.

The Inventory of Interpersonal Situations (IIS; Van Dam-Baggen & Kraaimaat, Citation1999; Kraaimaat, Citation2020) is a Dutch self-report questionnaire with two scales. One scale indicates social discomfort/anxiety; the other scale concerns the frequency of performance of social responses (i.e., social skills). Each scale consists of the same 35 items formulated as responses to specific social situations (e.g., “Telling a friend that he/she is doing something that bothers you.”). The reliability and validity of the IIS have been investigated in several adult psychiatric and non-psychiatric samples. Cronbach’s α has revealed a high internal consistency on both scales, while the conceptual structure was shown to be relatively invariant across socially anxious and non-socially anxious groups. The IIS scales discriminated between socially anxious and non-socially anxious samples and showed significant relationships with independent social anxiety measures. The IIS scales demonstrated high predictive validity for overt behavior in social situations (Van Dam-Baggen & Kraaimaat, Citation1999).

Procedure

The questionnaires and interviews of the patients were individually assessed at the start of an obligatory treatment program at the forensic psychiatric hospital. The prisoners and students answered the questionnaires collectively under the supervision of the first author and were paid €10 for contributing. Informed consent was obtained from all participants, and the study was approved by the Regional Ethical Committee, CMO of the region Rotterdam, the Netherlands.

Statistics

Data were analyzed using IBM SPSS (Statistical Package for the Social Sciences) version 25 and AMOS 26 (Byrne, Citation2016). Descriptive statistics were used to examine the questionnaires and scales. Due to some missing values, the SPSS list-wise procedure was applied in the various statistical analyses with the dataset.

Results

Descriptive data and differences between samples

Distributions of all variables were investigated and considered normal as skewness and kurtosis were within | 3 | (Tabachnick & Fiddell, Citation2007).

In , descriptive data are presented of the Adapted Version of the Picture-Frustration Study (PFS-AV) for the three samples in the present study. Data are presented separately for the entire group and the male and female students. Significant differences were found between the male and female students with their scores on the PFS-AV (F = 26.64, p < .001).

Table 1. PFS-AV data of the three Dutch samples.

Note that the possible lowest and maximum scores of the 12-item Adapted Version of the Picture-Frustration Study (PFS-AV) are 12 and 84. All samples had scores in the lower and medium to high range. The obtained Cronbach α’s refer to moderate to good construct validity of the PFS-AV. Also, there were significant differences between the three samples (F = 25.15; df = 2.849). Furthermore, comparing samples employing the Scheffé method showed that patients did not differ from prisoners. In contrast, patients and prisoners both differed from the secondary vocational students (respectively p < .001, ES d = .39 and p < .001, ES d = .40). Similar results were obtained when correcting for age differences between the samples. The significant differences between the patients and prisoners with the students support the PDF-AV’s discriminant validity.

The underlying structure, measurement invariance, and internal consistency

With the combined subjects (n = 842), an exploratory factor analysis (EFA) was performed on the 12 items of the PFS-AV scale to explore its underlying structure. First, principal component analysis and Varimax rotation with eigenvalue 1 resulted in a one-factor solution, explaining 33% of the total variance. Next, fit indices were calculated using CFA (AMOS 26) to assess the one-factor solution of the 12 PFS-AV items (Bentler, Citation1990). The sample size of n = 842 exceeded a Hoelter of n = 468 (p < .01), which indicates that our sample was satisfactory. Furthermore, a comparative fit index (CFI) of .912 and root mean square error of approximation (RMSEA) of .049 (confidence interval between .043 and .056) indicated that the one-factor model represented the data well. To test whether the underlying measurement structure was invariant, a multigroup confirmative factor analysis (MFCA) was performed across the data of the patients, prisoners, and secondary vocational students. The following configurative, metric, and scalar invariance indices were consecutively obtained: Δ CFI = .001, .014 and .013, and Δ RMSEA = .001, .000, and .001. The changes of about .01 between the steps indicate good measurement invariance of the PFS-AV (Chen, Citation2007). Finally, reliability analysis was performed to investigate PFS-AV’s internal consistency, which resulted in a Cronbach’s α of .81. The confirmatory factor analysis (CFA) satisfactory goodness of fit indices of a one-factor model and an α of .81 indicates that the PFS-AV is relatively homogeneous, and the separate items measure the same construct of hostility.

To further examine the content validity of the PFS-AV, floor or ceiling effects were investigated. These effects are present if more than 15% of the respondents achieved the lowest or highest possible score (McHorney & Tarlov, Citation1995). As can be inferred from , the PFS-AV scores' distributions in the three samples revealed no floor or ceiling effects, and no deviations of normality distribution were observed. They, therefore, supported the content validity of the instrument.

Table 3. Norms of the PFS-AV for the three Dutch samples.

Additional concurrent validity data of the PFS-AV

For the concurrent validity assessment, correlation coefficients are presented between the Adapted Version of the Picture-Frustration Study (PFS-AV) and questionnaires measuring related constructs for all three samples separately in .

Table 2. Correlation coefficients of the PFS-AV with the other questionnaires.

The present study’s results are generally similar to those found in an earlier study with a smaller group of patients (Hornsveld et al., Citation2007). In all three samples, the results further substantiate the concurrent validity of the PFS-AV. In all samples, moderate correlations were found for the PFS-AV with agreeableness, trait anger, state anger, and aggression measures. However, the Aggression Questionnaire hostility subscale had low to insignificant correlations with the PFS-AV. In addition, neuroticism was not significantly associated with the PFS-AV in all three samples. These latter findings may indicate that the PFS-AV measures hostile cognitions and not emotional distress. Taken together, the present results support the concurrent validity and generalizability of the PFS-AV.

Norms for the three samples

To facilitate using the Adapted Version of the Picture-Frustration Study (PFS-AV) for clinical diagnostics in Dutch-speaking countries, norms were calculated for patients, prisoners, and students. The PFS-AV norm scales have the following seven classes: very high, high, above the mean, mean, below the mean, low, and very low. The classes’ under and above limits are the 95th percentile, 80th percentile, Mean + standard error of measurement, Mean – standard error of measurement, 20th percentile, and fifth percentile. In , the norms are presented for patients, prisoners, and students (male and female students separately). Since there were differences between male and female students in their PFS-AV scores, norms are presented separately for the whole group and the male and female students.

Discussion

Moderate to good concurrent validity of the PFS-AV was found in patients, prisoners, and students samples. The confirmatory factor analysis (CFA) goodness of fit indices and Cronbach’s alpha supported a one-factor solution and the internal consistency of the present instrument. In addition, differences in PFS-AV scores between the three samples were indicative of the discriminant validity of this instrument. Furthermore, a corresponding relationship with the PFS-AV with related measures supported all three samples’ concurrent validity and generalizability. Lastly, the normal distribution of the PFS-AV was supported by insignificant deviations of skewness and kurtosis and the absence of floor or ceiling in the investigated samples’ scores distribution.

As mentioned before, some differences were observed in the correlation coefficients between patients and prisoners on the one hand with students on the other. A negative correlation was found between age and the PFS-AV in the samples of patients and prisoners. The absence of a negative correlation between age and the PFS-AV in the students’ sample might be due to their relatively low age range (16 − 27 years). Also, a significant but small correlation was found between the social anxiety subscale of the Inventory of Interpersonal Situations (IIS) and the PFS-AV in the sample of students. This finding contrasts with our expectations but is in line with the findings of DeWall et al. (Citation2010). They found in nonclinical samples positive correlations between social anxiety and feeling hostile toward others and hostile perceptions of others. This was not the case in patients and prisoners and needed further investigation. One possibility is that in patients and prisoners, anger is more prevalent than social anxiety. Note that anxiety and anger are counteracting negative emotions.

The hostility subscale of the Aggression Questionnaire (AQ) had small to insignificant correlations with the PFS-AV. One explanation for this is that the AQ hostility items refer predominantly to emotional distress and hardly to hostile cognitions as is the case with PFS-AV ratings (e.g., item 7: I wonder why sometimes I feel so bitter about things, or item 17: At times I feel I have gotten a raw deal out of life).

Limitations

Although this study had several strengths, a few limitations should be noted. For instance, the investigated clinical samples were composed of Dutch male patients and prisoners. It is therefore unknown whether the results also apply to female patients or prisoners or to patients and prisoners in other countries. In the Netherlands, about 75% of the forensic psychiatric inpatients have a personality disorder and about 25% have a psychotic disorder as their primary diagnosis (Van Emmerik, Citation2001). In addition, the sample of the students differed in age from that of the patients and prisoners samples. Therefore, future studies are needed to determine if findings generalize to patients with other diagnoses, female patients and prisoners, and different age groups in the general population. It is worth noticing that patients and prisoners may have influenced their responses by their desire to make a positive impression (Hornsveld et al., Citation2019). Finally, although the English and German versions of the PFS-AV are applicable for research purposes, a remark has to be made on using the PFS-AV for diagnostic purposes in English and German-speaking countries. That is to say that the measurement invariance findings may not be generalizable, and norms may not be applicable to the English and German versions of the PFS-AV.

Conclusion

The present findings corroborate and expand the results of an earlier study on violent patients (Hornsveld et al., Citation2007). Support was found for the PSF-AV for research and clinical purposes in Dutch-speaking countries. The PFS-AV can thus play a role in assessing and treating persons who are convicted of a violent crime. In patients or prisoners with a high score on the PFS-AV, it seems important to pay attention to how they interpret the behavior of others and to teach them that alternative interpretations of supposed hostile behavior are possible (cognitive restructuring).

Pictures and examples for scoring the answers on the seven-point Likert scale can be found on http://www.agressiehanteringstherapie.nl/nl/menu/meetinstrumenten.

Conflict of interest

The authors declared no potential conflicts of interest concerning this article’s research, authorship, and/or publication.

References

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596
  • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238
  • Berkowitz, L. (1993). Aggression: Its causes, consequences, and control. McGraw-Hill.
  • Buss, A. H. (1961). The psychology of aggression. Wiley.
  • Buss, A. H., & Durkee, A. (1957). An inventory for assessing different kinds of hostility. Journal of Consulting Psychology, 21(4), 343–349. https://doi.org/10.1037/h0046900
  • Buss, A. H., & Perry, M. (1992). The aggression questionnaire. Journal of Personality and Social Psychology, 63(3), 452–459. https://doi.org/10.1037/0022-3514.63.3.452
  • Byrne, B. M. (2016). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Routledge. https://doi.org/10.4324/9781315757421
  • Caine, T. M., Foulds, G. A., & Hope, K. (1967). Manual of the hostility and direction of hostility questionnaire (HDHQ). University of London Press.
  • Chen, F. F. (2007). Sensitivity of goodness of fit indices to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
  • Cook, W. W., & Medley, D. M. (1954). Proposed hostility and pharisaic-virtue scales for the MMPI. Journal of Applied Psychology, 38(6), 414–418. https://doi.org/10.1037/h0060667
  • Costa, P. T., Jr., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and the Five-Factor Inventory (NEO-FFI): Professional manual. Psychological Assessment Resources, Inc.
  • Daffern, M. D., Howells, K., & Ogloff, J. R. (2007). What’s the point? Towards a methodology for assessing the function of psychiatric inpatient aggression. Behaviour Research and Therapy, 45(1), 101–111. https://doi.org/10.1016/j.brat.2006.01.011
  • DeWall, C. N., Buckner, J. D., Lambert, N. M., Cohen, A. S., & Fincham, F. D. (2010). Bracing for the worst, but behaving the best: Social anxiety, hostility, and behavioral aggression. Journal of anxiety disorders, 24(2), 260–268. https://doi.org/10.1016/j.janxdis.2009.12.002
  • Dodge, K. A. (1986). A social information processing model of social competence in children. In M. Perlmutter (Ed.), Minnesota Symposium on Child Psychology (Vol. 18, pp. 77–125). Erlbaum.
  • Dodge, K. A. (2006). Translational science in action: Hostile attributional style and the development of aggressive behavior problems. Development and psychopathology, 18(3), 791–814. https://doi.org/10.1017/s0954579406060391
  • Eckhardt, C., Norlander, B., & Deffenbacher, J. (2004). The assessment of anger and hostility: a critical review. Aggression and Violent Behavior, 9(1), 17–43. https://doi.org/10.1016/S1359-1789(02)00116-7
  • Gavaghan, M. P., Arnold, K. D., & Gibbs, J. C. (1983). Moral judgment in delinquents and nondelinquents: Recognition versus production measures. The Journal of psychology, 114(2d Half), 267–274. https://doi.org/10.1080/00223980.1983.9915424
  • Haynes, S. N., & O’Brien, W. H. (2000). Principles and practice of behavioral assessment. Plenum.
  • Hoekstra, H. A., Ormel, J., & De Fruyt, F. (1996). Handleiding NEO Persoonlijkheidsvragenlijsten [Manual NEO personality questionnaires]. Swets Test Services.
  • Hornsveld, R. H. J., & Kraaimaat, F. W. (2019). Treatment programs for violent offenders and sexually violent offenders. In R. H. J. Hornsveld, F. W. Kraaimaat, L. A. C. L. Gijs, & E. J. Palmer, Assessment and obligatory treatment of violent and sexually violent offenders: Integrating research and practice (pp. 125–138). Springer Nature. https://doi.org/10.1007/978-3-030-27840-3
  • Hornsveld, R. H. J., Kraaimaat, F. W., Nunes, K. L., & Palmer, E. J. (2019). Dynamic risk factors of violent and sexually violent offenders. In R. H. J. Hornsveld, F. W. Kraaimaat, L. A. C. L. Gijs, & E. J. Palmer (Eds.), Assessment and obligatory treatment of violent and sexually violent offenders. Integrating research and practice (pp. 39–50). Springer Nature. https://doi.org/10.1007/978-3-030-27840-3
  • Hornsveld, R. H. J., Muris, P. E. H. M., & Kraaimaat, F. W. (2011). The Novaco Anger Scale-Provocation Inventory (1994 version) in Dutch patients. Psychological Assessment, 23(4), 937–944. https://doi.org/10.1037/a0024018
  • Hornsveld, R. H. J., Muris, P., Kraaimaat, F. W., & Meesters, C. (2009). Psychometric properties of the Aggression Questionnaire in Dutch violent patients and secondary vocational students. Assessment, 16(2), 181–192. https://doi.org/10.1177/1073191108325894
  • Hornsveld, R. H. J., Nijman, H. L. I., Hollin, C. R., & Kraaimaat, F. W. (2007). An adapted version of the Rosenzweig Picture-Frustration Study (PFS-AV) for the measurement of hostility in violent patients. Criminal Behaviour and mental health : CBMH, 17(1), 45–56. https://doi.org/10.1002/cbm.638
  • Klein Tuente, S. (2020). Understanding aggression and treating forensic psychiatric inpatients with Virtual Reality. University of Groningen, Netherlands. https://doi.org/10.33612/diss.147442033
  • Kraaimaat, F. W. (2020). Manual supplement 2020. Additional norms for the Inventory of Interpersonal Situations (ISS) [Inventarisatielijst Omgaan met Anderen (IOA)]. Free Publication. https://doi.org/10.13140/RG.2.2.16341.04322
  • Kraaimaat, F. W., & Hornsveld, R. H. J. (2022). Are anger, hostility, and aggression involved in social anxiety of forensic outpatients? Manuscript submitted for publication. Radboud University Nijmegen.
  • McHorney, C. A., & Tarlov, A. R. (1995). Individual-patient monitoring in clinical practice: are available health status surveys adequate? Quality of life research, 4(4), 293–307. https://doi.org/10.1007/BF01593882
  • Meesters, C., Muris, P., Bosma, H., Schouten, E., & Beuving, S. (1996). Psychometric evaluation of the Dutch version of the Aggression Questionnaire. Behaviour Research and Therapy, 34(10), 839–843. https://doi.org/10.1016/0005-7967(96)00065-4
  • Milich, R., & Dodge, K. A. (1984). Social information processing in child psychiatric populations. Journal of Abnormal Child Psychology, 12(3), 471–489. https://doi.org/10.1007/BF00910660
  • Miller, T. Q., Smith, T. W., Turner, C. W., Guijarro, M. L., & Hallet, A. J. (1996). A meta-analytic review of research on hostility and physical health. Psychological Bulletin, 119(2), 322–348. https://doi.org/10.1037/0033-2909.119.2.322
  • Nasby, W., Hayden, B., & DePaulo, B. M. (1980). Attributional bias among aggressive boys to interpret unambiguous social stimuli as displays of hostility. Journal of Abnormal Psychology, 89(3), 459–468. https://doi.org/10.1037/0021-843X.89.3.459
  • Novaco, R. W. (2003). The Novaco Anger Scale and Provocation Inventory: Manual. Western Psychological Services.
  • Orobio de Castro, B. O., Veerman, J. W., Koops, W., Bosch, J. D., & Monshouwer, H. J. (2002). Hostile attribution of intent and aggressive behavior: A meta-analysis. Child Development, 73(3), 916–934. https://doi.org/10.1111/1467-8624.00447
  • Ostrov, J. M., & Godleski, S. A. (2010). Toward an integrated gender-linked model of aggression subtypes in early and middle childhood. Psychological Review, 117(1), 233–242. https://doi.org/10.1037/a0018070
  • Rosenzweig, S. (1978). Aggressive behavior and the Rosenzweig picture-frustration study. Praeger.
  • Smith, T. W. (1994). Concepts and methods in the study of anger, hostility, and health. In A. W. Siegman, & T. W. Smith (Eds.), Anger, hostility, and the heart (pp. 23–42). Lawrence Erlbaum.
  • Spielberger, C. D. (1980). Preliminary manual for the State-Trait Anger Scale (STAS). University of South Florida.
  • Spielberger, C. D. (1988). State-Trait-Anger-Expression-Inventory. Consulting Psychologist Press.
  • Stams, G. J., Brugman, D., Dekovic, M., Rosmalen, L. v., Laan, P. v d., & Gibbs, J. C. (2006). The moral judgment of juvenile delinquents: A meta-analysis. Journal of Abnormal Child Psychology, 34(5), 692–708. https://doi.org/10.1007/s10802-006-9056-5
  • Tabachnick, B. G., & Fiddell, L. S. (2007). Using multivariate statistics (5th ed.). Pearson/Allyn & Bacon.
  • Van Dam-Baggen, R., & Kraaimaat, F. (1999). Assessing social anxiety: The inventory of Interpersonal Situations (IIS). European Journal of Psychological Assessment, 15(1), 25–38. https://doi.org/10.1027//1015-5759.15.1.25
  • Van der Ploeg, H. M., Defares, P. B., & Spielberger, C. D. (1982). Handleiding bij de Zelf- Analyse Vragenlijst, AQ [Manual for the Self-Analysis Questionnaire, AQ]. Swets & Zeitlinger.
  • Van Emmerik, J. L. (2001). De Terbeschikkingstelling in Maat en Getal [The TBS-measure (detainment under hospital order) in measures and numbers]. Ministry of Justice.
  • Van Vugt, E., Gibbs, J., Stams, G. J., Bijleveld, C., Hendriks, J., & Van der Laan, P. (2011). Moral development and recidivism: A meta-analysis. International Journal of offender therapy and comparative criminology, 55(8), 1234–1250. https://doi.org/10.1177/0306624X10396441
  • Wilkowski, B. M., & Robinson, M. D. (2010). The anatomy of anger: an integrative cognitive model of trait anger and reactive aggression. Journal of personality, 78(1), 9–38. https://doi.org/10.1111/j.1467-6494.2009.00607.x