The Incremental Validity of Self-Report and Performance-Based Methods for Assessing Hostility to Predict Cardiovascular Disease in Physicians

ABSTRACT

We evaluated the utility of an integrative, multimethod approach for assessing hostility-related constructs to predict premature cardiovascular disease (CVD) and premature coronary heart disease (CHD) using participants from the Johns Hopkins Precursors Study, which was designed to identify risk factors for heart disease. Participants were assessed at baseline while in medical school from 1946 to 1962 (M age = 24.6) and have been followed annually since then. Baseline assessment included individually administered Rorschach protocols (N = 416) scored for aggressive imagery (i.e., Aggressive Content, Aggressive Past) and self-reports of 3 possible anger responses to stress. Cox regression analyses predicting morbidity or mortality by age 55 revealed a significant interaction effect; high levels of Aggressive Content with high self-reported hostility predicted an increased rate of premature CVD and CHD, and incrementally predicted the rate of these events after controlling for the significant covariates of smoking (CVD and CHD) and cholesterol (CHD) that were also assessed at baseline. The hostility and anger measures, as well as other baseline covariates, were not predictors of CVD risk factors assessed at midlife during follow-up. Overall, this integrative model of hostility illustrates the potential value of multimethod assessment to areas of health psychology and preventive medicine.

Disclosure

The first and third authors receive royalties on the sale of the manual for the Rorschach Performance Assessment System and related products.

Acknowledgments

We thank Andy Geers and Jeanne Brockmyer for their helpful input on this project.

Notes

¹ Cohen's d is base rate insensitive because it indicates how far apart the dependent variable means are for two independent variable groups in pooled SD units. As such, the relative size of each group is immaterial; each group is considered equally sized or as if the half the sample was in one group and half in the other (i.e., a base rate of .50). In contrast, r is sensitive to the relative size of each independent variable group, with r being smaller as the relative difference in size increases because the cases in the smaller group have less influence on the correlation than the cases in the larger group. Stated differently, all other things being equal, a rare outcome (i.e., a low base rate) is harder to predict (i.e., lower r) than a relatively common outcome (see McGrath & Meyer, 2006).

² Rosenthal (1991) suggested that one could use the sample sizes (Ns = 67,187 and 5,899, for the healthy and existing CHD samples, respectively) in conjunction with the observed p values to very roughly estimate that the lower boundary rs would be about .01 and .04, respectively, which would be considered very small effects. However, hazard ratio effect sizes are a function of the number of events per unit of time, not the total sample size (e.g., Norman & Streiner, 2008). Chida and Steptoe (2009) did not provide this information, but the effective Ns would be a small fraction of the total Ns, as suggested by the base rates in Siontis et al. (2012).

³ Chida and Steptoe (2009) conducted secondary analyses using a subset of studies that they classified as having “fully controlled covariates” (p. 944), in which sets of covariates were forced into the equation regardless of their significance in the particular study. For the general population studies, the analyses were conducted on nine samples that at minimum controlled for age, smoking status, alcohol use, the traditional risk factors (blood pressure, diabetes, serum cholesterol), and SES. For the samples with existing CHD, the secondary analyses were conducted on three samples that at minimum controlled for age, smoking status, alcohol use, the traditional risk factors, SES, baseline disease status, and medical treatments received. In these smaller subsamples the effects remained positive but were no longer statistically significant (hazard ratios = 1.07 and 1.20 and p = .39 and .25, respectively).

⁴ Firm guidelines do not exist for the number of cases with missing data that can be accurately estimated with multiple imputation. Statistically, the key parameter is the fraction of missing information, which is less than the number of cases with missing data points when multiple variables are included in the analyses because correlated variables provide added information (Graham, 2009). Graham, Olchowski, and Gilreath (2007) evaluated the power and accuracy of multiple imputation as a function of the fraction of missing information (from 10%–90%) and the number of imputations performed (from 3–100). Across all analyses, coefficient estimates of the population parameter were accurate, with the standard error increasing slightly as expected with fewer imputations. With respect to power, across 10 imputations, as we used, the relative decrement in power to detect small differences was just 1% for variables with 10% missing information, 3% for variables with 30% missing information, and 4% for variables with 50% missing information. In addition, using ancillary variables to help estimate missing data, as we did, increases the accuracy of any single imputation (Graham, 2009). In our analyses, we set a priori limits to imputation, such that missing data were not imputed for variables with more than 40% of the cases missing. However, these limits were not reached for any of the variables used. The maximum was obtained for serum cholesterol (20.7% missing cases), after which there was no more than 9.4% of cases with missing observations.

⁵ Collinearity can be present even in the absence of strong correlations, so the primary tests for multicollinearity are variance proportions and their associated condition indexes. Ideally, condition indexes should be less than 5 with no more than one variable with a variance proportion greater than .40 to .50 on its eigenvalue; potential problems are indicated by condition indexes between 5 and 10 with two or more variance proportions greater than .40 to .50; and clear problems are indicated by condition indexes greater than or equal to 10 with two or more variance proportions greater than .40 to .50 (e.g., Friendly & Kwan, 2009). In our data, using results pooled across the original and imputed data sets, SBP and DBP had a correlation of just .56. However, they had variance proportions of .88 and .49 on the same small eigenvalue that had a condition index of 53.4. Paternal CHD and CVD had a correlation of .87, with variance proportions of .84 and .85 on a small eigenvalue that had a condition index of 10.8. Maternal CHD and CVD had a correlation of .65, with variance proportions of .72 and .74 on the same small eigenvalue, having a condition index of 6.4. Although the latter condition index is not clearly problematic, the others are and all are accompanied by variance proportions indicating a problematic ability to statistically discriminate the contributions made by these pairs of variables.

⁶ Some occupation values differ from the values reported in the Methods because they are based on multiple imputation across all cases, rather than just cases with nonmissing data.

⁷ These findings indicate there was a small propensity for those at baseline who were more reactive to physiological challenge (cold pressor test and Master two-step exercise test) than predicted to smoke less later in life, or conversely, that those who were less reactive to physiological challenge than predicted to smoke more later in life.