Responding to Personality Tests in a Selection Context: The Role of the Ability to Identify Criteria and the Ideal-Employee Factor

Abstract

Personality assessments are often distorted during personnel selection, resulting in a common “ideal-employee factor” (IEF) underlying ratings of theoretically unrelated constructs. However, this seems not to affect the personality measures' criterion-related validity. The current study attempts to explain this set of findings by combining the literature on response distortion with the ones on cognitive schemata and on candidates' ability to identify criteria (ATIC). During a simulated selection process, 149 participants filled out Big Five personality measures and participated in several high- and low-fidelity work simulations to estimate their managerial performance. Structural equation modeling showed that the IEF presents an indicator of response distortion and that ATIC accounted for variance between the IEF and performance during the work simulations, even after controlling for self-monitoring and general mental ability.

ACKNOWLEDGMENTS

We thank Gerald Richter and Arne Domsch for their help with the data collection and Neil Christiansen for his generous comments on an earlier version of this manuscript. The research reported in this article was partially supported by grants from the German Research Foundation (Deutsche Forschungsgemeinschaft, Grant Kl 823/6-1) and the Swiss National Science Foundation (Schweizerischer Nationalfonds, Grant 100014-124449).

Notes

¹An example: A participant assumes that in the first group discussion participants were evaluated on “teamwork,” “creativity,” and “goal setting.” In the second group discussion, this participant assumed that she was evaluated on “teamwork” and “influencing others.” Later, she assumed for the first discussion that the hypothesis “teamwork” fit completely (rating = 4) with the dimension “Cooperation,” that “creativity” somewhat (rating = 1) reflected “Handling of Information,” and that “goal setting” rather well (rating = 3) reflected “Planning.” For the second group discussion, she rated the strength of fit between “teamwork” and “Cooperation” again with 4 and the strength of fit between “influencing others” and “Leadership” with a 4 as well. In summary, this participant thus received an ATIC score of 4 for Cooperation in both group discussions. In the first group discussion, she also received an ATIC score of 3 for Planning but a score of 0 for Leadership. In the second group discussion, she received a score of 4 for Leadership and score of 0 for Planning. In average, this would imply an overall ATIC score of 2.5 (ATIC = (4+4+3+0+0+4)/6).

²Besides being a statistical necessity in order to render an identifiable solution (Byrne, 1994, 1998; see also Cellar et al., 1996, p. 699), the absence of covariances between the original personality dimensions and the additional ideal-employee factor is warranted for both conceptual and empirical reasons. Conceptually, the ideal-employee factor results from a cognitive schema associated with the job application situation (Fiske & Taylor, 1991; Holden et al., 1992; Mischel & Shoda, 1995, 1998), rather than representing another aspect of personality (Cellar et al., 1996; Van Iddekinge et al., 2001). Empirically, this assumption has been supported by different studies using different methodological approaches that found that faking effects were independent of person effects (Pauls & Crost, 2005) and that the increased common variance seems unrelated to the personality test content variance (Zickar & Robie, 1999).

³As would have been expected, a model-comparison based on the 60 individual items rather than the 15 parcels yielded a miserable fit for both Models 1 and 2 on the usual goodness to fit measures (IFI, TLI, and CFI = .54 to .57 for Model 1 and .61 to .65 for Model 2). Yet this comparison, too, confirmed that Model 2, χ²(1640) = 2787.01, p < .01, χ²/_df = 1.70, assuming all 60 individual items to load on the ideal-employee factor, yielded a significantly better fit than the baseline measurement Model 1, χ²(1700) = 3085.68, p < .01, χ²/_df = 1.82, Δχ²(60) = 298.67, p < .01. This, again, confirms the necessity of including an ideal-employee factor to the model and shows that results mentioned above are not caused by any distribution of items onto parcels. The average loading of the individual items onto the IEF was γ = .46 for the items belonging to Emotional Stability, γ = .39 for the items belonging to Conscientiousness, γ = .18 for the Extraversion items, γ = .14 for the Agreeableness items, and γ = .08 for the Openness items, respectively.

⁴We gained these estimates for each personality item's loading onto the ideal-employee factor via the item-based measurement Model 2 mentioned in footnote 3.

⁵We thank an anonymous reviewer at Human Performance for raising the subsequent points.