Are predictions based on situational judgement tests precise enough for feedback in leadership development?

REFERENCES

• Anderson, N., Salgado, J. F., & Hülsheger, U. R. (2010). Applicant reactions in selection: Comprehensive meta-analysis into reaction generalization versus situational specificity. International Journal of Selection and Assessment, 18, 291–304. doi:10.1111/j.1468-7702389.2010.00512.x
   Google Scholar
• Arthur, W., Jr., Woehr, D. J., & Maldegen, R. M. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical reexamination of the assessment center construct-related validity paradox. Journal of Management, 26, 813–835. doi:10.1177/014920630002600410
   Google Scholar
• Bales, R. F. (1950). A set of categories for the analysis of small group interaction. American Sociological Review, 15, 257–263. doi:10.2307/2086790
   Web of Science ®Google Scholar
• Bergman, M. E., Drasgow, F., Donovan, M. A., Henning, J. B., & Juraska, S. E. (2006). Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment, 14, 223–235. doi:10.1111/j.1468-2389.2006.00345.x
   Web of Science ®Google Scholar
• Bowler, M. C., & Woehr, D. J. (2006). A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings. Journal of Applied Psychology, 91, 1114–1124. doi:10.1037/0021-9010.91.5.1114
   PubMed Web of Science ®Google Scholar
• Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth & Brooks/Cole Advanced Books & Software.
   Google Scholar
• Cabrera, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9, 103–113. doi:10.1111/1468-2389.00167
   Web of Science ®Google Scholar
• Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait multimethod matrix. Psychological Bulletin, 56, 81–105. doi:10.1037/h0046016
   PubMed Web of Science ®Google Scholar
• Catano, V. M., Brochu, A., & Lamerson, C. D. (2012). Assessing the reliability of situational judgment tests used in high-stakes situations. International Journal of Selection and Assessment, 20, 333–346. doi:10.1111/j.1468-2389.2012.00604.x
   Web of Science ®Google Scholar
• Chan, D., & Schmitt, N. (2002). Situational judgment and job performance. Human Performance, 15, 233–254. doi:10.1207/S15327043HUP1503_01
   Web of Science ®Google Scholar
• Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., & Harvey, V. S. (2001). Incremental validity of situational judgment tests. Journal of Applied Psychology, 86, 410–417. doi:10.1037/0021-9010.86.3.410
   PubMed Web of Science ®Google Scholar
• Fleishman, E., & Harris, E. F. (1962). Patterns of leadership behavior related to employee grievances and turnover. Personnel Psychology, 15, 43–56. doi:10.1111/j.1744-6570.1962.tb01845.x
   Web of Science ®Google Scholar
• Guenole, N., Chernyshenko, O., Stark, S., Cockerill, T., & Drasgow, F. (2012). We’re doing better than you might think: A large-scale demonstration of assessment centre convergent and discriminant validity. In N. Povah & G. Thornton (Eds.), Assessment centres and global talent management (pp. 15–32). London: Gower. ISBN 978-1-4094-0386-9.
   Google Scholar
• Guenole, N., Chernyshenko, O., Stark, S., Cockerill, T., & Drasgow, F. (2013). More than a mirage: A large scale assessment centre with more dimension variance than exercise variance. Journal of Occupational and Organizational Psychology, 86, 5–21. doi:10.1111/j.2044-8325.2012.02063.x
   Web of Science ®Google Scholar
• Guenole, N., Cockerill, T., Chamorro-Premuzic, T., & Smillie, L. D. (2011). Evidence for the validity of 360 dimensions in the presence of rater-source factors. Consulting Psychology Journal: Practice and Research, 63, 203–218. doi:10.1037/a0026537
   Google Scholar
• Lance, C. E., Lambert, T. A., Gewin, A. G., Lievens, F., & Conway, J. M. (2004). Revised estimates of dimension and exercise variance components in assessment center post-exercise dimension ratings. Journal of Applied Psychology, 89, 377–385. doi:10.1037/0021-9010.89.2.377
   PubMed Web of Science ®Google Scholar
• Lievens, F., Buyse, T., & Sackett, P. R. (2005). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90, 442–452. doi:10.1037/0021-9010.90.3.442
   PubMed Web of Science ®Google Scholar
• Lievens, F., & Conway, J. M. (2001). Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology, 86, 1202–1222. doi:10.1037/0021-9010.86.6.1202
   PubMed Web of Science ®Google Scholar
• Lievens, F., & Patterson, F. (2011). The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. Journal of Applied Psychology, 96, 927–940. doi:10.1037/a0023496
   PubMed Web of Science ®Google Scholar
• Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37, 426–441. doi:10.1108/00483480810877598
   Web of Science ®Google Scholar
• Lievens, F., & Sackett, P. R. (2007). Situational judgment tests in high-stakes settings: Issues and strategies with generating alternate forms. Journal of Applied Psychology, 92, 1043–1055. doi:10.1037/0021-9010.92.4.1043
   PubMed Web of Science ®Google Scholar
• Likert, R. (1961). New patterns of management. New York, NY: McGraw-Hill.
   Google Scholar
• McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730–740. doi:10.1037/0021-9010.86.4.730
   PubMed Web of Science ®Google Scholar
• McDaniel, M. A., & Whetzel, D. L. (2007). Situational judgment tests. In D. L. Whetzel & G. R. Wheaton (Eds.), Applied measurement: Industrial psychology in human resources management (pp. 235–257). Mahwah, NJ: Erlbaum.
   Google Scholar
• Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640–647. doi:10.1037/0021-9010.75.6.640
   Web of Science ®Google Scholar
• Olson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P. A., & Donovan, M. A. (1998). An interactive video assessment of conflict resolution skills. Personnel Psychology, 51, 1–24. doi:10.1111/j.1744-6570.1998.tb00714.x
   Web of Science ®Google Scholar
• Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A. (2007). In support of personality assessment in organizational settings. Personnel Psychology, 60, 995–1027. doi:10.1111/j.1744-6570.2007.00099.x
   Web of Science ®Google Scholar
• Ployhart, R. E., & Ehrhart, M. G. (2003). Be careful what you ask for: Effects of response instructions on the construct validity and reliability of situational judgment tests. International Journal of Selection and Assessment, 11, 1–16. doi:10.1111/1468-2389.00222
   Web of Science ®Google Scholar
• Sackett, P. R., & Lievens, F. (2008). Personnel selection. Annual Review of Psychology, 59, 419–450. doi:10.1146/annurev.psych.59.103006.093716
   PubMed Web of Science ®Google Scholar
• Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychological Methods, 8, 206–224. doi:10.1037/1082-989X.8.2.206
   PubMed Web of Science ®Google Scholar
• Schmitt, N., & Chan, D. (2006). Situational judgment tests: Method or construct? In J. Weekley & R. E. Ployhart (Eds.), Situational judgment tests (pp. 135–156). Mahwah, NJ: Lawrence Erlbaum.
   Google Scholar
• Stogdill, R. M. (1950). Leadership, membership and organization. Psychological Bulletin, 47, 1–14. doi:10.1037/h0053857
   Web of Science ®Google Scholar
• Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2002). The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty. Journal of Applied Psychology, 87, 345–354. doi:10.1037/0021-9010.87.2.345
   PubMed Web of Science ®Google Scholar
• Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25–49. doi:10.1111/j.1744-6570.1997.tb00899.x
   Web of Science ®Google Scholar
• Weekley, J. A., & Jones, C. (1999). Further studies of situational tests. Personnel Psychology, 52, 679–700. doi:10.1111/j.1744-6570.1999.tb00176.x
   Web of Science ®Google Scholar
• Weekley, J. A., & Ployhart, R. E. (2006). Situational judgment tests: Theory, measurement, and application. Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Whetzel, D. L., & McDaniel, M. A. (2009). Situational judgment tests: An overview of current research. Human Resource Management Review, 19, 188–202. doi:10.1016/j.hrmr.2009.03.007
   Web of Science ®Google Scholar
• Woehr, D. J., & Arthur, W. (2003). The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management, 29, 231–258. doi:10.1177/014920630302900206
   Google Scholar