Introducing a Subject Matter Expert–Based Utility Analysis Approach to Assessing the Utility of Organizational Interventions Such as Crew Resource Management Training

REFERENCES

• Arthur , W. Jr. , Bennett , W. Jr. , Edens , P. S. and Bell , S. T. 2003 . Effectiveness of training in organizations: A meta-analysis of design and evaluation features . Journal of Applied Psychology , 88 : 234 – 245 .
   PubMed Web of Science ®Google Scholar
• Arthur , W. Jr. , Bennett , W. Jr. and Huffcutt , A. I. 2001 . Conducting meta-analysis using SAS , Mahwah, NJ : Lawrence Erlbaum Associates, Inc .
   Google Scholar
• Arthur , W. Jr. , Bennett , W. Jr. , Stanush , P. L. and McNelly , T. L. 1998 . Factors that influence skill decay and retention: A quantitative review and analysis . Human Performance , 11 : 57 – 101 .
   Web of Science ®Google Scholar
• Aviation Information Resources, Inc. (2004). April 2004 pilot hiring summary. http://www.jet-jobs.com/guestframe.html (http://www.jet-jobs.com/guestframe.html)
   Google Scholar
• Boudreau , J. W. and Ramstad , P. M. 2003 . “ Strategic industrial and organizational psychology and the role of utility analysis ” . In Handbook of psychology: Vol. 12. Industrial and organizational psychology , Edited by: Borman , W. C. , Ilgen , D. R. and Klimoski , R. J. 193 – 221 . New York, NY : Wiley .
   Google Scholar
• Cabrera , E. F. and Raju , N. S. 2001 . Utility analysis: Current trends and future directions . International Journal of Selection and Assessment , 9 : 92 – 102 .
   Web of Science ®Google Scholar
• Carson , K. P. , Becker , J. S. and Henderson , J. A. 1998 . Is utility really futile? . Journal of Applied Psychology , 83 : 84 – 96 .
   Web of Science ®Google Scholar
• Cascio , W. F. and Boudreau , J. W. 2008 . Investing in people: Financial impact of human resource initiatives , Upper Saddle River, NJ : FT Press .
   Google Scholar
• Cohen , J. 1992 . A power primer . Psychological Bulletin , 112 : 155 – 159 .
   PubMed Web of Science ®Google Scholar
• Day , E. A. , Arthur , W. Jr. and Gettman , D. 2001 . Knowledge structures and the acquisition of a complex skill . Journal of Applied Psychology , 86 : 1022 – 1033 .
   PubMed Web of Science ®Google Scholar
• Federal Aviation Administration . Advisory circular: Crew resource management training . No. 120-51E . 2004 . Washington, DC: U.S. Government Printing Office
   Google Scholar
• Hazer , J. T. and Highhouse , S. 1997 . Factors influencing manager's reactions to utility analysis: Effects of SDy method, information frame, and focal intervention . Journal of Applied Psychology , 82 : 104 – 112 .
   Web of Science ®Google Scholar
• Helmreich , R. L. , Merritt , A. C. and Wilhelm , J. A. 1999 . The evolution of crew resource management training in commercial aviation . International Journal of Aviation Psychology , 9 : 19 – 32 .
   PubMed Web of Science ®Google Scholar
• Hunter , J. E. The economic benefits of personnel selection using ability tests: A state-of-the-art review including a detailed analysis of the dollar benefit of U.S. Employment Service placements and critique of the low-cutoff method of test use . USES Test Research Rep. No. 47 . 1983 . Washington, DC: U.S. Employment Service, U.S. Department of Labor
   Google Scholar
• Hunter , J. E. and Schmidt , F. L. 1982 . “ Fitting people to jobs: The impact of personnel selection on national productivity ” . In Human performance and productivity: Vol. 1. Human capability assessment , Edited by: Fleishman , E. A. and Dunnette , M. D. 232 – 284 . Hillsdale, NJ : Lawrence Erlbaum Associates, Inc .
   Google Scholar
• Hunter , J. E. and Schmidt , F. L. 1983 . Quantifying the effects of psychological interventions on employee job performance and work-force productivity . American Psychologist , 38 : 473 – 478 .
   Web of Science ®Google Scholar
• James , L. R. , Demaree , R. G. and Wolf , G. 1984 . Estimating within-group interrater reliability with and without response bias . Journal of Applied Psychology , 69 : 85 – 98 .
   Web of Science ®Google Scholar
• Kirkpatrick , D. L. 1976 . “ Evaluation of training ” . In Training and development handbook: A guide to human resource development , 2nd , Edited by: Craig , R. L. 301 – 319 . New York, NY : McGraw-Hill .
   Google Scholar
• Kirkpatrick , D. L. 1996 . Invited reaction: Reaction to Holton article . Human Resource Development Quarterly , 7 : 23 – 25 .
   Google Scholar
• Kraiger , K. , Ford , J. K. and Salas , E. 1993 . Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation . Journal of Applied Psychology , 78 : 311 – 328 .
   Web of Science ®Google Scholar
• Latham , G. P. and Whyte , G. 1994 . The futility of utility analysis . Personnel Psychology , 47 : 31 – 46 .
   Web of Science ®Google Scholar
• Morris , S. B. 2008 . Estimating effect sizes from pretest–posttest–control group designs . Organizational Research Methods , 11 : 364 – 386 .
   Web of Science ®Google Scholar
• Muchinsky , P. M. 2006 . Psychology applied to work , 8th , Belmont, CA : Thomson Wadsworth .
   Google Scholar
• Musson , D. M. and Helmreich , R. L. 2004 . Team training and resource management in healthcare: Current issues and future directions . Harvard Health Policy Review , 5 : 25 – 35 .
   Google Scholar
• National Transportation Safety Board. (2005). Accident and incident data system. http://www.nasdac.faa.gov/portal/page?_pageid=33,37327&_dad=portal&_schema=PORTAL (http://www.nasdac.faa.gov/portal/page?_pageid=33,37327&_dad=portal&_schema=PORTAL)
   Google Scholar
• O'Connor , P. , Campbell , J. , Newon , J. , Melton , J. , Salas , E. and Wilson , K. A. 2008 . Crew resource management training effectiveness: A meta-analysis and some critical needs . International Journal of Aviation Psychology , 18 : 353 – 368 .
   Web of Science ®Google Scholar
• Patankar , M. S. and Taylor , J. C. Making a business case for the human factors programs in aviation maintenance . Proceedings of the IEA 2000/HFES 2000 Congress . pp. 767 – 770 . Santa Monica, CA : Human Factors and Ergonomics Society . Vol. 3
   Google Scholar
• Pizzi, L., Goldfarb, N. I., & Nash, D. B. (2001). Crew resource management and its applications in medicine. In A. J. Markowitz (Ed.), Making health care safer: A critical analysis of patient safety practices (Chap. 44, Evidence Report/Technology Assessment No. 43). Rockville, MD: Agency for Healthcare Research and Quality. http://www.ahrq.gov/clinic/ptsafety/chap44.htm (http://www.ahrq.gov/clinic/ptsafety/chap44.htm)
   Google Scholar
• Roop , S. S. , Morgan , C. A. , Kyte , T. B. , Arthur , W. Jr. , Villado , A. J. and Beneigh , T. Rail crew resource management (CRM): The business case for CRM training in the railroad industry . Tech. Rep. No. DOT/FRA/ORD-07-21 . 2007 . College Station, TX: Texas A&M University, Texas Transportation Institute
   Google Scholar
• Roth , P. L. , Bobko , P. and Mabon , H. 2001 . “ Utility analysis: A review and analysis at the turn of the century ” . In Handbook of industrial, work, and organizational psychology: Vol. 1. Personnel psychology , Edited by: Anderson , N. , Ones , D. S. , Sinangil , H. K. and Viswesvaran , C. 363 – 384 . London, UK : Sage .
   Google Scholar
• Sackett , P. R. and Mullen , E. J. 1993 . Beyond formal experimental design: Towards an expanded view of the training evaluation process . Personnel Psychology , 46 : 613 – 627 .
   Web of Science ®Google Scholar
• Salas , E. , Burke , C. S. , Bowers , C. A. and Wilson , K. A. 2001 . Team training in the skies: Does crew resource management (CRM) training work? . Human Factors , 43 : 641 – 674 .
   PubMed Web of Science ®Google Scholar
• Schmidt , F. L. and Hunter , J. E. 1983 . Individual differences in productivity: An empirical test of estimates derived from studies of personnel selection procedure utility . Journal of Applied Psychology , 68 : 407 – 414 .
   Web of Science ®Google Scholar
• Schmidt , F. L. and Hunter , J. E. 1989 . Interrater reliability coefficients cannot be computed when only one stimulus is rated . Journal of Applied Psychology , 74 : 368 – 370 .
   Web of Science ®Google Scholar
• Schmidt , F. L. , Hunter , J. E. , McKenzie , R. C. and Muldrow , T. W. 1979 . Impact of valid selection procedures on work-force productivity . Journal of Applied Psychology , 64 : 609 – 626 .
   Web of Science ®Google Scholar
• Schmidt , F. L. , Hunter , J. E. , Outerbridge , A. N. and Trattner , M. H. 1986 . The economic impact of job selection methods on size, productivity, and payroll costs of the federal work force: An empirical based demonstration . Personnel Psychology , 39 : 1 – 29 .
   Web of Science ®Google Scholar
• Schmidt , F. L. , Hunter , J. E. and Pearlman , K. 1982 . Assessing the economic impact of personnel programs on productivity . Personnel Psychology , 35 : 333 – 347 .
   Web of Science ®Google Scholar
• Shappell , S. A. and Wiegmann , D. A. The Human Factors Analysis and Classification System (HFACS) . Office of Aviation Medicine Rep. No. DOT/FAA/AM-00/7 . 2000 . Washington, DC: Office of Aviation Medicine, Federal Aviation Administration
   Google Scholar
• Thompson , D. E. and Thompson , T. A. 1982 . Court standards for job analysis in test validation . Personnel Psychology , 35 : 865 – 874 .
   Web of Science ®Google Scholar
• U.S. Department of Labor. (2007a). Career guide to industries, 2006–07 edition, air transportation. http://www.bls.gov/oco/cg/cgs016.htm (http://www.bls.gov/oco/cg/cgs016.htm)
   Google Scholar
• U.S. Department of Labor. (2007b). Occupational outlook handbook, 2006–07 edition, aircraft pilots and flight engineers. http://www.bls.gov/oco/ocos107.htm (http://www.bls.gov/oco/ocos107.htm)
   Google Scholar
• U.S. Department of Transportation. (2007). Annual employee statistics by labor category (P10). http://www.bts.gov/programs/airline_information/number_of_employees/labor_category (http://www.bts.gov/programs/airline_information/number_of_employees/labor_category)
   Google Scholar
• Von Thaden , T. L. and Steelman , K. S. Classifying crew performance failures in commercial aviation accidents: Can we get the numbers right? . Paper presented at the second Safety Across High-Consequence Industries Conference . St. Louis, MO. September .
   Google Scholar