Interchangeability of Results From Computerized and Traditional Administration of the BIDR: Convenience Can Match Reality

References

• American Psychological Association. (1986). Guidelines for computer-based tests and interpretations. Washington, DC: Author.
   Google Scholar
• American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: Author.
   Google Scholar
• Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord, M. R. Novick, & A. Birnbaum (Ed.), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison-Wesley.
   Google Scholar
• Boo, J., & Vispoel, W. (2012). Computer versus paper-and-pencil assessment of educational development: A comparison of psychometric features and examinee preferences. Psychological Reports, 111, 443–460. doi:10.2466/10.03.11.PR0.111.5.443-460
   PubMed Web of Science ®Google Scholar
• Booth-Kewley, S., Edwards, J. E., & Rosenfeld, P. (1992). Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? Journal of Applied Psychology, 77, 562–566. doi:10.1037/0021-9010.77.4.562
   Web of Science ®Google Scholar
• Booth-Kewley, S., Larson, G. E., & Miyoshi, D. K. (2007). Social desirability effects on computerized and paper-and-pencil questionnaires. Computers in Human Behavior, 23(1), 463–477. doi:10.1016/j.chb.2004.10.020
   Web of Science ®Google Scholar
• Box, G. E. P. (1950). Problems in the analysis of growth and wear curves. Biometrics, 6, 362–389.
   PubMed Web of Science ®Google Scholar
• Byrne, B. M. (1996). Measuring self-concept across the life span: Methodological issues and selected instrumentation. Washington, DC: APA Books.
   Google Scholar
• Cai, L. (2017). flexMIRT^(R) version 3.51: Flexible multilevel multidimensional item analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group.
   Google Scholar
• Cervellione, K. L., Lee, Y. S., & Bonanno, G. A. (2009). Rasch modeling of the Self-Deception Scale of the Balanced Inventory of Desirable Responding. Educational and Psychological Measurement, 69, 438–458. doi:10.1177/0013164408322020
   Web of Science ®Google Scholar
• Charter, R. A., & Feldt, L. S. (1996). Testing the equality of two alpha coefficients. Perceptual and Motor Skills, 82, 763–768. doi:10.2466/pms.1996.82.3.763
   Web of Science ®Google Scholar
• Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. doi:10.1080/10705510701301834
   Web of Science ®Google Scholar
• Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. doi:10.1207/S15328007SEM0902_5
   Web of Science ®Google Scholar
• Clark, L. A. (1993). Schedule for Nonadaptive and Adaptive Personality (SNAP). Manual for administration, scoring, and interpretation. Minneapolis, MN: University of Minnesota Press.
   Google Scholar
• Clough, S. J. (2008). Computerized versus paper-and-pencil assessment of socially desirable responding: Score congruence, completion time, and respondent preferences (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses database (UMI No. 3340215).
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavior sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
   Google Scholar
• CPP, Inc. (2009). Strong Interest Inventory. Retrieved from https://www.cpp.com/products/strong/index.aspx
   Google Scholar
• Crowne, D., & Marlowe, D. (1964). The approval motive. New York, NY: Wiley.
   Google Scholar
• Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. (1972). An MMPI handbook: Vol. II. Research applications. Minneapolis, MN: University of Minnesota Press.
   Google Scholar
• Davis, C., & Cowles, M. (1989). Automated psychological testing: Method of administration, need for approval, and measures of anxiety. Educational and Psychological Measurement, 49, 311–320. doi:10.1177/0013164489492002
   Web of Science ®Google Scholar
• Diedenhofen, B. (2016). cocron: Statistical comparisons of two or more alpha coefficients (Version 1.0-1). Retrieved from http://comparingcronbachalphas.org
   Google Scholar
• Diedenhofen, B., & Musch, J. (2015). cocor: A comprehensive solution for the statistical comparison of correlations. PLoS ONE, 10(4), e0121945. doi:10.1371/journal.pone.0121945
   PubMed Web of Science ®Google Scholar
• Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online, and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. doi:10.1016/j.chb.2014.04.005
   Web of Science ®Google Scholar
• Dwight, S. A., & Feigelson, M. E. (2000). A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and Psychological Measurement, 60, 340–360. doi:10.1177/00131640021970583
   Web of Science ®Google Scholar
• Edwards, A. L. (1957). The social desirability variable in personality assessment and research. New York, NY: Dryden.
   Google Scholar
• Everitt, B. S. (1979). A Monte Carlo investigation of the robustness of Hotelling's one and two sample T2 tests. Journal of the American Statistical Association, 74, 48–51.
   Web of Science ®Google Scholar
• Eysenck, H. J. (1968). Eysenck Personality Inventory manual. San Diego, CA: Educational and Industrial Testing Service.
   Google Scholar
• Finegan, J. E., & Allen, N. J. (1994). Computerized and written questionnaires: Are they equivalent? Computers in Human Behavior, 10, 483–496. doi:10.1016/0747-5632(94)90042-6
   Web of Science ®Google Scholar
• Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84–96. doi:10.1016/j.jrp.2005.08.007
   Web of Science ®Google Scholar
• Gough, H. G. (1987). Manual: The California Psychological Inventory. Palo Alto, CA: Consulting Psychologists Press.
   Google Scholar
• Hathaway, S. R., & McKinley, J. C. (1983). Minnesota Multiphasic Personality Inventory: Manual for administration and scoring: MMPI. Minneapolis, MN: University of Minnesota Press.
   Google Scholar
• Honaker, L. M., Harrell, T. H., & Buffaloe, J. D. (1988). Equivalency of microtest computer MMPI administration for standard and special scales. Computers in Human Behavior, 4, 323–337. doi:10.1016/0747-5632(88)90004-0
   Google Scholar
• Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118
   Web of Science ®Google Scholar
• Huang, C. (2013). Relation between self-esteem and socially desirable responding and the role of socially desirable responding in the relation between self-esteem and performance. European Journal of Psychology of Education, 28, 663–683. doi:10.1007/s10212-012-0134-5
   Web of Science ®Google Scholar
• International Test Commission. (2006). International guidelines on computer-based and internet-delivered testing. International Journal of Testing, 6, 143–171. doi:10.1207/s15327574ijt0602_4
   Google Scholar
• Jackson, D. N. (1976). Jackson Personality Inventory manual. Port Huron, MI: Research Psychologists Press.
   Google Scholar
• King, W. C., & Miles, E. W. (1995). A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: A test of measurement equivalence. Journal of Applied Psychology, 80, 643–651. doi:10.1037/0021-9010.80.6.643
   Web of Science ®Google Scholar
• Kreidler, S. M., Muller, K. E., Grunwald, G. K., Ringham, B. M., Coker-Dukowitz, Z. T., Sakhadeo, U. R., … Glueck, D. H. (2013). GLIMMPSE: Online power computation for linear models with and without a baseline covariate. Journal of Statistical Software, 54(10), 1–26. doi:10.18637/jss.v054.i10
   Web of Science ®Google Scholar
• Levene, H. (1960). Robust tests for equality of variances. In I. Olkin, S. Ghurye, W. Hoeffding, W. Madow, & H. Mann, (Eds.), Contributions to probability and statistics: Essay in honor of Harold Hotelling (pp. 278–292). Stanford, CA: Stanford University Press.
   Google Scholar
• Li, A., & Bagger, J. (2007). The Balanced Inventory of Desirable Responding (BIDR): A reliability generalization study. Educational and Psychological Measurement, 67, 525–544. doi:10.1177/0013164406292087
   Web of Science ®Google Scholar
• Liu, Y., Millsap, R. E., West, S. G., Tein, J. Y., Tanaka, R., & Grimm, K. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22, 486–506. doi:10.1037/met0000075
   PubMed Web of Science ®Google Scholar
• Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
   Google Scholar
• Marsh, H. W. (1992). Self-Description Questionnaire (SDQ) III: A theoretical and empirical basis for the measurement of multiple dimensions of late adolescent self-concept. An interim test manual and research monograph. Macarthur, NSW, Australia: University of Western Sydney.
   Google Scholar
• Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479–515. doi:10.1207/S15327906MBR3903_4
   Web of Science ®Google Scholar
• Murphy, C. A., Coover, D., & Owen, S. V. (1989). Development and validation of the Computer Self-Efficacy Scale. Educational and Psychological Measurement, 49, 893–899. doi:10.1177/001316448904900412
   Web of Science ®Google Scholar
• Myers & Briggs Foundation. (2016). The Myers & Briggs Foundation: Take the MBTI® Instrument. Retrieved from http://www.myersbriggs.org/my-mbti-personality-type/take-the-mbti-instrument/
   Google Scholar
• Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA: Academic.
   Google Scholar
• Paulhus, D. L. (1999). Paulhus Deception Scales (PDS): The Balanced Inventory of Desirable Responding–7, user's manual. Toronto, ON, Canada: Multi-Health Systems.
   Google Scholar
• Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In H. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 67–88). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Paulhus, D. L. & Trapnell, P. D. (2008). Self-presentation: An agency-communion framework. In O. P. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality psychology: Theory and research (3rd ed., pp. 492–517). New York, NY: Guilford.
   Google Scholar
• Pearson Education. (2016). Minnesota Multiphasic Personality Inventory–2. Retrieved from http://www.pearsonclinical.com/psychogy/products/100000461/minneasota-multiphasic-personality-inventory-2-mmpi-2.html
   Google Scholar
• Potosky, D., & Bobko, P. (1997). Computer versus paper-and-pencil administration mode and response distortion in noncognitive selection tests. Journal of Applied Psychology, 82, 293–299. doi:10.1037/0021-9010.82.2.293
   PubMed Web of Science ®Google Scholar
• Reynolds, W. M. (1982). Development of reliable and valid short forms of the Marlowe–Crowne Social Desirability Scale. Journal of Clinical Psychology, 38, 119–125. doi:10.1002/1097-4679(198201)38:1<119::AID-JCLP2270380118>3.0.CO;2-I
   Web of Science ®Google Scholar
• Richman, W. L., Kiesler, S., Weisband, S., & Drasgow, F. (1999). A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology, 84, 754–775. doi:10.1037/0021-9010.84.5.754
   Web of Science ®Google Scholar
• Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.
   Google Scholar
• Rosenfeld, P., Booth-Kewley, S., Edwards, J. E., & Thomas, M. D. (1996). Responses on computer surveys: Impression management, social desirability, and the big brother syndrome. Computers in Human Behavior, 12, 263–274. doi:10.1016/0747-5632(96)00006-4
   Web of Science ®Google Scholar
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.
   Web of Science ®Google Scholar
• Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34 (Suppl. 1), 1–97. doi:10.1007/BF03372160
   Web of Science ®Google Scholar
• Samejima, F. (1997). Graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85–100). New York, NY: Springer.
   Google Scholar
• Sanitioso, R., & Reynolds, J. H. (1992). Comparability of standard and computerized administration of two personality questionnaires. Personality and Individual Differences, 13, 899–907. doi:10.1016/0191-8869(92)90007-C
   Web of Science ®Google Scholar
• Schuldberg, D. (1988). The MMPI is less sensitive to the automated testing format than it is to repeated testing: Item and scale effects. Computers in Human Behavior, 4, 285–298. doi:10.1016/0747-5632(88)90001-5
   Google Scholar
• Scissons, E. H. (1976). Computer administration of the California Psychological Inventory. Measurement and Evaluation in Guidance, 9, 22–25.
   Web of Science ®Google Scholar
• SIGMA Assessment Systems. (2012). Jackson Vocational Interest Survey. Retrieved from http://www.sigmaassessmentsystems.com/assessments/jvis.asp
   Google Scholar
• Simms, L. J., & Clark, L. A. (2005). Validation of a computerized adaptive version of the Schedule for Nonadaptive and Adaptive Personality (SNAP). Psychological Assessment, 17(1), 28–43. doi:10.1037/1040-3590.17.1.28
   PubMed Web of Science ®Google Scholar
• Smiley, D. S. (1991). The effects of the computer assessor on social desirability response bias (Unpublished doctoral dissertation). Florida State University, Tallahassee, FL.
   Google Scholar
• Stöber, J., Dette, D. E., & Musch, J. (2002). Comparing continuous and dichotomous scoring of the Balanced Inventory of Desirable Responding. Journal of Personality Assessment, 78, 370–389. doi:10.1207/S15327752JPA7802_10
   PubMed Web of Science ®Google Scholar
• Tabachnick B. G., & Fidell L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights, MA: Allyn & Bacon.
   Google Scholar
• Thomson, J. (2010). Equivalence of mode of administration: Pencil-and-paper versus Internet-administered psychological assessments used in psychological support (Master's thesis). Retrieved from ProQuest Dissertations & Theses Full Text database (UMI No. 1482802).
   Google Scholar
• Tourangeau, R., Couper, M. P., & Conrad, F. (2004). Spacing, position, and order interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368–393. doi:10.1093/poq/nfh035
   Web of Science ®Google Scholar
• Vispoel, W. P. (2000). Computerized versus paper-and-pencil assessment of self-concept: Score comparability and respondent preferences. Measurement and Evaluation in Counseling and Development, 33, 130–143.
   Web of Science ®Google Scholar
• Vispoel, W. P., Boo, J., & Bleiler, T. (2001). Computerized and paper-and-pencil versions of the Rosenberg Self-Esteem Scale: A comparison of psychometric features and respondent preferences. Educational and Psychological Measurement, 61, 461–474. doi:10.1177/00131640121971329
   Web of Science ®Google Scholar
• Vispoel, W. P., & Forte Fast, E. E. (2000). Response biases and their relation to sex differences in multiple domains of self-concept. Applied Measurement in Education, 13(1), 79–97. doi:10.1207/s15324818ame1301_4
   Web of Science ®Google Scholar
• Vispoel, W. P., & Kim, H. Y. (2014). Psychometric properties for the Balanced Inventory of Desirable Responding: Dichotomous versus polytomous conventional and IRT scoring. Psychological Assessment, 26, 878–891. doi:10.1037/a0036430
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2016). Using G-theory to enhance evidence of reliability and validity for common uses of the Paulhus Deception Scales. Assessment. Advance online publication. doi:10.1177/1073191116641182
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2017). Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychological Methods. Advance online publication. doi:10.1037/met0000107
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018). Practical applications of generalizability theory for designing, evaluating, and improving psychological assessments. Journal of Personality Assessment 100, 53–67. doi:10.1080/00223891.2017.1296455
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., & Tao, S. (2013). A generalizability analysis of score consistency for the Balanced Inventory of Desirable Responding. Psychological Assessment, 25, 94–104. doi:10.1037/a0029061
   PubMed Web of Science ®Google Scholar
• Watson, C. G., Manifold, V., Klett, W. G., Brown, J., Thomas, D., & Anderson, D. (1990). Comparability of computer- and booklet-administered Minnesota Multiphasic Personality Inventories among primarily chemically dependent patients. Psychological Assessment, 2, 276–280. doi:10.1037/1040-3590.2.3.276
   Google Scholar
• Weigold, A., Weigold, I. K., Drakeford, N. M., Dykema, S. A., & Smith, C. A. (2016). Equivalence of paper-and-pencil and computerized self-report surveys in older adults. Computers in Human Behavior, 54, 407–413. doi:10.1016/j.chb.2015.08.033
   Web of Science ®Google Scholar
• Weigold, A., Weigold, I. K., & Russell, E. J. (2013). Examination of the equivalence of self-report survey-based paper-and-pencil and Internet data collection methods. Psychological Methods, 18, 53–70. doi:10.1037/a0031607
   PubMed Web of Science ®Google Scholar
• Weijters, B., De Beuckelaer, A., & Baumgartner, H. (2014). Discriminant validity where there should be none: Positioning same-scale items in separated blocks of a questionnaire. Applied Psychological Measurement, 38, 450–463. doi:10.1177/0146621614531850
   Web of Science ®Google Scholar
• Wiggins, J. S. (1966). Substantive dimensions of self-report in the MMPI item pool. Psychological Monographs: General and Applied, 80(22), 1–42. doi:10.1037/h0093901
   PubMedGoogle Scholar
• Wilcox, R. (2015). Comparing the variances of two dependent variables. Journal of Statistical Distributions and Applications, 2, 1. doi:10.1186/s40488-015-0030-z
   Google Scholar
• Wilkerson, J. M., Nagao, D. H., & Martin, C. L. (2002). Socially desirable responding in computerized questionnaires: When questionnaire purpose matters more than the mode. Journal of Applied Social Psychology, 32, 544–559. doi:10.1111/j.1559-1816.2002.tb00229.x
   Web of Science ®Google Scholar
• Yu, C.-Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses database (UMI No. 3066425).
   Google Scholar