Liberal and Conservative Differential Item Functioning Detection Using Mantel-Haenszel and SIBTEST: Implications for Type I and Type II Error Rates

References

• Ackerman, T. A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29(1), 67-91.
   Google Scholar
• Allalouf, A. (2003). Revising translated differential item functioning items as a tool for improving cross-lingual assessment. Applied Measurement in Education, 16, 55-73.
   Google Scholar
• Aptech Systems. (1993). The GAUSS System. Kent, WA: Author.
   Google Scholar
• Bennett, R. E., Rock, D. A., & Novatkoski, I. (1989). Differential item functioning on the SAT-M Braille edition. Journal of Educational Measurement, 26, 67-79.
   Google Scholar
• Bond, L. (1993). Comments on the O'Neill & McPeek paper. In P. W. Holland and H. Wainer, Differential item functioning (pp. 277-279). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Bradley, J. V. (1978). Robustness? The British Journal of Mathematical & Statistical Psychology, 31, 144-152.
   Google Scholar
• Budgell, G. R., Raju, N. S., & Quartetti, D. A. (1995). Analysis of differential item functioning in translated assessment instruments. Applied Psychological Measurement, 19, 309-321.
   Google Scholar
• Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Newbury Park, CA: Sage.
   Google Scholar
• Chan, D. (2000). Detection of differential item functioning on the Kirton Adaptation-Innovation Inventory using multiple-group mean and covariance structure analyses. Multivariate Behavioral Research, 35, 169-199.
   Google Scholar
• Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31-44.
   Google Scholar
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
   Google Scholar
• Collins, W. C., Raju, N. S., & Edwards, J. E. (2000). Assessing differential functioning in a satisfaction scale. Journal of Applied Psychology, 85, 451-461.
   Google Scholar
• Ellis, B. B. (1995). A partial test of Hulin's psychometric theory of measurement equivalence in translated tests. European Journal of Psychological Assessment, 11, 184-193.
   Google Scholar
• Ellis, B. B., Becker, P., & Kimmel, H. D. (1995). An item response theory evaluation of an English version of the Trier Personality Inventory (TPI). Journal of Cross-Cultural Psychology, 24, 133-148.
   Google Scholar
• Elosua, P., López, A., & Egaña, J. (2000). Fuentes potenciales de sesgo en una prueba de aptitud numérica [Potential sources of bias in a numerical aptitude test]. Psicothema, 12, 376-382.
   Google Scholar
• Elosua, P., López, A., & Torres, E. (2000). Desarrollos didácticos y funcionamiento diferencial de los ítems. Problemas inherentes a toda investigación empírica sobre sesgo [Didactic developments and differential item functioning]. Psicothema, 12, 198-202.
   Google Scholar
• Engelhard, G., Hansche, L., & Rutledge, E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
   Google Scholar
• Ferreres, D. (1998). Funcionamiento diferencial de los ítems de una prueba de aptitud intelectual en función de la lengua familiar y la lengua de escolarización [Differential item functioning across the different language groups in a Spanish ability test]. Unpublished doctoral dissertation, Universitat de Valencia, Spain.
   Google Scholar
• Ferreres, D., González-Romá, V., & Gómez, J. (2000). Comparación del estadístico Mantel-Haenszel y la regresión logística en el funcionamiento diferencial de los ítems en dos pruebas de aptitud intelectual en un contexto bilingüe [Mantel-Haenszel statistics and the logistic regression in the detection of DIF in two aptitude tests]. Psicothema, 12, 214-219.
   Google Scholar
• Ferreres, D., González-Romá, V., & Gómez, J. (2002). Funcionamiento diferencial de los ítems en una situación de contacto entre lenguas [Differential item functioning and linguistic characteristics of examinees]. Psicothema, 12, 214-219.
   Google Scholar
• Fidalgo, A. M. (1994). MHDIF: A computer program for detecting uniform and nonuniform differential item functioning with the Mantel-Haenszel procedure. Applied Psychological Measurement, 18, 300.
   Google Scholar
• Fidalgo, A. M. (in press). Mantel-Haenszel methods. In B. Everitt & D. Howell, Encyclopedia of statistics in behavioral science. London: Wiley.
   Google Scholar
• Fidalgo, A. M., Ferreres, D., & Muñiz, J. (in press). Utility of the Mantel-Haenszel procedure for detecting differential item functioning with small samples. Educational and Psychological Measurement.
   Google Scholar
• Gómez, J., & Navas, M. J. (1998). Impacto y funcionamiento diferencial de los ítems respecto al género en una prueba de aptitud numérica [Gender-related impact and differential item functioning in a test of numerical ability]. Psicothema, 10, 717-728.
   Google Scholar
• Hambleton, R. K., & Jones, R. W. (1994). Comparison of empirical and judgmental procedures for detecting differential item functioning. Educational Research Quarterly, 18, 21-36.
   Google Scholar
• Hammer, C. S., Pennock-Roman, M., Rzasa, S., & Tomblin, J. B. (2002). An analysis of the test of language development-primary for item bias. American Journal of Speech-Language Pathology, 11, 274-284.
   Google Scholar
• Holland, W. P., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun, Test validity (pp. 129-145). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Huang, C. D., Church, A. T., & Katigbak, M. S. (1997). Identifying cultural differences in items and traits. Differential item functioning in the NEO Personality Inventory. Journal of Cross-Cultural Psychology, 28, 192-218.
   Google Scholar
• Jiang, H., & Stout, W. (1998). Improved Type I error control and reduced estimation bias for DIF detection using SIBTEST. Journal of Educational and Behavioral Statistics, 23, 291-322.
   Google Scholar
• Kim, M. (2001). Detecting DIF across the different language groups in a speaking test. Language Testing, 18, 89-114.
   Google Scholar
• Kim, S.-H., & Cohen, A. S. (1995). A comparison of Lord's chi-square, Raju's area measures, and the likelihood ratio test on detection of differential item functioning. Applied Measurement in Education, 8, 291-312.
   Google Scholar
• Kok, F. (1988). Item bias and test multidimensionality. In R. Langeheine & J. Rost, Latent trait and latent class models (pp. 349-364). New York: Plenum Press.
   Google Scholar
• Linn, R. L. (1993). The use of differential item functioning statistics: A discussion of current practice and future implications. In P. W. Holland & H. Wainer, Differential item functioning (pp. 349-364). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334.
   Google Scholar
• Muñiz, J., Hambleton, R. K., & Xing, D. (2001). Small sample studies to detect flaws in item translations. International Journal of Testing, 1, 115-135.
   Google Scholar
• Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315-328.
   Google Scholar
• Oort, F. J. (1992). Using restricted factor analysis to detect item bias. Methodika, 6, 150-166.
   Google Scholar
• Oort, F. J. (1996). Using restricted factor analysis in test construction. Amsterdam: Universiteit van Amsterdam.
   Google Scholar
• Oshima, T. C., McGinty, D., & Flowers, C. P. (1994). Differential item functioning for a test with a cutoff score: Use of limited closed-interval measures. Applied Measurement in Education, 7, 195-209.
   Google Scholar
• Parshall, C. G., & Miller, T. R. (1995). Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302-316.
   Google Scholar
• Penfield, R. D., & Lam, T. C. M. (2000). Assessing differential item functioning in performance assessment: Review and recommendations. Educational Measurement: Issues and Practice, 19, 5-15.
   Google Scholar
• Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for politomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19, 23-37.
   Google Scholar
• Raju, N. S., Drasgow, F., & Slinde, J. A. (1993). An empirical comparison of the area methods, Lord's chi-square test, and the Mantel-Haenszel technique for assessing differential item functioning. Educational and Psychological Measurement, 53, 301-314.
   Google Scholar
• Raju, N. S., & Ellis, B. B. (2002). Differential item and test functioning. In F. Drasgow & N. Schmitt, Measuring and analysing behavior in organizations: Advances in measurement and data analysis (pp. 156-188). San Francisco, CA: Jossey-Bass.
   Google Scholar
• Reise, S. P., Smith, L., & Furr, R. M. (2001). Invariance on the NEO PI-R neuroticism scale. Multivariate Behavioral Research, 36, 83-110.
   Google Scholar
• Roussos, L., & Stout, W. (1996). A multidimensionality-based DIF paradigm. Applied Psychological Measurement, 20, 355-371.
   Google Scholar
• Scheuneman, J. D., & Gerritz, K. (1990). Using differential item functioning procedures to explore sources of item difficulty and group performance characteristics. Journal of Educational Measurement, 27, 109-131.
   Google Scholar
• Schwarz, R. D. (1998). Trace lines for classification decisions. Applied Measurement in Education, 11, 311-330.
   Google Scholar
• Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
   Google Scholar
• Smith, L. L. (2002). On the usefulness of item bias analysis to personality psychology. Personality and Social Psychology Bulletin, 28, 754-763.
   Google Scholar
• Teresi, J. A., Kleinman, M., Ocepek-Welikson, K., Ramirez, M., Gurland, B., Lantigua, R., et al. (2000). Applications of item response theory to the examination of the psychometric properties and differential item functioning of the comprehensive assessment and referral evaluation dementia diagnostic scale among samples of Latino, African American, and white non-Latino elderly. Research on Aging, 22, 738-773.
   Google Scholar
• Walker, C. M., & Beretvas, S. N. (2001). An empirical investigation demonstrating the multidimensional DIF paradigm: A cognitive explanation for DIF. Journal of Educational Measurement, 38, 147-163.
   Google Scholar
• Wang, N., & Lane, S. (1996). Detection of gender-related differential item functioning in a mathematics performance assessment. Applied Measurement in Education, 9, 175-199.
   Google Scholar
• William Stout Institute for Measurement. (1999). Dimensionality-Based DIF/DBF Package [Computer software]. Urbana, IL: Author.
   Google Scholar
• Yuste, C. (1988). B.A.D.Y.G.: Batería de aptitudes diferenciales y generales—Elemental y medio. [B.A.D.Y.G.: Differential general aptitudes battery—Elementary and medium]. Madrid: Ciencias de la Educación preescolar y especial.
   Google Scholar
• Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In W. P. Holland & H. Wainer, Differential item functioning (pp. 337-347). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36, 1-28.
   Google Scholar
• Zwick, R., Thayer, D. T., & Lewis, C. (2000). Using loss functions for DIF detection: An empirical Bayes approach. Journal of Educational and Behavioral Statistics, 25, 225-247.
   Google Scholar
• Zwick, R., Thayer, D. T., & Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10, 321-344.
   Google Scholar
• Zwick, R., Thayer, D. T., & Wingersky, M. (1994). A simulation study of methods for assessing differential item functioning in computerized adaptative tests. Applied Psychological Measurement, 18, 121-140.
   Google Scholar
• Zwick, R., Thayer, D. T., & Wingersky, M. (1995). Effect of Rasch calibration on ability and DIF estimation in computer-adaptative tests. Journal of Educational Measurement, 32, 341-363.
   Google Scholar