A Revisit of Zumbo’s Third Generation DIF: How Are We Doing in Language Testing?

ABSTRACT

The purpose of this study is to review the status of differential item functioning (DIF) research in language testing, particularly as it relates to the investigation of sources (or causes) of DIF, which is a defining characteristic of the third generation DIF. This review included 110 DIF studies of language tests dated from 1985 to 2019. We found that DIF researchers did not address sources of DIF more frequently in recent years than in earlier years. Nevertheless, DIF research in language testing has expanded with new DIF analysis procedures, more grouping variables, and more diversified methods for investigating sources of DIF. In addition, in the early years of DIF research, methods to identify sources of DIF relied heavily on content analysis. This review showed that while more sophisticated statistical procedures have been adopted in recent years to address sources of DIF, understanding sources of DIF still remains a challenging task. We also discuss the pros and cons of existing methods to detect sources of DIF and implications for future investigations.

摘要

差别试题功能 (DIF) 用来检测能力相匹配的群体在应答某个题目时是否表现出差异。本文旨在对语言测试领域的DIF研究进行一个综述, 尤其是引起DIF的来源是什么。DIF来源是Zumbo的第三代DIF研究的一个主要特点。本综述包括了1985年到2019年之间的110篇语言测试领域的DIF文章。和以往相比, 近些年的语言测试领域的DIF研究并没有更加频繁地研究DIF的来源。我们发现DIF研究扩展到了更新的DIF分析方法, 更多的DIF分组变量, 和更加多元化的研究DIF来源的方法。在早期关于DIF来源的文章中, 研究人员使用的方法主要是试题内容分析。近些年来, 研究人员开始使用更加复杂的统计分析方法。然而了解DIF的来源仍然是一个充满挑战性的题目。最后, 我们讨论了探寻DIF来源的各个分析手段的优缺点, 也对未来研究方法提出了建议.

Acknowledgments

The authors are indebted to Antony Kunnan and two anonymous reviewers for their valuable suggestions.

Disclosure statement

No potential conflict of interest was reported by the author(s).