Using DDF in a Post Hoc Analysis to Understand Sources of DIF

Abstract

The purpose of this article is to describe and demonstrate a three-step process of using differential distractor functioning (DDF) in a post hoc analysis to understand sources of differential item functioning (DIF) in multiple-choice testing. The process is demonstrated on two multiple-choice tests that used complex alternatives (e.g., “No Mistakes”) as distractors. Comparisons were made between different gender and race groups. DIF analyses were conducted using Simultaneous Item Bias Test, whereas DDF analyses were conducted using loglinear model fitting and odds ratios. Five items made it through all three steps and were identified as those with DIF results related to DDF. Implications of the results, as well as suggestions for future research, are discussed.

Notes

¹Item 12 can be found at http://www.actstudent.org/sampletest/english/eng_03.html

²SAT Writing Practice Exam can be found at http://www.tea.state.tx.us/student.assessment/resources/release/taks/2006/grxltaksjuly.pdf