The Development of a Standardized Effect Size for the SIBTEST Procedure

Abstract

In this study a standardized effect size was created for use with the SIBTEST procedure. Using this standardized effect size, a single set of heuristics was developed that are appropriate for data fitting different item response models (e.g., 2-parameter logistic, 3-parameter logistic). The standardized effect size rescales the raw beta-uni value using a pooled variation that incorporates the beta-uni inclusion factor. Although the heuristics for the standardized and unstandardized effect sizes provide similar true-positive and false-positive rates in most conditions, the standardized effect size provides higher true-positive rates for conditions where item response variability is smaller in proportion to raw score differences. Inflated false-positive rates were solely impacted by smaller sample sizes, whereas larger sample sizes improved true-positive rates. An empirical application is provided to demonstrate how the standardized effect size provides for a more consistent comparison across items with varying response distributions. This study lays the foundation for the utilization of a standardized effect size for both dichotomous and polytomous item response models using the suite of SIBTEST procedures.

Keywords:

• DIF
• differential item functioning
• SIBTEST
• standardized effect size

Disclosure statement

We have no known conflict of interest to disclose.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.