CUSUM Hypothesis Tests and Alternative Response Probabilities for Finite Poisson Trials

Abstract

This article develops a new cumulative sum statistic to identify aberrant behavior in a sequentially administered multiple-choice standardized examination. The examination responses can be described as finite Poisson trials, and the statistic can be used for other applications which fit this framework. The standardized examination setting uses a maximum likelihood estimate of examinee ability and an item response theory model. Aberrant and non aberrant probabilities are computed by an odds ratio analogous to risk adjusted CUSUM schemes. The significance level of a hypothesis test, where the null hypothesis is non-aberrant examinee behavior, is computed with Markov chains. A smoothing process is used to spread probabilities across the Markov states. The practicality of the approach to detect aberrant examinee behavior is demonstrated with results from both simulated and empirical data.

Keywords:

• Hypothesis testing
• Item response theory
• Poisson trials
• Risk adjusted CUSUM
• Simulation
• Markov chains

Mathematics Subject Classification:

• 60J22

Acknowledgment

This research has been partially funded by a grant from the Law School Admission Council, 662 Penn Street, Newtown, PA 18940, USA.