Paired-Samples Tests of Equivalence

Abstract

Equivalence tests are used when the objective is to find that two or more groups are nearly equivalent on some outcome, such that any difference is inconsequential. Equivalence tests are available for several research designs, however, paired-samples equivalence tests that are accessible and relevant to the research performed by psychologists have been understudied. This study evaluated parametric and nonparametric two one-sided paired-samples equivalence tests and a standardized paired-samples equivalence test developed by Wellek (2003). The two one-sided procedures had better Type I error control and greater power than Wellek's test, with the nonparametric procedure having increased power with non normal distributions.

Keywords:

• Test of equivalence
• Paired-samples

2000 Mathematics Subject Classification:

• 91Cxx
• 91Exx
• 62-07

Acknowledgment

This research was supported in part by the Social Sciences and Humanities Research Council.

Notes

Note. λ₃ = skewness, λ₄ = kurtosis

*Standard deviation of the differences which only applies to Wellek's test.

Note. Vars = relationship between the true population variance and the estimated population variance, which only affects the Wellek procedure; ρ refers to the correlation between paired data; TOST-P = the two one-sided testing procedure for equivalence introduced by Schuirmann (1981) applied to paired observations; NPAR = the non-parametric TOST procedure for equivalence applied to paired observations, Student = traditional Student's paired-samples t-statistic.

¹This is actually measuring Type II errors, or “power” to detect equivalence.

Note. Vars = relationship between the true population variance and the estimated population variance, which only affects the Wellek procedure; ρ refers to the correlation between paired data; TOST-P = the two one-sided testing procedure for equivalence introduced by Schuirmann (1981) applied to paired observations; NPAR = the non-parametric TOST procedure for equivalence applied to paired observations, Student = traditional Student's paired-samples t-statistic.

¹This is actually measuring Type II errors, or “power” to detect equivalence.

Note. Vars = relationship between the true population variance and the estimated population variance, which only affects the Wellek procedure; ρ refers to the correlation between paired data; TOST-P = the two one-sided testing procedure for equivalence introduced by Schuirmann (1981) applied to paired observations; NPAR = the non-parametric TOST procedure for equivalence applied to paired observations, Student = traditional Student's paired-samples t-statistic.

¹This is actually measuring Type II errors, or “power” to detect equivalence.