Robust and scale-free effect sizes for non-Normal two-sample comparisons, with applications in e-commerce

Abstract

The effect size (ES) has been mainly introduced and investigated for changes in location under an assumption of Normality for the underlying population. However, there are many circumstances where populations are non-Normal, or depend on scale and shape and not just a location parameter. Our motivating application from e-commerce requires an ES which is appropriate for long-tailed distributions. We review some common ES measures. We then introduce two novel alternative ES for two-sample comparisons, one scale-free and one on the original scale of measurement, and analyse some theoretical properties. We examine these ES for two-sample comparison studies under an assumption of Normality and investigate what happens when both location and scale parameters differ. We explore ES for phenomena for non-Normal situations, using the Weibull family for illustration. Finally, for an application, we assess differences in customer behaviour when browsing E-commerce websites.

Keywords:

• effect size
• two-sample comparison
• non-Normal distribution
• Weibull
• quantile function
• e-commerce
• long-tail distribution