Using a Fractional Factorial Design to Increase Direct Mail Response at Mother Jones Magazine

Abstract

Experimental design methods have been widely applied to problems in manufacturing, but little attention has been given to applying these tools to service problems. In the world of direct marketing, the traditional approach is called A/B testing and involves changing one factor at a time. In this article we address the problem of improving direct mail response at Mother Jones magazine, employing a 16-run two-level fractional factorial design that tests seven factors simultaneously. We compare this design to other possible design choices, discuss sample size determination, and show how we labeled factors to isolate likely two-factor interactions. We discuss the results and conclusions of our study and the benefits to Mother Jones magazine.

Keywords:

• Confounding
• Direct mail marketing
• Fractional factorial experiments
• Mother Jones magazine
• Sample size

ACKNOWLEDGMENT

The authors thank Jay Harris, publisher of Mother Jones, for his valuable input and support.

Notes

Significant effects with probability values 0.10 or smaller are in boldface.