Testing 40 Predictions From the Transtheoretical Model Again, With Confidence

Abstract

Testing Theory-based Quantitative Predictions (TTQP) represents an alternative to traditional Null Hypothesis Significance Testing (NHST) procedures and is more appropriate for theory testing. The theory generates explicit effect size predictions and these effect size estimates, with related confidence intervals, are used to test the predictions. The focus of a study is shifted to a quantitative approach in contrast to the NHST dyadic decision centered on testing a prediction not based on the theory. This article describes the TTQP as an alternative approach by replicating and extending a test of 40 a priori predictions based on the Transtheoretical Model (TTM). Specific quantitative predictions were made about the magnitude of the effect size (ω²). The predictions involved movement from 1 of 3 initial stages (Precontemplation, Contemplation, and Preparation) to stage membership 12 months later. In the initial study, 36 of the 40 predictions were confirmed. The same 40 predictions are evaluated on a sample (N = 3,923) of smokers recruited from a large New England HMO for a smoking cessation study. The predictions were recalibrated based on the first study and 99% confidence intervals were employed to test the predictions. Thirty-two of the 40 predictions were confirmed. Of the 8 failures, 4 were judged to reflect a need for further recalibration, 1 was attributed to sampling fluctuation, and 3 suggested revisions of the theory are needed. The results provide overall support for the TTM. The study also illustrates some of the challenges of testing quantitative predictions.