An Adjusted Bonferroni Method for Elimination of Parameters in Specification Addition Searches

Abstract

An adjusted Bonferroni method for controlling familywise error rate when deleting parameters in specification addition searches was presented and evaluated in a 2-part study. First, data were generated based on a factor model and then analyzed using backward selection, with and without applying the adjusted Bonferroni method. Three factors were manipulated: sample size, magnitude of weights, and number of parameters in a search. Under most of the 30 explored conditions, the empirical familywise error rates were relatively close to the nominal alpha level of. 05 when the adjusted Bonferroni method was applied. Error rates that exceeded the. 05 level appeared to be a function of the multivariate Wald test and not of the adjusted Bonferroni method. Second, data were generated based on a path model and analyzed using 2-stage and backward selection methods when the initial model was misspecified. Controlling stringently for Type I errors in the initial addition stage of a 2-stage search created selection errors when N was small (200). The adjusted Bonferroni method controlled adequately for Type I errors in the deletion stage of the 2-stage search and with backward selection.