Additive Isotonic Models

Abstract

Additive isotonic models generalize linear models by replacing lines with isotonic (nondecreasing) transformations. Fitted transformations of several explanatory variables are added together and then transformed by a known function to yield fitted values of the response variable. The isotonic transformations are chosen to minimize an explicit criterion, such as the negative log-likelihood, by an algorithm that optimizes one transformation at a time while adjusting for the current fitted values of the others, cycling until the criterion converges. This approach can be used in various situations, notably for generalizing ordinary linear regression and linear logistic regression. At each step of the algorithm, the needed optimal isotonic transformation is found using a simple generalization of the standard pool-adjacent-violators algorithm (Ayer, Brunk, Ewing, Reid, and Silverman 1955). The fitted transformations are always made up of flat steps, so the technique is useful for finding optimal stratifications of the explanatory variables, but not for finding smooth transformations. The technique speeds the process of checking variables for possible addition to an existing model, because the possibility of finding a useful isotonic transformation can be ruled out if the best such transformation performs poorly. Isotonic regression is usually extended to multivariate settings by fitting a multivariate function that preserves a partial order on the values of all of the explanatory variables. Such models are more general than additive isotonic models, but they are less interpretable (because of their high dimension) and more prone to overfit the data. Transformations that preserve a partial order on a subset of the explanatory variables can be incorporated into additive isotonic models to model interactions within the subset. More general mixtures of techniques within the additive framework are also possible; smooth, isotonic, and parametric transformations can be applied to different explanatory variables within one additive model.

Key Words:

• Logistic regression
• Multivariate isotonic regression
• Nonparametric regression