The Efficiency of Multinomial Logistic Regression Compared with Multiple Group Discriminant Analysis

Abstract

Multinomial logistic regression (also referred to as polychotomous logistic regression) is frequently used for the analysis of categorical response data with continuous or categorical explanatory variables. Parameter estimates are usually obtained through direct maximum likelihood estimation. Normal discriminant analysis has been used as an alternative approach to this methodology, although it is strictly appropriate only when the usual normal discriminant assumptions concerning the explanatory variables are valid.

Comparative evaluations of the two procedures, including the investigation of bias and efficiency in parameter estimation, have been almost entirely limited to the case of two response groups for both normal and nonnormal explanatory variables. Bull and Donner (1987) extended the comparison of logistic regression and normal discrimination in the normal case to more than two response groups, deriving the large sample distribution of the slope parameter estimates for each of the two procedures. The asymptotic relative estimating efficiency (ARE) associated with multiple logistic regression was defined by the ratio of the large sample variances.

In this article detailed evaluations are given for two specific cases of interest in which the ARE is parameterized only by the response group frequencies and the generalized distances between them. For the case of collinear logistic slope vectors, the presence of additional response groups can have substantial influence on the ARE, especially for equal response group frequencies. The range of the ARE changes from 50% to 65% with two groups to 35% to 95% with four groups for generalized distances of 3.0 to 3.5 between the first two response groups. For the case of orthogonal logistic slope vectors, the ARE generally decreases in the presence of an additional response group.

Key Words:

• Polychotomous logistic regression
• Asymptotic relative efficiency
• Normal discrimination
• Maximum likelihood estimation
• Categorical data