The Restricted EM Algorithm for Maximum Likelihood Estimation under Linear Restrictions on the Parameters

Abstract

The EM algorithm is one of the most powerful algorithms for obtaining maximum likelihood estimates for many incomplete-data problems. But when the parameters must satisfy a set of linear restrictions, the EM algorithm may be too complicated to apply directly. In this article we propose maximum likelihood estimation procedures under a set of linear restrictions for situations in which the EM algorithm could be used if there were no such restrictions on the parameters. We develop a modification to the EM algorithm, which we call the restricted EM algorithm, incorporating the linear restrictions on the parameters. This algorithm is easily updated by using the code for the complete data information matrix and the code for the usual EM algorithm. Major applications of the restricted EM algorithm are to construct likelihood ratio tests and profile likelihood confidence intervals. We illustrate the procedure with two models: a variance component model and a bivariate normal model.

Key Words:

• Bivariate normal model
• Incomplete data problems
• Lagrange multipliers
• Likelihood ratio test
• Profile likelihood confidence interval
• Restricted maximum likelihood estimation
• Variance component model