Adaptive Parameter Training in an Interpolated N-gram Language Model^∗

Abstract

An interpolated n -gram language model is an essential component of any large vocabulary speech recognition system (Katz, 1987; O’Boyle, Owens & Smith, 1994; O’Boyle, Ming, McMahon, & Smith, 1996). It can provide estimates of the probability of any particular sequence of words. These probabilities are combined with likelihoods derived from acoustic models to determine the most likely word sequence for any given acoustic signal. An n -gram model provides estimates for the probability of each word in a fixed vocabulary following any given sequence of words. Simple maximum likelihood estimates can be calculated from the frequency of n word sequences (or n -grams) in a training text; however, these estimates are unreliable due to the sparseness of the data, even for small values of n with large amounts of training text. Interpolation between n -gram maximum likelihood estimates and lower order estimates can be used to overcome this sparse data problem. We have previously shown how a simple weighting function can be used to produce an effective weighted average n -gram language model (O’Boyle et al., 1994). This has a performance approaching that of similar models where optimized parameters are used to determine the best mixture of the component probability distributions. In this paper we present a new approach in which the interpolation parameters used to determine the final blend of probability distributions are dynamically adapted as the test material is processed.