Remap: Recursive Estimation and Maximization of a Posteriori Probabilities

TitleRemap: Recursive Estimation and Maximization of a Posteriori Probabilities
Publication TypeTechnical Report
Year of Publication1994
AuthorsBourlard, H., Konig Y., & Morgan N.
Other Numbers934

In this report, we describe the theoretical formulation of REMAP, an approach for the training and estimation of posterior probabilities using a recursive algorithm that is reminiscent of the EM (Expectation Maximization) algorithm for the estimation of data likelihoods. Although very general, the method is developed in the context of a statistical model for transition-based speech recognition using Artificial Neural Networks (ANN) to generate probabilities for hidden Markov models (HMMs). In the new approach, we use local conditional posterior probabilities of transitions to estimate global posterior probabilities of word sequences given acoustic speech data. Although we still use ANNs to estimate posterior probabilities, the network is trained with targets that are themselves estimates of local posterior probabilities. These targets are iteratively re-estimated by the REMAP equivalent of the forward and backward recursions of the Baum-Welch algorithm to guarantee regular increase (up to a local maximum) of the global posterior probability. Convergence of the whole scheme is proven.Unlike most previous hybrid HMM/ANN systems that we and others have developed, the new formulation determines the most probable word sequence, rather than the utterance corresponding to the most probable state sequence. Also, in addition to using all possible state sequences, the proposed training algorithm uses posterior probabilities at both local and global levels and is discriminant in nature.

Bibliographic Notes

ICSI Technical Report TR-94-064

Abbreviated Authors

H. Bourlard, Y. Konig, and N. Morgan

ICSI Publication Type

Technical Report