Publication Details
Title: The Mean Field Theory in EM Procedures for Markov Random Fields
Author: J. Zhang
Group: ICSI Technical Reports
Date: January 1991
PDF: http://www.icsi.berkeley.edu/pubs/techreports/tr-91-001.pdf
Overview:
The EM (expectation maximization) algorithm is a maximum-likelihood parameter estimation procedure for incomplete data problems in which part of the data is hidden, or unobservable. In many signal processing and pattern recognition applications, the hidden data are modeled as Markov processes and the main difficulty of using the EM algorithm for these applications is the calculation of the condition expectations of the hidden Markov processes. In this paper, we show how the mean field theory from statistical mechanics can be used to efficiently calculate the conditional expectations for these problems. The efficacy of the mean field theory approach is demonstrated on the parameter estimation for one-dimensional mixture data and two-dimensional unsupervised stochastic model-based image segmentation. Experimental results indicate that in the 1-D case, the mean field theory approach provides comparable results to those obtained by Baum's algorithm, which is known to be optimal. In the 2-D case, where Baum algorithm can no longer be used, the mean field theory provides good parameter estimates and image segmentation for both synthetic and real-world images.
Bibliographic Information:
ICSI Technical Report TR-91-001
Bibliographic Reference:
J. Zhang. The Mean Field Theory in EM Procedures for Markov Random Fields. ICSI Technical Report TR-91-001, January 1991
Author: J. Zhang
Group: ICSI Technical Reports
Date: January 1991
PDF: http://www.icsi.berkeley.edu/pubs/techreports/tr-91-001.pdf
Overview:
The EM (expectation maximization) algorithm is a maximum-likelihood parameter estimation procedure for incomplete data problems in which part of the data is hidden, or unobservable. In many signal processing and pattern recognition applications, the hidden data are modeled as Markov processes and the main difficulty of using the EM algorithm for these applications is the calculation of the condition expectations of the hidden Markov processes. In this paper, we show how the mean field theory from statistical mechanics can be used to efficiently calculate the conditional expectations for these problems. The efficacy of the mean field theory approach is demonstrated on the parameter estimation for one-dimensional mixture data and two-dimensional unsupervised stochastic model-based image segmentation. Experimental results indicate that in the 1-D case, the mean field theory approach provides comparable results to those obtained by Baum's algorithm, which is known to be optimal. In the 2-D case, where Baum algorithm can no longer be used, the mean field theory provides good parameter estimates and image segmentation for both synthetic and real-world images.
Bibliographic Information:
ICSI Technical Report TR-91-001
Bibliographic Reference:
J. Zhang. The Mean Field Theory in EM Procedures for Markov Random Fields. ICSI Technical Report TR-91-001, January 1991