Maximum Likelihood Linear Regression

Maximum likelihood linear regression or MLLR computes a set of transformations that will reduce the mismatch between an initial model set and the adaptation data^9.2. More specifically MLLR is a model adaptation technique that estimates a set of linear transformations for the mean and variance parameters of a Gaussian mixture HMM system. The effect of these transformations is to shift the component means and alter the variances in the initial system so that each state in the HMM system is more likely to generate the adaptation data. Note that due to computational reasons, MLLR is only implemented within HTK for diagonal covariance, single stream, continuous density HMMs.

The transformation matrix used to give a new estimate of the adapted mean is given by

$$\hat{{\mbox{\boldmath$\mu$}}} = {\mbox{\boldmath$W$}}{\mbox{\boldmath$\xi$}},$$ (9.1)

where ${\mbox{\boldmath$W$}}$ is the $n \times \left( n + 1 \right)$ transformation matrix (where $n$ is the dimensionality of the data) and ${\mbox{\boldmath$\xi$}}$ is the extended mean vector,

$${\mbox{\boldmath$\xi$}} = \left[\mbox{ }w\mbox{ }\mu_1\mbox{ }\mu_2\mbox{ }\dots\mbox{ }\mu_n\mbox{ }\right]^T$$

where $w$ represents a bias offset whose value is fixed (within HTK) at 1.
Hence ${\mbox{\boldmath$W$}}$ can be decomposed into

$${\mbox{\boldmath$W$}} = \left[\mbox{ }{\mbox{\boldmath$b$}}\mbox{ }{\mbox{\boldmath$A$}}\mbox{ }\right]$$ (9.2)

where ${\mbox{\boldmath$A$}}$ represents an $n \times n$ transformation matrix and ${\mbox{\boldmath$b$}}$ represents a bias vector.

The transformation matrix ${\mbox{\boldmath$W$}}$ is obtained by solving a maximisation problem using the Expectation-Maximisation (EM) technique. This technique is also used to compute the variance transformation matrix. Using EM results in the maximisation of a standard auxiliary function. (Full details are available in section 9.4.)