|
|
#### Initialisation and training of the GLLiM model
|
|
|
|
|
|
The Gaussian Locally Linear Mapping (GLLiM) model is a parametric statistical model closely related to Gaussian mixtures and mixtures of expert models. It can describe the direct and inverse interaction between X and Y by a combination of local affine transformations and is adapted to solving inversion regression problems in a Bayesian framework. We refer the reader to [Kugler et al. 2020 and Deleforge et al., 2014] for a definition and detailed description of the model.
|
|
|
The Gaussian Locally Linear Mapping (GLLiM) model is a parametric statistical model closely related to Gaussian mixtures and mixtures of expert models. It can approximate the direct and inverse relationships between X and Y by a combination of K local affine transformations and is adapted to solving inversion regression problems in a Bayesian framework. We refer the reader to
|
|
|
[Deleforge et al 2015](https://link.springer.com/article/10.1007/s11222-014-9461-5) and [Kugler et al 2022](https://link.springer.com/article/10.1007/s11222-021-10019-5) for a definition and detailed description of the model.
|
|
|
|
|
|
The local transformation function $`\tau_k`$ from X to Y is given by:
|
|
|
The mapping from X to Y is assumed to satisfy the following relationship, where all parameters have to be estimated to fit the learning data:
|
|
|
|
|
|
```math
|
|
|
Y=\sum_{k=1}^{K} I(Z=k)\left(A_{k} X+b_{k}+E_{k}\right)
|
|
|
```
|
|
|
|
|
|
where $`A_{k} \in R^{D \times L}`$ and $`b_{k} \in R^{D}`$ define the transformation $`\tau_{k}`$, $`Z`$ is a latent variable ($`Z=k`$ if and only if Y is the image of X by the transformation $`\tau_{k}`$), and $`I`$ is the indicator function. $`E_{k} \in R^{D}`$ represents both the noise errors in the observations and the transformation errors.
|
|
|
where $`A_{k} \in R^{D \times L}`$ and $`b_{k} \in R^{D}`$ define the local affine transformation $`\tau_{k}`$, $`Z`$ is a latent variable ($`Z=k`$ if and only if Y is the image of X by the transformation $`\tau_{k}`$), and $`I`$ is the indicator function. $`E_{k} \in R^{D}`$ is a random variable representing both the error coming from the observation noise and from the affine transformation approximation.
|
|
|
|
|
|
Under the assumption that $`E_{k}`$ is a centered Gaussian variable with a covariance matrix $`\Sigma_{k} \in R^{D \times D}`$ which does not depend on $`X`$ and $`Y`$, we obtain :
|
|
|
|
... | ... | @@ -81,7 +82,7 @@ Y_{k} &=\frac{1}{\sqrt{r_{k}}}\left[\sqrt{r_{1 k}}\left(y_{1}-\overline{y_{k}}\r |
|
|
\end{aligned}
|
|
|
```
|
|
|
|
|
|
This training algorithm ("GLLiM-EM method") is recommended if the covariance matrices $`\Gamma_{k}\left(\operatorname{resp} \Sigma_{k}\right)`$ are diagonal or isotropic matrices (same variance imposed for all the variables). It is very sensitive to the initialisation of the parameter vector $`\theta`$. So, in the case where no constraint is available, it is advised to use the joint Gaussian Mixtures Model (GMM) model equivalent to the GLLiM model (Deleforge et al., 2014) to carry out the training phase ("GMM-EM method"). This equivalence is very interesting in implementation because it makes it possible to use existing libraries of the EM algorithm on the GMMs. The idea is to transform the GLLiM model into GMM, train on the GMM with the EM algorithm and then return to the GLLiM model. Note that the GMM-EM method is itself initialised by an iterative kmeans algorithm.
|
|
|
This training algorithm ("GLLiM-EM method") is recommended if the covariance matrices $`\Gamma_{k}\left(\operatorname{resp} \Sigma_{k}\right)`$ are diagonal or isotropic matrices (same variance imposed for all the variables). It is very sensitive to the initialisation of the parameter vector $`\theta`$. So, in the case where no constraint is available, it is advised to use the joint Gaussian Mixtures Model (GMM) model equivalent to the GLLiM model [Deleforge et al 2015](https://link.springer.com/article/10.1007/s11222-014-9461-5) to carry out the training phase ("GMM-EM method"). This equivalence is very interesting in implementation because it makes it possible to use existing libraries of the EM algorithm on the GMMs. The idea is to transform the GLLiM model into GMM, train on the GMM with the EM algorithm and then return to the GLLiM model. Note that the GMM-EM method is itself initialised by an iterative kmeans algorithm.
|
|
|
|
|
|
##### Initialisation step
|
|
|
|
... | ... | |