Commit 02160c27 authored by MARMORET Axel's avatar MARMORET Axel
Browse files

Comments update.

parent 40d0a0b9
# nn-fac: Nonnegative Factorization techniques toolbox
Python code for computing some Nonnegative Factorization, using an accelerated version of Hierarchical Alternating Least Squares algorithm (HALS) with resolution of Nonnegative Least Squares problem (NNLS) [1].
Python code for computing some Nonnegative Factorizations, using either an accelerated version of Hierarchical Alternating Least Squares algorithm (HALS) with resolution of Nonnegative Least Squares problem (NNLS) [1] for factorizations subject to the minimization of the Euclidean/Frobenius norm, or the Multiplicative Update [2,3] for factors, by minimizing the $\beta$-divergence [3]..
This work has been done during my Research Master's (SIF, master.irisa.fr) internship in PANAMA team at IRISA/Inria Rennes, under the direction of BERTIN Nancy and COHEN Jeremy.
......@@ -25,38 +25,49 @@ Don't hesitate to reach the author in case of problem. Comments are welcomed!
### NNLS
This toolbox contains a NNLS resolution algorithm, developed as described in [1]. This code is based on COHEN Jeremy python code adaptation of GILLIS Nicolas MatLab code.
### MU
This toolbox contains a MU resolution algorithm, developed as described in [3] for NMF, also extended for tensorial factorizations. This code is based on an internship of Florian VOORWINDEN and has been improved with COHEN Jeremy and Valentin LEPLAT.
This toolbox also contains 4 factorization methods:
### NMF
Nonnegative Matrix Factorization [2] - Factorization of a nonnegative matrix X in two nonnegative matrices W and H, where WH approach X.
This is solved by minimizing the Frobenius norm between both matrices X and WH by NNLS.
In the hals condition, this is solved by minimizing the Frobenius norm between both matrices X and WH by NNLS.
In the mu condition, this is solved by minimizing the $\beta$-divergence [3] between both matrices X and WH by Multiplicative Updates.
### NTF - Nonnegative PARAFAC
Nonnegative Tensor Factorization, also called Nonnegative PARAFAC decomposition. PARAFAC decomposition consists in factorizing a tensor T in a sum of rank-one tensors [3]. By concatenating the vectors along each mode of this sum, we obtain as much factors as the number of modes of the tensor [4]. This algorithm returns these factors.
Nonnegative Tensor Factorization, also called Nonnegative PARAFAC decomposition. PARAFAC decomposition consists in factorizing a tensor T in a sum of rank-one tensors [4]. By concatenating the vectors along each mode of this sum, we obtain as much factors as the number of modes of the tensor [5]. This algorithm returns these factors.
In the hals condition, this factorization is computed as an ALS algorithm, described in [6], solved with NNLS, and using the toolbox Tensorly [7]. It returns the nonnegative factors of a nonnegative tensor T.
This factorization is computed as an ALS algorithm, described in [5], solved with NNLS, and using the toolbox Tensorly [6]. It returns the nonnegative factors of a nonnegative tensor T.
In the mu condition, this factorization is computed as NMF subproblems, described in [3], solved with Multiplicative Update, and using the toolbox Tensorly [7]. It returns the nonnegative factors of a nonnegative tensor T subject to the minimization of the $\beta$-divergence.
### Nonnegative PARAFAC2
Nonnegative Tensor Factorization admitting variability over a factor [7]. More precisely, this implemented version is based on a flexible coupling approach [8], where the coupling is enforced by a penalty term.
Nonnegative Tensor Factorization admitting variability over a factor [8]. More precisely, this implemented version is based on a flexible coupling approach [9], where the coupling is enforced by a penalty term.
### NTD - Nonnegative Tucker Decomposition
Nonnegative Tucker Decomposition, which consists in factorizing a tensor T in factors (one per mode) and a core tensor, generally of smaller dimensions than T, which links linearly all factors [5]. This algorithm returns these factors and this core tensor.
Nonnegative Tucker Decomposition, which consists in factorizing a tensor T in factors (one per mode) and a core tensor, generally of smaller dimensions than T, which links linearly all factors [6]. This algorithm returns these factors and this core tensor.
This factorization is computed as an ALS algorithm, described in [5], solved with NNLS, and using the toolbox Tensorly [6]. It also uses a gradient update rule for the core.
In the hals condition, this factorization is computed as an ALS algorithm, described in [6], solved with NNLS, and using the toolbox Tensorly [7]. It also uses a gradient update rule for the core.
In the mu condition, this factorization is computed as NMF subproblems (computationally optimized with tensor contractions), described in [3], solved with the Multiplicative Update [3], and using the toolbox Tensorly [7].
## References
[1] N. Gillis and F. Glineur, "Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization," Neural Computation 24 (4): 1085-1105, 2012.
[2] D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, no. 6755, p. 788, 1999.
[3] R. A Harshman et al. "Foundations of the PARAFAC procedure: Models and conditions for an" explanatory" multimodal factor analysis," 1970.
[3] Févotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the β-divergence. Neural computation, 23(9), 2421-2456.
[4] R. A Harshman et al. "Foundations of the PARAFAC procedure: Models and conditions for an" explanatory" multimodal factor analysis," 1970.
[4] J. E. Cohen, and N. Gillis, "Dictionary-based tensor canonical polyadic decomposition," IEEE Transactions on Signal Processing, 66(7), 1876-1889, 2017.
[5] J. E. Cohen, and N. Gillis, "Dictionary-based tensor canonical polyadic decomposition," IEEE Transactions on Signal Processing, 66(7), 1876-1889, 2017.
[5] T. G. Kolda, and B. W. Bader, "Tensor decompositions and applications," SIAM review, 51(3), 455-500, 2009.
[6] T. G. Kolda, and B. W. Bader, "Tensor decompositions and applications," SIAM review, 51(3), 455-500, 2009.
[6] J. Kossaifi, Y. Panagakis, A. Anandkumar and M. Pantic, "TensorLy: Tensor Learning in Python," Journal of Machine Learning Research (JMLR), volume 20, number 26, 2019.
[7] J. Kossaifi, Y. Panagakis, A. Anandkumar and M. Pantic, "TensorLy: Tensor Learning in Python," Journal of Machine Learning Research (JMLR), volume 20, number 26, 2019.
[7] R. A Harshman, "PARAFAC2: Mathematical and technical notes," UCLA working papers in phonetics 22.3044, 1972.
[8] R. A Harshman, "PARAFAC2: Mathematical and technical notes," UCLA working papers in phonetics 22.3044, 1972.
[8] J. E Cohen and R. Bro, "Nonnegative PARAFAC2: a flexible coupling approach," International Conference on Latent Variable Analysis and Signal Separation. Springer. 2018.
[9] J. E Cohen and R. Bro, "Nonnegative PARAFAC2: a flexible coupling approach," International Conference on Latent Variable Analysis and Signal Separation. Springer. 2018.
......@@ -15,9 +15,9 @@ import nn_fac.errors as err
def mu_betadivmin(U, V, M, beta):
"""
============================================================
=====================================================
Beta-Divergence NMF solved with Multiplicative Update
============================================================
=====================================================
Computes an approximate solution of a beta-NMF
[3] with the Multiplicative Update rule [2,3].
......
......@@ -32,21 +32,25 @@ def nmf(data, rank, init = "random", U_0 = None, V_0 = None, n_iter_max=100, tol
The objective function is:
||M - UV||_Fro^2
d(M - UV)_{\beta}
+ sparsity_coefficients[0] * (\sum\limits_{j = 0}^{r}||U[:,k]||_1)
+ sparsity_coefficients[1] * (\sum\limits_{j = 0}^{r}||V[k,:]||_1)
With:
||A||_Fro^2 = \sum_{i,j} A_{ij}^2 (Frobenius norm)
d(A)_{\beta} the elementwise $\beta$-divergence,
||a||_1 = \sum_{i} abs(a_{i}) (Elementwise L1 norm)
The objective function is minimized by fixing alternatively
one of both factors U and V and optimizing on the other one,
the problem being reduced to a Nonnegative Least Squares problem.
one of both factors U and V and optimizing on the other one.
More precisely, the chosen optimization algorithm is the HALS [1],
which updates each factor columnwise, fixing every other columns.
which updates each factor columnwise, fixing every other columns,
each subproblem being reduced to a Nonnegative Least Squares problem
if the update_rule is "hals",
or by using the Multiplicative Update [4,5] on each factor
if the update_rule is "mu".
The MU is minimizes the $\beta$-divergence,
whereas the HALS minimizes the Frobenius norm only.
Parameters
----------
......@@ -81,6 +85,20 @@ def nmf(data, rank, init = "random", U_0 = None, V_0 = None, n_iter_max=100, tol
Between two succesive iterations, if the difference between
both cost function values is below this threshold, the algorithm stops.
Default: 1e-8
update_rule: string "hals" | "mu"
The chosen update rule.
HALS performs optimization with the euclidean norm,
MU performs the optimization using the $\beta$-divergence loss,
which generalizes the Euclidean norm, and the Kullback-Leibler and
Itakura-Saito divergences.
The chosen beta-divergence is specified with the parameter `beta`.
Default: "hals"
beta: float
The beta parameter for the beta-divergence.
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
Default: 2
sparsity_coefficients: List of float (two)
The sparsity coefficients on U and V respectively.
If set to None, the algorithm is computed without sparsity
......@@ -138,6 +156,14 @@ def nmf(data, rank, init = "random", U_0 = None, V_0 = None, n_iter_max=100, tol
[3]: B. Zupan et al. "Nimfa: A python library for nonnegative matrix
factorization", Journal of Machine Learning Research 13.Mar (2012),
pp. 849{853.
[4] Févotte, C., & Idier, J. (2011).
Algorithms for nonnegative matrix factorization with the β-divergence.
Neural computation, 23(9), 2421-2456.
[5] Lee, D. D., & Seung, H. S. (1999).
Learning the parts of objects by non-negative matrix factorization.
Nature, 401(6755), 788-791.
"""
if init.lower() == "random":
k, n = data.shape
......@@ -167,6 +193,7 @@ def compute_nmf(data, rank, U_in, V_in, n_iter_max=100, tol=1e-8,
"""
Computation of a Nonnegative matrix factorization via
hierarchical alternating least squares (HALS) [1],
or Multiplicative Update (MU) [2],
with U_in and V_in as initialization.
Parameters
......@@ -187,6 +214,20 @@ def compute_nmf(data, rank, U_in, V_in, n_iter_max=100, tol=1e-8,
Between two iterations, if the difference between
both cost function values is below this threshold, the algorithm stops.
Default: 1e-8
update_rule: string "hals" | "mu"
The chosen update rule.
HALS performs optimization with the euclidean norm,
MU performs the optimization using the $\beta$-divergence loss,
which generalizes the Euclidean norm, and the Kullback-Leibler and
Itakura-Saito divergences.
The chosen beta-divergence is specified with the parameter `beta`.
Default: "hals"
beta: float
The beta parameter for the beta-divergence.
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
Default: 2
sparsity_coefficients: List of float (two)
The sparsity coefficients on U and V respectively.
If set to None, the algorithm is computed without sparsity
......@@ -222,6 +263,10 @@ def compute_nmf(data, rank, U_in, V_in, n_iter_max=100, tol=1e-8,
[1]: N. Gillis and F. Glineur, Accelerated Multiplicative Updates and
Hierarchical ALS Algorithms for Nonnegative Matrix Factorization,
Neural Computation 24 (4): 1085-1105, 2012.
[2] Févotte, C., & Idier, J. (2011).
Algorithms for nonnegative matrix factorization with the β-divergence.
Neural computation, 23(9), 2421-2456.
"""
# initialisation
U = U_in.copy()
......@@ -276,7 +321,10 @@ def one_nmf_step(data, rank, U_in, V_in, norm_data, update_rule, beta,
sparsity_coefficients, fixed_modes, normalize):
"""
One pass of updates for each factor in NMF
Update the factors by solving a nonnegative least squares problem per mode.
Update the factors by solving a nonnegative least squares problem per mode
if the update_rule is "hals",
or by using the Multiplicative Update on each factor
if the update_rule is "mu".
Parameters
----------
......@@ -290,6 +338,20 @@ def one_nmf_step(data, rank, U_in, V_in, norm_data, update_rule, beta,
Initial V factor, of size r*n
norm_data: float
The Frobenius norm of the input matrix (data)
update_rule: string "hals" | "mu"
The chosen update rule.
HALS performs optimization with the euclidean norm,
MU performs the optimization using the $\beta$-divergence loss,
which generalizes the Euclidean norm, and the Kullback-Leibler and
Itakura-Saito divergences.
The chosen beta-divergence is specified with the parameter `beta`.
Default: "hals"
beta: float
The beta parameter for the beta-divergence.
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
Default: 2
sparsity_coefficients: List of float (two)
The sparsity coefficients on U and V respectively.
If set to None, the algorithm is computed without sparsity
......
......@@ -390,7 +390,7 @@ def one_ntd_step(tensor, ranks, in_core, in_factors, norm_tensor,
alpha=0.5, delta=0.01):
"""
One pass of Hierarchical Alternating Least Squares update along all modes,
and hals or gradient update on the core (depends on hals parameter),
and gradient update on the core,
which decreases reconstruction error in Nonnegative Tucker Decomposition.
Update the factors by solving a least squares problem per mode, as described in [1].
......@@ -592,10 +592,150 @@ def one_ntd_step(tensor, ranks, in_core, in_factors, norm_tensor,
######################### Temporary, to test mu and not break everything (should be merged to ntd in the end)
######################### Temporary, to test mu and not break everything (should be merged to ntd in the end, but there is still development on this function)
def ntd_mu(tensor, ranks, init = "random", core_0 = None, factors_0 = [], n_iter_max=1000, tol=1e-6,
sparsity_coefficients = [], fixed_modes = [], normalize = [], mode_core_norm = None, beta = 2,
beta = 2, sparsity_coefficients = [], fixed_modes = [], normalize = [], mode_core_norm = None,
verbose=False, return_costs=False, deterministic=False):
"""
=============================================================================
Nonnegative Tucker Decomposition (NTD) minimizing the $\beta$-divergence loss
=============================================================================
Factorization of a tensor T in nonnegative matrices,
linked by a nonnegative core tensor, of dimensions equal to the ranks
(in general smaller than the tensor).
See more details about the NTD in [1].
For example, in the third-order case, resolution of:
T \approx (W \otimes H \otimes Q) G
In this example, W, H and Q are the factors, one per mode, and G is the core tensor.
W is of size T.shape[0] * ranks[0],
H is of size T.shape[1] * ranks[1],
Q is of size T.shape[2] * ranks[2],
G is of size ranks[0] * ranks[1] * ranks[2].
In this context, \approx means that the tensor product (W \otimes H \otimes Q) G
is meant to be as close as possible to T, with respect to some distance or divergence function.
In `ntd_mu`, this divergence is the $\beta$-divergence loss [2].
This divergence function generalizes some other divergences such as the Euclidean norm,
the Kullback-Leibler divergence and the Itakura-Saito divergence.
This approximation is obtained with an optimization algorithm, and more specfically,
by using the Multiplicative Update as defined for NMF in [3] and redeveloped in [2].
This algorithm will be described in a future publication
(TODO: update this comments at submission/publication time.)
Tensors are manipulated with the tensorly toolbox [4].
In tensorly and in our convention, tensors are unfolded and treated as described in [5].
Parameters
----------
tensor: tensorly tensor
The nonnegative tensor T, to factorize
ranks: list of integers
The ranks for each factor of the decomposition
init: "random" | "tucker" | "custom" |
- If set to random:
Initializes with random factors of the correct size.
The randomization is the uniform distribution in [0,1),
which is the default from numpy random.
- If set to tucker:
Resolve a tucker decomposition of the tensor T (by HOSVD) and
initializes the factors and the core as this resolution, clipped to be nonnegative.
The tucker decomposition is performed with tensorly [3].
- If set to custom:
core_0 and factors_0 (see below) will be used for the initialization
Default: random
core_0: None or tensor of nonnegative floats
A custom initialization of the core, used only in "custom" init mode.
Default: None
factors_0: None or list of array of nonnegative floats
A custom initialization of the factors, used only in "custom" init mode.
Default: None
n_iter_max: integer
The maximal number of iteration before stopping the algorithm
Default: 100
tol: float
Threshold on the improvement in cost function value.
Between two succesive iterations, if the difference between
both cost function values is below this threshold, the algorithm stops.
Default: 1e-6
beta: float
The beta parameter for the beta-divergence.
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
sparsity_coefficients: list of float (as much as the number of modes + 1 for the core)
The sparsity coefficients on each factor and on the core respectively.
If set to None or [], the algorithm is computed without sparsity
Default: []
fixed_modes: list of integers (between 0 and the number of modes + 1 for the core)
Has to be set not to update a factor, taken in the order of modes and lastly on the core.
Default: []
normalize: list of boolean (as much as the number of modes + 1 for the core)
Indicates whether the factors need to be normalized or not.
The normalization is a l_2 normalization on each of the rank components
(For the factors, each column will be normalized, ie each atom of the dimension of the current rank).
Default: []
mode_core_norm: integer or None
The mode on which normalize the core, or None if normalization shouldn't be enforced.
Will only be useful if the last element of the previous "normalise" argument is set to True.
Indexes of the modes start at 0.
Default: None
verbose: boolean
Indicates whether the algorithm prints the successive
normalized cost function values or not
Default: False
return_costs: boolean
Indicates whether the algorithm should return all normalized cost function
values and computation time of each iteration or not
Default: False
deterministic:
Runs the algorithm as a deterministic way, by fixing seed in all possible randomisation,
and optimization techniques in the NNLS, function of the runtime.
This is made to enhance reproducible research, and should be set to True for computation of results.
Returns
-------
core: tensorly tensor
The core tensor linking the factors of the decomposition
factors: numpy #TODO: For tensorly pulling, replace numpy by backend
An array containing all the factors computed with the NTD
cost_fct_vals: list
A list of the normalized cost function values, for every iteration of the algorithm.
toc: list, only if return_errors == True
A list with accumulated time at each iterations
Example
-------
tensor = np.random.rand(80,100,120)
ranks = [10,20,15]
core, factors = NTD.ntd(tensor, ranks = ranks, init = "tucker", verbose = True, hals = False,
sparsity_coefficients = [None, None, None, None], normalize = [True, True, False, True])
References
----------
[1] Tamara G Kolda and Brett W Bader. "Tensor decompositions and applications",
SIAM review 51.3 (2009), pp. 455{500.
[2] Févotte, C., & Idier, J. (2011).
Algorithms for nonnegative matrix factorization with the β-divergence.
Neural computation, 23(9), 2421-2456.
[3] Lee, D. D., & Seung, H. S. (1999).
Learning the parts of objects by non-negative matrix factorization.
Nature, 401(6755), 788-791.
[4] J. Kossai et al. "TensorLy: Tensor Learning in Python",
arxiv preprint (2018)
[5] Jeremy E Cohen. "About notations in multiway array processing",
arXiv preprint arXiv:1511.01306, (2015).
"""
factors = []
nb_modes = len(tensor.shape)
......@@ -660,6 +800,10 @@ def ntd_mu(tensor, ranks, init = "random", core_0 = None, factors_0 = [], n_iter
def compute_ntd_mu(tensor_in, ranks, core_in, factors_in, n_iter_max=100, tol=1e-6,
sparsity_coefficients = [], fixed_modes = [], normalize = [], beta = 2, mode_core_norm=None,
verbose=False, return_costs=False, deterministic=False):
"""
Computes the NTD, with parameters initialized in the `ntd_mu` function.
"""
# initialisation - store the input varaibles
core = core_in.copy()
......@@ -731,6 +875,10 @@ def compute_ntd_mu(tensor_in, ranks, core_in, factors_in, n_iter_max=100, tol=1e
def one_ntd_step_mu(tensor, ranks, in_core, in_factors, beta, norm_tensor,
fixed_modes, normalize, mode_core_norm):
"""
One step of Multiplicative Uodate applied for every mode of the tensor
and on the core.
"""
# Copy
core = in_core.copy()
factors = in_factors.copy()
......
......@@ -42,17 +42,19 @@ def ntf(tensor, rank, init = "random", factors_0 = [], n_iter_max=100, tol=1e-8,
The optimization problem, for M_n, the n-th factor, is:
min_{M_n}||T - M_n (khatri-rao(M_l))^T||_Fro^2
min_{M_n} d(T - M_n (khatri-rao(M_l))^T)_{\beta}
+ 2 * sparsity_coefficients[n] * (\sum\limits_{j = 0}^{r}||V[k,:]||_1)
With:
l the index of all the modes except the n-th one
||A||_Fro^2 = \sum_{i,j} A_{ij}^2 (Frobenius norm)
d(A)_{\beta} the elementwise beta-divergence,
||a||_1 = \sum_{i} abs(a_{i}) (Elementwise L1 norm)
More precisely, the chosen optimization algorithm is the HALS [1],
which updates each factor columnwise, fixing every other columns.
More precisely, the chosen optimization algorithm is either the HALS [1],
which updates each factor columnwise, fixing every other columns,
and optimizes factors according to the euclidean norm,
or the MU [5,6], which optimizes the factors according to the $\beta$-divergence loss.
Parameters
----------
......@@ -84,6 +86,19 @@ def ntf(tensor, rank, init = "random", factors_0 = [], n_iter_max=100, tol=1e-8,
Between two succesive iterations, if the difference between
both cost function values is below this threshold, the algorithm stops.
Default: 1e-8
update_rule: string "hals" | "mu"
The chosen update rule.
HALS performs optimization with the euclidean norm,
MU performs the optimization using the $\beta$-divergence loss,
which generalizes the Euclidean norm, and the Kullback-Leibler and
Itakura-Saito divergences.
The chosen beta-divergence is specified with the parameter `beta`.
beta: float
The beta parameter for the beta-divergence.
Only useful if update_rule is set to "mu".
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
sparsity_coefficients: array of float (as much as the number of modes)
The sparsity coefficients on U and V respectively.
If set to None, the algorithm is computed without sparsity
......@@ -147,6 +162,14 @@ def ntf(tensor, rank, init = "random", factors_0 = [], n_iter_max=100, tol=1e-8,
[4] J. Kossai et al. "TensorLy: Tensor Learning in Python",
arxiv preprint (2018)
[5] Févotte, C., & Idier, J. (2011).
Algorithms for nonnegative matrix factorization with the β-divergence.
Neural computation, 23(9), 2421-2456.
[6] Lee, D. D., & Seung, H. S. (1999).
Learning the parts of objects by non-negative matrix factorization.
Nature, 401(6755), 788-791.
- Tamara G Kolda and Brett W Bader. "Tensor decompositions and applications",
SIAM review 51.3 (2009), pp. 455{500.
......@@ -195,6 +218,7 @@ def compute_ntf(tensor_in, rank, factors_in, n_iter_max=100, tol=1e-8,
"""
Computation of a Nonnegative matrix factorization via
hierarchical alternating least squares (HALS) [1],
or Multiplicative Update (MU) [2],
with factors_in as initialization.
Parameters
......@@ -213,6 +237,18 @@ def compute_ntf(tensor_in, rank, factors_in, n_iter_max=100, tol=1e-8,
Between two iterations, if the difference between
both cost function values is below this threshold, the algorithm stops.
Default: 1e-8
update_rule: string "hals" | "mu"
The chosen update rule.
HALS performs optimization with the euclidean norm,
MU performs the optimization using the $\beta$-divergence loss,
which generalizes the Euclidean norm, and the Kullback-Leibler and
Itakura-Saito divergences.
The chosen beta-divergence is specified with the parameter `beta`.
beta: float
The beta parameter for the beta-divergence.
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
sparsity_coefficients: List of float (as much as the number of modes)
The sparsity coefficients on U and V respectively.
If set to None, the algorithm is computed without sparsity
......@@ -248,6 +284,10 @@ def compute_ntf(tensor_in, rank, factors_in, n_iter_max=100, tol=1e-8,
[1]: N. Gillis and F. Glineur, Accelerated Multiplicative Updates and
Hierarchical ALS Algorithms for Nonnegative Matrix Factorization,
Neural Computation 24 (4): 1085-1105, 2012.
[2] Févotte, C., & Idier, J. (2011).
Algorithms for nonnegative matrix factorization with the β-divergence.
Neural computation, 23(9), 2421-2456.
- Tamara G Kolda and Brett W Bader. "Tensor decompositions and applications",
SIAM review 51.3 (2009), pp. 455{500.
......@@ -322,13 +362,10 @@ def one_ntf_step(unfolded_tensors, rank, in_factors, norm_tensor, update_rule, b
"""
One pass of Hierarchical Alternating Least Squares update along all modes
Update the factors by solving a least squares problem per mode, as described in [1].
Update the factors by solving a least squares problem per mode (in hals), as described in [1],
or using the Multiplicative Update for the entire factors [2].
Note that the unfolding order is the one described in [2], which is different from [1].
This function is strictly superior to a least squares solver ran on the
matricized problems min_X ||Y - AX||_F^2 since A is structured as a
Kronecker product of other factors.
Note that the unfolding order is the one described in [3], which is different from [1].
Parameters
----------
......@@ -343,6 +380,18 @@ def one_ntf_step(unfolded_tensors, rank, in_factors, norm_tensor, update_rule, b
Rank of the decomposition.
norm_tensor : float
The Frobenius norm of the input tensor
update_rule: string "hals" | "mu"
The chosen update rule.
HALS performs optimization with the euclidean norm,
MU performs the optimization using the $\beta$-divergence loss,
which generalizes the Euclidean norm, and the Kullback-Leibler and
Itakura-Saito divergences.
The chosen beta-divergence is specified with the parameter `beta`.
beta: float
The beta parameter for the beta-divergence.
2 - Euclidean norm
1 - Kullback-Leibler divergence
0 - Itakura-Saito divergence
sparsity_coefficients : List of floats
sparsity coefficients for every mode.
fixed_modes : List of integers
......@@ -373,8 +422,12 @@ def one_ntf_step(unfolded_tensors, rank, in_factors, norm_tensor, update_rule, b
----------
[1] Tamara G Kolda and Brett W Bader. "Tensor decompositions and applications",
SIAM review 51.3 (2009), pp. 455{500.
[2] Févotte, C., & Idier, J. (2011).
Algorithms for nonnegative matrix factorization with the β-divergence.
Neural computation, 23(9), 2421-2456.
[2] Jeremy E Cohen. "About notations in multiway array processing",
[3] Jeremy E Cohen. "About notations in multiway array processing",
arXiv preprint arXiv:1511.01306, (2015).
"""
......
......@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup(
name="nn_fac",
version="0.1.2",
version="0.2.0",
author="Marmoret Axel",
author_email="axel.marmoret@irisa.fr",
description="Nonnegative factorization toolbox.",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment