-
hhakim authored
- Fix a few PEP warnings in pyfaust.py (a lot remain but they're not critical). - Mention in TODOLIST.developer that we rather use gitlab issues (the file should be deleted later).
hhakim authored- Fix a few PEP warnings in pyfaust.py (a lot remain but they're not critical). - Mention in TODOLIST.developer that we rather use gitlab issues (the file should be deleted later).
TODOLIST.developer 9.75 KiB
##########################################################################
######## TODOLIST for developer ##########
######## FAuST Toolbox ##########
######## Flexible Approximate Multi-Layer Sparse Transform ##########
##########################################################################
NOTE (04/2019): this TODOLIST is deprecated, look at gitlab issues.
##########################################################################
General purpose:
The FAuST toolbox contains a C++ code implementing a general framework
designed to factorize matrices of interest into multiple sparse factors.
It contains a template CPU/GPU C++ code and a Matlab wrapper.
The algorithms implemented here are described in details in [1]- Le Magoarou
For more information on the FAuST Project, please visit the website of the
project: <http://faust.inria.fr>
##########################################################################
This file presents the current state of the FAuST toolbox on the svn repository
and what could be made by future developper.
[Across this doc you'll read in brackets some tmp remarks from Hakim, the current developer on Faust (04/2018)]
Here are the different topics presented in this documents :
##########################
I -C++ code :
##########################
I-1) the folder src/faust_linear_operator/
I-2) the folder src/algorithm
##########################
II - Wrappers :
##########################
II-1) Matlab wrapper
II-2) Python wrapper
II-3) Commandline wrapper
##########################
III - TEST :
##########################
III-1) test with CTest
III-2) test with CDash on Virtual Machine
##########################
I -C++ code :
##########################
In the folder src/ :
The C++ code is split in 2 part :
--> I-1) the folder src/faust_linear_operator/ is where
all the different matrix type are stored,
implemented in CPU and GPU.
-To presents the different type of matrices,
you have :
-LinearOperator : class which is the mother class of all the different LinearOperator.
| that can make a multiplication.
|
|-> -MatGeneric : class inherits from LinearOperator
| |
| |-> - MatDense : (full-storage matrices )
| |-> - MatSparse : (sparse matrices in the Compressed Row Storage).
| This 2 classes just wrapped class from Eigen library (http://eigen.tuxfamily.org/index.php?title=Main_Page)
| and inherits from MatGeneric.
|
|-> -Transform : class inherits from LinearOperator and represents the Flexible Approximate Multi-layer Sparse Transform,
| it is a list of MatGeneric (std::vector<MatGeneric*>) representing its factors.Currently, a factor of a Faust can be
| sparse matrix or a dense matrix.
|[-> TransformHelper class was added to wrap Transform class and ease the use of Transform objects from wrappers,
especially to enhance memory management for different Faust/Must objects relying on the same background Transform object (typically for a Faust and its tranpose, etc.).
See commit c775e359 for further details (Hakim)]
All this different matrix class are templated in Device (CPU/GPU) and scalar type.
They support scalar type such as :
- double / float (real scalar)
-std::complex<double> / std::complex<float> (complex scalar)
**** TODO **** : - new sublass of MatGeneric can be implemented that are efficient for multiplication
(Block-Diagonal matrices, Permutation matrices). This allowing the factors of a Faust to be
more faster for multiplication if they have the required structure (Block-diagonal or permutation matrix)
- The compatibility with complex scalar is ok.
Currently, you can multiply the transposed of a Trasnform by a MatDense.
But it could be interesting to multiply by the conjugate or the transconjugate of a Transform [Hakim: it's done for TransformHelper class used in wrappers, so it's ok].
- WARNING : the GPU matrices are not well tested ... and no documentation is available on the structure of the code...
- library Openblas : when the C++ library named on Openblas is active (cmake configuration BUILD_OPENBLAS is ON),
the following multiplication template function cblas_gemv<SCALAR> and cblas_gemm<SCALAR> are not compatible with complex scalar.
So, they will raised an error when they are called :
void cblas_gemv<std::complex<double> > ,
void cblas_gemv<std::complex<float> > ,
void cblas_gemm<std::complex<double> > ,
void cblas_gemm<std::complex<float> > ,
--> I-2) the folder src/algorithm is where all the factorization algorithm
(hierarchical_fact and palm4MSA) are implemented.
As the matrix, this 2 algorithms are templated with scalar and Device.
The code is not duplicated from the point of view GPU/CPU.
But these algorithm only support real scalar matrices (i.e float/double).
--> This folder contains two subfolder :
- constraint/ is where all the projection/proximal P operator are implemented.
For example, a P operator can project a matrix on the k-sparse matrices space (matrix with only k non-zeros coefficients).
- factorization/ is where factorization algorithm (hierarchical_fact and palmMSA) are.
They transform a matrix into a Faust (i.e a list factors)
**** TODO **** :
- make these algorithm compatible with complex scalar (std::complex<double> / std::complex<float>).
- the input parameter of the algorithm may be not very intuitive, so a work to make these parameter
more friendly-used.
##########################
II - Wrappers :
##########################
--> II-1) Matlab wrapper make the C++ code available from Matlab environment.
This matlab wrapper is located in the folder wrapper/matlab/ .
It supplies :
- the Faust Matlab class which is the equivalent of the C++ class Transform.
It is compatible with complex scalar and real scalar.
- the factorization algorithm (hierarchical_fact and palm4MSA). They only support
real scalar matrices and only the CPU matrices are interface with Matlab.
- and little demos are available (WARNING : they are located in misc/demo/ folder) :
- Brain Source localization illustrating the speed-up of using a Faust,
- Hadamard matrix factorization
- FFT where the use of a Faust is compared to FFT algorithm for multiplication with the Fourier matrix.
- Quick_start to understand the use of the Matlab Faust class
- runtime comparison that compare Faust multiplication with dense multiplication for various configuration.
(For more information on the Matlab wrapper, see wrapper/matlab/README.developper)
**** TODO **** :
--> Make the Faust Matlab class operation conjugate and transconjugate are not implemented.
So, it will be easy to make, this operation implementation should be very similar to
the transpose implementation.[DONE (Hakim)]
--> Make the Matlab factorization algorithm compatible with complex matrices and GPU matrices.
--> II-2) Python wrapper make the C++ code available from Python environment.
This matlab wrapper is located in the folder wrapper/python/ .
It supplies :
--> the FaustPy Python module.
This module supplies the Faust class wich is the equivalent of the C++ class Transform.
It is only compatible with real scalar [Not anymore, now complex scalar are supported (Hakim)].
(For more information on the Matlab wrapper, see wrapper/python/README.developper)
**** TODO **** :
--> Make the FaustPy.faust class compatible with complex scalar [DONE (Hakim)].
--> Develop the inferface with the factoziration algorithm.
--> Implements some Python demos similar to the Matlab Demo (Brain Source localization ...)
--> II-3) Commandline wrapper, it delivered the commandline interface to factorized a matrix
from xml configuration file and text file for storing the matrix.
to build it use the CMake configuration variable (BUILD_READ_XML)
**** TODO **** :
This wrapper is not used anymore. Currently, it will not compile.
Maybe, the question of it's accuracy is important in order to delete it from the project.
##########################
III - TEST :
##########################
III-1) test with CTest
all the test are defined in the directory misc/test/.
(For more information on this, see misc/test/README.developper)
[The script tests located in misc/CTest_nightly directory are not relevent anymore because since the gitlab migration
of the project we rely on gitlab continuous integration and use .gitlab-ci.yml script to launch tests,
but cdash and ci.inria.fr are still used (Hakim)]
III-2) test with CDash on VM
Daily, all the test are launched on virtual machine
and the result of this test are send to a CDash server.
(For more information on this, see wrapper/Ctest_nightly/README.txt)
Current state of the Virtual Machines :
- LINUX (ubuntu) : all the test passed on the ubuntu's machine.
Exception to the test using Openblas and complex scalar.
they throw an error of the style :
cblas_gemv<std::complex<double>> : not yet implemented
May be some Python test won't succeed.
It 's seems only a problem due to incompatible version
of Python module (scipy ...)
- MAC OS X : It's the same as the LINUX Machines.
- Windows : all the test on Windows VM currently won't work...
There are problem into the step of CMake configuration.
[Some progress have been made to make windows version build and run (Hakim)]