FMR Documentation
C++ API for the Fast Methods for Randomized numerical linear algebra
Inria
About
This package is a subset of the master branch of FMR, that contains only classes over LaPACK and Chameleon. Authors: P. Blanchard, O. Coulaud (Olivier.Coulaud@inria.fr)
Developpers F. Pruvost R. Peressoni
Fast and accurate Methods for Randomized numerical linear algebra. This project provides routines for performing low-rank matrix approximations based on randomized techniques:
- Random projection based LRA: Randomized {SVD, Cholesky, ID, ...}
Please refer to
- Pierre Blanchard, Olivier Coulaud, Eric Darve, Alain Franc. FMR: Fast randomized algorithms for covariance matrix computations. Platform for Advanced Scientific Computing (PASC), Jun 2016, Lausanne, Switzerland. 2016. (https://hal.inria.fr/hal-01334747v1)
- Pierre Blanchard, Olivier Coulaud, Eric Darve. Fast hierarchical algorithms for generating Gaussian random fields. [Research Report] 8811, Inria Bordeaux Sud-Ouest. 2015. ⟨https://hal.inria.fr/hal-01228519v2⟩
- Pierre Blanchard. Fast hierarchical algorithms for the low-rank approximation of matrices, with applications to materials physics, geostatistics and data analysis. Université de Bordeaux, 2017. ⟨https://hal.inria.fr/tel-01534930⟩
and for details on algorithms
What's inside ?
-
CMakeModules/
: Contains cmake scripts to find libraries installed on your machine -
include/
:-
fmr/
: Contains all th directories of the header only library-
Algorithms
: low-rank approximation algorithms such as truncated SVD, random projection methods, ... -
MatrixWrappers
: Wrapper for matrix -
StandardLRA
: Classical matrix operation or factorization GEMM, QR, SVD -
RandomizedLRA
: Randomized classes for algorithms -
Utils
: Contains scripts to find libraries installed on your machine
-
-
Get and Build FMR
How to get it
To get fmr
git clone --recursive https://gitlab.inria.fr/compose/legacystack/fmr.git
Dependencies
- Blas/Lapack
- zlib and bzip2
- optional
- Chameleon, MPI, StarPU
- MKL random generators
- HDF5
Install FMR
FMR was tested on Linux system and MacosX (10.15) with
- compilers (Intel, clang, GNU)
- Blas/Lapack: openblas and MKL, Chameleon
- zlib and bzip2
- HDF5 1.10 or 1.12
FMR is a header only library, if you don't build the tests.
git clone git@gitlab.inria.fr:compose/legacystack/fmr.git
Create a directory to build and to install FMR with cmake.
```
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX:PATH=/your/installation/path_for_fmr
cmake --build . --config Release --target install
```
Installation examples with dependencies
Docker image
We provide a docker image with FMR installed with Chameleon and MPI. Please refer to this dockerfile for the recipe. The docker image name is registry.gitlab.inria.fr/compose/legacystack/fmr/ubuntu and you can use it as follows:
docker login registry.gitlab.inria.fr -u gitlab+deploy-token-310 -p CLyjjWw_DPxTQF_jmeJf
docker run -it registry.gitlab.inria.fr/compose/legacystack/fmr/ubuntu
Linux Debian/Ubuntu
sudo apt update -y
sudo apt install -y git python3 cmake build-essential gfortran pkg-config libmkl-dev libstarpu-dev libhdf5-mpi-dev zlib1g-dev libbz2-dev
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1
# install chameleon
cd $HOME
git clone --recursive git@gitlab.inria.fr:solverstack/chameleon.git
cd chameleon
export CHAMELEON_ROOT=$PWD/install
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$CHAMELEON_ROOT -DCHAMELEON_USE_MPI=ON -DBLA_VENDOR=Intel10_64lp -DCBLAS_MT=ON -DLAPACKE_MT=ON -DBUILD_SHARED_LIBS=ON
make -j3 install
# install fmr
cd $HOME
git clone git@gitlab.inria.fr:compose/legacystack/fmr.git
cd fmr
export FMR_ROOT=$PWD/install
mkdir build && cd build
# check
cmake .. -DCMAKE_INSTALL_PREFIX=$FMR_ROOT -DFMR_BUILD_TESTS=ON -DFMR_USE_HDF5=ON -DFMR_USE_CHAMELEON=ON -DFMR_USE_MKL_AS_BLAS=ON
make -j5
ctest -V
# install
cmake .. -DFMR_BUILD_TESTS=OFF
make install
In your cmake project use CHAMELEON_ROOT and FMR_ROOT or set CMAKE_PREFIX_PATH with CHAMELEON_ROOT and FMR_ROOT.
MacOSX
Considering git, gcc and cmake are already installed you need to install the following: zlib, bzip2, CBLAS/LAPACKE (example here with OpenBLAS but you can install Intel MKL), StarPU and Chameleon.
brew install automake autoconf libtool zlib bzip2 hwloc pkgconfig openblas openmpi
# use pkg-config .pc files to detect some dependencies
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/Cellar/openblas/0.3.13/lib/pkgconfig/:${PKG_CONFIG_PATH}
# cmake checks blas.pc not openblas.pc
sudo cp /usr/local/Cellar/openblas/0.3.13/lib/pkgconfig/openblas.pc /usr/local/Cellar/openblas/0.3.13/lib/pkgconfig/blas.pc
# install hdf5 parallel openmpi
# download a source tarball here https://www.hdfgroup.org/downloads/hdf5/source-code/
tar xvf hdf5-1.12.0.tar.gz
cd hdf5-1.12.0
CC=/usr/local/bin/mpicc CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure --with-zlib=/usr/local/opt --prefix=/usr/local/ --enable-shared --disable-fortran --disable-static --disable-tests --enable-threadsafe --with-pthread --enable-unsupported --enable-parallel
make -j5
sudo make install
# install starpu
# download a source tarball here https://files.inria.fr/starpu/
tar xvf starpu-1.3.7.tar.gz
cd starpu-1.3.7/
./configure --enable-blas-lib=none --disable-starpufft --disable-mlr --disable-opencl --disable-cuda --disable-hdf5 --disable-fortran --prefix=/usr/local
make -j5
sudo make install
# install chameleon
cd $HOME
git clone --recursive git@gitlab.inria.fr:solverstack/chameleon.git
cd chameleon
export CHAMELEON_ROOT=/usr/local/
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=${CHAMELEON_ROOT} -DCHAMELEON_USE_MPI=ON -DBLA_PREFER_PKGCONFIG=ON -DBUILD_SHARED_LIBS=ON
make -j5
sudo make install
export PKG_CONFIG_PATH=${CHAMELEON_ROOT}/lib/pkgconfig/:${PKG_CONFIG_PATH}
Don't forget to add Chameleon in the PKG Config path in your .basrc
file.
Then follow the install procedure described in the Linux Debian/Ubuntu section
Usefull define to enable some FMR features
These following keywords allow you to access other functionalities or improve performance
-
FMR_HDF5
if you want to fill a matrix from a dataset in ```hdf5`` file -
FMR_USE_MKL_AS_BLAS
if used with Intel MKL -
FMR_MKL_RANDOM
if theMKL
library is used for Blas/Lapack or parallel random generator -
FMR_USE_OPENBLAS_AS_BLAS
if used with OpenBlas -
FMR_CHAMELEON
if you consider Chamelon library (Blas/Lapack) - Mandatory for distributed version of the library -
FMR_USE_MPI
to use if Chameleon enabled
How to use FMR in other project
When FMR is installed, we add files to find FMR as a package through cmake.
These files are located in dir_to_install_fmr/lib/cmake/fmr
So to use FMR in a project, for instance diodon, we add in CMakeLists.txt the
following lines
## Example with a project called diodon
# use for example CMAKE_PREFIX_PATH to give the fmr root install dir.
find_package(FMR CONFIG REQUIRED)
#
# link diodon target to fmr libraries
#
target_link_libraries(diodon INTERFACE fmr::fmr)
Environment variables to be used during execution
The following environment variables can be used to enable some features at runtime:
- FMR_TIME: if set to 1, enables timing of some functions
- FMR_CHAMELEON_NUM_THREADS: number of threads (cpu workers) to be used by chameleon
- FMR_CHAMELEON_NUM_CUDAS: number of gpus (cuda devices) to be used by chameleon
- FMR_CHAMELEON_TILE_SIZE: size nb of tiles (square data sub-blocks nb*nb)
- FMR_CHAMELEON_ASYNC: if set to 1, enables the chameleon async interface (avoid tasks synchronization between some algorithms)
- FMR_CHAMELEON_READH5_BYTILE: if set to 1, read HDF5 files directly by chameleon tiles instead of the default StarPU handmade task algorithm (read by panel of rows + build tiles + tiles migration to get 2d block-cyclic distribution)
- FMR_CHAMELEON_READH5_BYTILE_DISTRIB_1D: to be used when FMR_CHAMELEON_READH5_BYTILE is used, if set to 1, read by tile but follow a 1D distribution of tiles over MPI processes instead of 2d block-cyclic
- FMR_CHAMELEON_READH5_SYM: if set to 1, read HDF5 files considering matrix is symmetric in main memory
- FMR_CHAMELEON_SBC: set to 1 to enable SBC MPI distribution for symmetric matrices
- FMR_CHAMELEON_SBC_R: to set a specific value for the parameter R of the SBC MPI distribution
To have access to the different timers inside the different algorithms we have to set the FMR_TIMEenvironment variable to 1. ''' export FMR_TIME=1 '''
Development part.
Developements
Improvements todo
-
HDF5 reads are not stable and take the main part of the cpu time for large matrix. * Check the chunk_size in compress matrix * improve the parallel file system access * block version of the read (the Upper part is stored by block) works in the Blas version (shared memory)
-
Consider parallel random generator in shared memory if the MKL is used. The blocking method is available in the MKL vector library (https://software.intel.com/en-us/mkl-vsnotes) (done September 11, 2019)
New features to add
- Put the Nystrom version in the new FMR class format (wrapper and LA)