Mentions légales du service

Skip to content
Snippets Groups Projects
Florent Pruvost's avatar
PRUVOST Florent authored
f94a7c2 Merge branch 'license' into 'master'
2b90082 Add license file and license header in cpp files

git-subtree-dir: external/fmr
git-subtree-split: f94a7c26ff72fda3702a0c4b409fdf651cb8ed6d
58fc739b
History

FMR Documentation

C++ API for the Fast Methods for Randomized numerical linear algebra

Inria



About

This package is a subset of the master branch of FMR, that contains only classes over LaPACK and Chameleon. Authors: P. Blanchard, O. Coulaud (Olivier.Coulaud@inria.fr)

Developpers F. Pruvost R. Peressoni

Fast and accurate Methods for Randomized numerical linear algebra. This project provides routines for performing low-rank matrix approximations based on randomized techniques:

  • Random projection based LRA: Randomized {SVD, Cholesky, ID, ...}

Please refer to

  • Pierre Blanchard, Olivier Coulaud, Eric Darve, Alain Franc. FMR: Fast randomized algorithms for covariance matrix computations. Platform for Advanced Scientific Computing (PASC), Jun 2016, Lausanne, Switzerland. 2016. (https://hal.inria.fr/hal-01334747v1)
  • Pierre Blanchard, Olivier Coulaud, Eric Darve. Fast hierarchical algorithms for generating Gaussian random fields. [Research Report] 8811, Inria Bordeaux Sud-Ouest. 2015. ⟨https://hal.inria.fr/hal-01228519v2⟩
  • Pierre Blanchard. Fast hierarchical algorithms for the low-rank approximation of matrices, with applications to materials physics, geostatistics and data analysis. Université de Bordeaux, 2017. ⟨https://hal.inria.fr/tel-01534930⟩

and for details on algorithms

What's inside ?

  • CMakeModules/: Contains cmake scripts to find libraries installed on your machine
  • include/:
    • fmr/: Contains all th directories of the header only library
      • Algorithms: low-rank approximation algorithms such as truncated SVD, random projection methods, ...
      • MatrixWrappers: Wrapper for matrix
      • StandardLRA: Classical matrix operation or factorization GEMM, QR, SVD
      • RandomizedLRA: Randomized classes for algorithms
      • Utils: Contains scripts to find libraries installed on your machine

Get and Build FMR

How to get it

To get fmr

git clone --recursive https://gitlab.inria.fr/compose/legacystack/fmr.git

Dependencies

  • Blas/Lapack
  • zlib and bzip2
  • optional
    • Chameleon, MPI, StarPU
    • MKL random generators
    • HDF5

Install FMR

FMR was tested on Linux system and MacosX (10.15) with

  • compilers (Intel, clang, GNU)
  • Blas/Lapack: openblas and MKL, Chameleon
  • zlib and bzip2
  • HDF5 1.10 or 1.12

FMR is a header only library, if you don't build the tests.

git clone git@gitlab.inria.fr:compose/legacystack/fmr.git

Create a directory to build and to install FMR with cmake.

```
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX:PATH=/your/installation/path_for_fmr
cmake --build . --config Release --target install
```

Installation examples with dependencies

Docker image

We provide a docker image with FMR installed with Chameleon and MPI. Please refer to this dockerfile for the recipe. The docker image name is registry.gitlab.inria.fr/compose/legacystack/fmr/ubuntu and you can use it as follows:

docker login registry.gitlab.inria.fr -u gitlab+deploy-token-310 -p CLyjjWw_DPxTQF_jmeJf
docker run -it registry.gitlab.inria.fr/compose/legacystack/fmr/ubuntu

Linux Debian/Ubuntu

sudo apt update -y
sudo apt install -y git python3 cmake build-essential gfortran pkg-config libmkl-dev libstarpu-dev libhdf5-mpi-dev zlib1g-dev libbz2-dev
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1

# install chameleon
cd $HOME
git clone --recursive git@gitlab.inria.fr:solverstack/chameleon.git
cd chameleon
export CHAMELEON_ROOT=$PWD/install
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$CHAMELEON_ROOT -DCHAMELEON_USE_MPI=ON -DBLA_VENDOR=Intel10_64lp -DCBLAS_MT=ON -DLAPACKE_MT=ON -DBUILD_SHARED_LIBS=ON
make -j3 install

# install fmr
cd $HOME
git clone git@gitlab.inria.fr:compose/legacystack/fmr.git
cd fmr
export FMR_ROOT=$PWD/install
mkdir build && cd build
# check
cmake .. -DCMAKE_INSTALL_PREFIX=$FMR_ROOT -DFMR_BUILD_TESTS=ON -DFMR_USE_HDF5=ON -DFMR_USE_CHAMELEON=ON -DFMR_USE_MKL_AS_BLAS=ON
make -j5
ctest -V
# install
cmake .. -DFMR_BUILD_TESTS=OFF
make install

In your cmake project use CHAMELEON_ROOT and FMR_ROOT or set CMAKE_PREFIX_PATH with CHAMELEON_ROOT and FMR_ROOT.

MacOSX

Considering git, gcc and cmake are already installed you need to install the following: zlib, bzip2, CBLAS/LAPACKE (example here with OpenBLAS but you can install Intel MKL), StarPU and Chameleon.

brew install automake autoconf libtool zlib bzip2 hwloc pkgconfig openblas openmpi

# use pkg-config .pc files to detect some dependencies
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/Cellar/openblas/0.3.13/lib/pkgconfig/:${PKG_CONFIG_PATH}
# cmake checks blas.pc not openblas.pc
sudo cp /usr/local/Cellar/openblas/0.3.13/lib/pkgconfig/openblas.pc /usr/local/Cellar/openblas/0.3.13/lib/pkgconfig/blas.pc

# install hdf5 parallel openmpi
# download a source tarball here https://www.hdfgroup.org/downloads/hdf5/source-code/
tar xvf hdf5-1.12.0.tar.gz
cd hdf5-1.12.0
CC=/usr/local/bin/mpicc CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure --with-zlib=/usr/local/opt --prefix=/usr/local/ --enable-shared --disable-fortran --disable-static --disable-tests --enable-threadsafe --with-pthread --enable-unsupported --enable-parallel
make -j5
sudo make install

# install starpu
# download a source tarball here https://files.inria.fr/starpu/
tar xvf starpu-1.3.7.tar.gz
cd starpu-1.3.7/
./configure --enable-blas-lib=none --disable-starpufft --disable-mlr --disable-opencl --disable-cuda --disable-hdf5 --disable-fortran --prefix=/usr/local
make -j5
sudo make install

# install chameleon
cd $HOME
git clone --recursive git@gitlab.inria.fr:solverstack/chameleon.git
cd chameleon
export CHAMELEON_ROOT=/usr/local/
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=${CHAMELEON_ROOT} -DCHAMELEON_USE_MPI=ON -DBLA_PREFER_PKGCONFIG=ON -DBUILD_SHARED_LIBS=ON
make -j5
sudo make install
export PKG_CONFIG_PATH=${CHAMELEON_ROOT}/lib/pkgconfig/:${PKG_CONFIG_PATH}

Don't forget to add Chameleon in the PKG Config path in your .basrc file.

Then follow the install procedure described in the Linux Debian/Ubuntu section

Usefull define to enable some FMR features

These following keywords allow you to access other functionalities or improve performance

  • FMR_HDF5 if you want to fill a matrix from a dataset in ```hdf5`` file
  • FMR_USE_MKL_AS_BLAS if used with Intel MKL
  • FMR_MKL_RANDOM if the MKL library is used for Blas/Lapack or parallel random generator
  • FMR_USE_OPENBLAS_AS_BLAS if used with OpenBlas
  • FMR_CHAMELEON if you consider Chamelon library (Blas/Lapack) - Mandatory for distributed version of the library
  • FMR_USE_MPI to use if Chameleon enabled

How to use FMR in other project

When FMR is installed, we add files to find FMR as a package through cmake. These files are located in dir_to_install_fmr/lib/cmake/fmr So to use FMR in a project, for instance diodon, we add in CMakeLists.txt the following lines

    ## Example with a project called diodon

    # use for example CMAKE_PREFIX_PATH to give the fmr root install dir.
    find_package(FMR CONFIG REQUIRED)
    #
    # link diodon target to fmr libraries
    #
    target_link_libraries(diodon INTERFACE fmr::fmr)

Environment variables to be used during execution

The following environment variables can be used to enable some features at runtime:

  • FMR_TIME: if set to 1, enables timing of some functions
  • FMR_CHAMELEON_NUM_THREADS: number of threads (cpu workers) to be used by chameleon
  • FMR_CHAMELEON_NUM_CUDAS: number of gpus (cuda devices) to be used by chameleon
  • FMR_CHAMELEON_TILE_SIZE: size nb of tiles (square data sub-blocks nb*nb)
  • FMR_CHAMELEON_ASYNC: if set to 1, enables the chameleon async interface (avoid tasks synchronization between some algorithms)
  • FMR_CHAMELEON_READH5_BYTILE: if set to 1, read HDF5 files directly by chameleon tiles instead of the default StarPU handmade task algorithm (read by panel of rows + build tiles + tiles migration to get 2d block-cyclic distribution)
  • FMR_CHAMELEON_READH5_BYTILE_DISTRIB_1D: to be used when FMR_CHAMELEON_READH5_BYTILE is used, if set to 1, read by tile but follow a 1D distribution of tiles over MPI processes instead of 2d block-cyclic
  • FMR_CHAMELEON_READH5_SYM: if set to 1, read HDF5 files considering matrix is symmetric in main memory
  • FMR_CHAMELEON_SBC: set to 1 to enable SBC MPI distribution for symmetric matrices
  • FMR_CHAMELEON_SBC_R: to set a specific value for the parameter R of the SBC MPI distribution

To have access to the different timers inside the different algorithms we have to set the FMR_TIMEenvironment variable to 1. ''' export FMR_TIME=1 '''

Development part.

Developements

Improvements todo

  • HDF5 reads are not stable and take the main part of the cpu time for large matrix. * Check the chunk_size in compress matrix * improve the parallel file system access * block version of the read (the Upper part is stored by block) works in the Blas version (shared memory)

  • Consider parallel random generator in shared memory if the MKL is used. The blocking method is available in the MKL vector library (https://software.intel.com/en-us/mkl-vsnotes) (done September 11, 2019)

New features to add

  • Put the Nystrom version in the new FMR class format (wrapper and LA)