INSTALL.org

#+TITLE: A quick installation guide
#+LANGUAGE:  en
#+OPTIONS: H:3 num:t \n:nil @:t ::t |:t _:nil ^:nil -:t f:t *:t <:t
#+OPTIONS: TeX:t LaTeX:t skip:nil d:nil pri:nil tags:not-in-toc html-style:nil

This is a brief discussion about Chameleon installation and usage.
For more information, please read the document users_guide.

Chameleon is written in C, it provides an interface to be called from
Fortran and depends on a couple of external libraries that must be
installed on the system.

* Chameleon prerequisites

  To install Chameleon's libraries, header files, and executables, one
  needs:
  - CMake (version 2.8 minimum): the build system
  - C and Fortran compilers: GNU compiler suite, Clang, Intel or IBM
    can be used
  - python: to generate files in the different precisions
  - external libraries: this depends on the configuration, by default
    the required libraries are
    - StarPU: http://runtime.bordeaux.inria.fr/StarPU/
    - CBLAS, LAPACKE: these are interfaces and there exist several
      providers that can be used with Chameleon
      - Intel MKL, Netlib, OpenBlas
    - BLAS, LAPACK, TMGLIB: there exist several providers that can be
      used with Chameleon
      - Eigen, Intel MKL, Netlib, OpenBlas
    - pthread (libpthread)
    - math (libm)

  Optional libraries:
  - quark: http://icl.cs.utk.edu/quark/
  - cuda: https://developer.nvidia.com/cuda-downloads
  - cublas: comes with cuda http://docs.nvidia.com/cuda/cublas/
  - mpi: openmpi http://www.open-mpi.org/

  These packages must be installed on the system before trying to
  configure/build chameleon.  Please look at the distrib/ directory
  which gives some hints for the installation of dependencies for Unix
  systems.

  We give here some examples for a Debian system:
  #+begin_src

  # Update Debian packages list
  sudo apt-get update
  # Install Netlib blas, lapack, tmglib, cblas and lapacke suite
  sudo apt-get install -y liblapack-dev liblapacke-dev
  # Alernatively to Netlib, OpenBLAS could be used (faster kernels)
  sudo apt-get install -y libopenblas-dev liblapacke-dev
  # Install OpenMPI
  sudo apt-get install -y libopenmpi-dev
  # Install hwloc (used by StarPU or QUARK, already a dependency of OpenMPI)
  sudo apt-get install -y libhwloc-dev
  # install FxT, usefull to export some nice execution traces with StarPU
  sudo apt-get install -y libfxt-dev
  # Install cuda and cuBLAS : only if you have a GPU cuda compatible
  sudo apt-get install -y nvidia-cuda-toolkit nvidia-cuda-dev

  # Install StarPU (with MPI and FxT enabled)
  mkdir -p $HOME/install
  cd $HOME/install
  wget http://starpu.gforge.inria.fr/files/starpu-1.2.2/starpu-1.2.2.tar.gz
  tar xvzf starpu-1.2.2.tar.gz
  cd starpu-1.2.2/
  ./configure --prefix=$HOME/install/starpu --disable-opencl --disable-cuda --with-fxt=/usr/lib/x86_64-linux-gnu/
  make
  make install
  cd $HOME/install
  rm starpu-1.2.2/ starpu-1.2.2.tar.gz -rf

  # Install QUARK : to be used in place of StarPU
  mkdir -p $HOME/install
  cd $HOME/install
  wget http://icl.cs.utk.edu/projectsfiles/quark/pubs/quark-0.9.0.tgz
  tar xvzf quark-0.9.0.tgz
  cd quark-0.9.0/
  sed -i -e "s#prefix=\.\/install#prefix=$HOME/install/quark#g" make.inc
  sed -i -e "s#CFLAGS=-O2#CFLAGS=-O2 -fPIC#g" make.inc
  make
  make install
  cd $HOME/install
  rm quark-0.9.0/ quark-0.9.0.tgz -rf

  #+end_src

** Distribution of Chameleon using Spack

  To get support to install a full distribution (Chameleon +
  dependencies) we encourage users to use the morse branch of *Spack*.

  Please read these documentations:

  * [[http://morse.gforge.inria.fr/spack/spack.html][Spack Morse]]
  * [[http://morse.gforge.inria.fr/spack/spack.html#orgd5b1afe][Section Chameleon]]

*** Usage example for a simple distribution of Chameleon

    #+begin_src sh
    git clone https://github.com/solverstack/spack.git
    . ./spack/share/spack/setup-env.sh
    spack install -v chameleon
    # chameleon is installed here:
    `spack location -i chameleon`
    #+end_src

* Build and install Chameleon with CMake
  Compilation of Chameleon libraries and executables are done with
  CMake (http://www.cmake.org/). This version has been tested with
  CMake 3.5.1 but any version superior to 2.8 should be fine.

  Usage: three steps are required to compile and install Chameleon

  1. configure :
     #+begin_src
     cmake path/to/chameleon -DOPTION1= -DOPTION2= ...
     # see the "Options" section to get list of options
     # see the "Dependencies detection" for details about libraries detection
     #+end_src
  2. build :
     #+begin_src
     make
     # do not hesitate to use -j[ncores] option to speedup the compilation
     #+end_src
  3. install (optional) :
     #+begin_src
     make install
     #+end_src
     Do not forget to specify the install directory with
     *-DCMAKE_INSTALL_PREFIX* at configure
     #+begin_example
     cmake /home/jdoe/chameleon -DCMAKE_INSTALL_PREFIX=/home/jdoe/install/chameleon
     #+end_example

** Dependencies detection
   You have different choices to detect dependencies on your system,
   either by setting some environment variables containing paths to
   the libs and headers or by specifying them directly at cmake
   configure. Different cases :

   1) detection of dependencies through environment variables:
      - LD_LIBRARY_PATH should contain the list of paths where to find
        the libraries:
        #+begin_src
        export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:install/path/to/your/lib
        #+end_src
      - INCLUDE should contain the list of paths where to find the
        header files of libraries
        #+begin_src
        export INCLUDE=$INCLUDE:install/path/to/your/headers
        #+end_src
   2) detection with user's given paths:
      - you can specify the path at cmake configure by invoking ~cmake
        path/to/your/CMakeLists.txt -DLIB_DIR=path/to/your/lib~ where
        LIB stands for the name of the lib to look for
        #+begin_src
        cmake path/to/your/CMakeLists.txt -DSTARPU_DIR=path/to/starpudir \
                                          -DCBLAS_DIR= ...
        #+end_src
        it is also possible to specify headers and library directories
        separately
        #+begin_src
        cmake path/to/your/CMakeLists.txt -DSTARPU_INCDIR=path/to/libstarpu/include/starpu/1.1 \
                                          -DSTARPU_LIBDIR=path/to/libstarpu/lib
        #+end_src
      - note: BLAS and LAPACK detection can be tedious so that we
        provide a verbose mode you can set *-DBLAS_VERBOSE=ON* or
        *-DLAPACK_VERBOSE=ON* to activate it
   3) using pkg-config for libraries that provide .pc files
      - update your *PKG_CONFIG_PATH* to the paths where to find .pc
        files of installed external libraries like hwloc, starpu, some
        blas/lapack, etc

** Options
   You can optionally activate some options at cmake configure (like CUDA, MPI, ...)
   invoking ~cmake path/to/your/CMakeLists.txt -DOPTION1= -DOPTION2= ...~
   #+begin_src
   cmake /home/jdoe/chameleon/ -DCMAKE_BUILD_TYPE=Debug \
                               -DCMAKE_INSTALL_PREFIX=/home/jdoe/install/ \
                               -DCHAMELEON_USE_CUDA=ON \
                               -DCHAMELEON_USE_MPI=ON \
                               -DBLA_VENDOR=Intel10_64lp \
                               -DSTARPU_DIR=/home/jdoe/install/starpu-1.2/ \
                               -DCHAMELEON_ENABLE_TRACING=ON
   #+end_src

   You can get the full list of options with *-L[A][H]* options of cmake command
   #+begin_src
   cmake -LH /home/jdoe/chameleon/
   #+end_src

   You can also set the options thanks to the *ccmake* interface. Some
   options (non-exhaustive list):
   - Native into CMake:
     * *CMAKE_BUILD_TYPE=Debug|Release*
     * *CMAKE_INSTALL_PREFIX=path/to/your/install/dir* (where headers,
       libraries, executables, etc, will be copied when invoking make
       install)
     * *BUILD_SHARED_LIBS=ON|OFF*
     * *CMAKE_C_COMPILER=gcc|icc|...*
     * *CMAKE_Fortran_COMPILER=gfortran|ifort|...*
   - Related to specific modules (find_package) to find external
     libraries:
     * *BLA_VENDOR=Intel10_64lp* (to use intel mkl for example, see the
       list of BLA_VENDOR in FindBLAS.cmake in
       cmake_modules/morse/find)
     * *STARPU_DIR=path/to/root/starpu/install*, see section
       "Dependencies detection"
     * *STARPU_INCDIR=path/to/root/starpu/install/headers*, see section
       "Dependencies detection"
     * *STARPU_LIBDIR=path/to/root/starpu/install/libs*, see section
       "Dependencies detection"
     * List of packages that can searched just like STARPU (with _DIR,
       _INCDIR and _LIBDIR):
       * *BLAS*, *CBLAS*, *EZTRACE*, *FXT*, *HWLOC*, *LAPACK*, *LAPACKE*, *QUARK*,
         *SIMGRID, *TMG*
   - Chameleon specific:
     * *CHAMELEON_USE_MPI=ON|OFF* (default OFF)
     * *CHAMELEON_USE_CUDA=ON|OFF* (default OFF)
     * *CHAMELEON_ENABLE_DOCS=ON|OFF* (default OFF)
     * *CHAMELEON_ENABLE_EXAMPLE=ON|OFF* (default ON)
     * *CHAMELEON_ENABLE_PRUNING_STATS=ON|OFF* (default OFF)
     * *CHAMELEON_ENABLE_TESTING=ON|OFF* (default ON)
     * *CHAMELEON_ENABLE_TIMING=ON|OFF* (default ON)
     * *CHAMELEON_ENABLE_TRACING=ON|OFF* (default OFF)
     * *CHAMELEON_SCHED_STARPU=ON|OFF* (default ON)
     * *CHAMELEON_SCHED_QUARK=ON|OFF* (default OFF)
     * *CHAMELEON_SIMULATION=ON|OFF* (default OFF)

   Libraries detected with an official cmake module (see module files
   in CMAKE_ROOT/Modules/): CUDA - MPI - Threads

   Libraries detected with our cmake modules (see module files in
   cmake_modules/morse_cmake/modules/find/ directory of Chameleon
   sources): BLAS - CBLAS - EZTRACE - FXT - HWLOC - LAPACK - LAPACKE -
   QUARK - SIMGRID - STARPU - TMG

* Use FxT profiling through StarPU
  StarPU can generate its own trace log files by compiling it with the
  ~--with-fxt~ option at the configure step (you can have to specify the
  directory where you installed FxT by giving ~--with-fxt=...~ instead
  of ~--with-fxt~ alone).  By doing so, traces are generated after each
  execution of a program which uses StarPU in the directory pointed by
  the STARPU_FXT_PREFIX environment variable.
  #+begin_example
  export STARPU_FXT_PREFIX=/home/jdoe/fxt_files/
  #+end_example
  When executing a ~./timing/...~ Chameleon program, if it has been
  enabled (StarPU compiled with FxT and
  *-DCHAMELEON_ENABLE_TRACING=ON*), you can give the option ~--trace~ to
  tell the program to generate trace log files.

  Finally, to generate the trace file which can be opened with Vite
  program (http://vite.gforge.inria.fr/), you can use the
  *starpu_fxt_tool* executable of StarPU.  This tool should be in
  ~$STARPU_INSTALL_REPOSITORY/bin~.  You can use it to generate the
  trace file like this:
  #+begin_src
  path/to/your/install/starpu/bin/starpu_fxt_tool -i prof_filename
  #+end_src
  There is one file per mpi processus (prof_filename_0,
  prof_filename_1 ...).  To generate a trace of mpi programs you can
  call it like this:
  #+begin_src
  path/to/your/install/starpu/bin/starpu_fxt_tool -i prof_filename*
  #+end_src
  The trace file will be named paje.trace (use -o option to specify an
  output name).  Alternatively, for non mpi execution (only one
  processus and profiling file), you can set the environment variable
  *STARPU_GENERATE_TRACE=1* to automatically generate the paje trace
  file.

* Use simulation mode with StarPU-SimGrid
  Simulation mode can be activated by setting the cmake option
  CHAMELEON_SIMULATION to ON.  This mode allows you to simulate
  execution of algorithms with StarPU compiled with SimGrid
  (http://simgrid.gforge.inria.fr/).  To do so, we provide some
  perfmodels in the simucore/perfmodels/ directory of Chameleon
  sources.  To use these perfmodels, please set your *STARPU_HOME*
  environment variable to
  ~path/to/your/chameleon_sources/simucore/perfmodels~.  Finally, you
  need to set your *STARPU_HOSTNAME* environment variable to the name of
  the machine to simulate.  For example: *STARPU_HOSTNAME=mirage*.  Note
  that only POTRF kernels with block sizes of 320 or 960 (simple and
  double precision) on mirage and sirocco machines are available for
  now.  Database of models is subject to change.