Mentions légales du service

Skip to content
Snippets Groups Projects

Draft: Rabab/al4san

Closed Draft: Rabab/al4san
faverge/chameleon:rabab/altanal into master
Closed Rabab Alomairy requested to merge faverge/chameleon:rabab/altanal into master

I have initiated a merge request for chameleon with al4san integration. I would really appreciate if you can accept it.

Edited by Mathieu Faverge

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Mathieu Faverge
    Mathieu Faverge @faverge ·
    Owner

    Hi @Rabab, Thanks for the merge request, I'll check that next week. During this time, can you start discussing with @fpruvost to integrate al4san in the hiepacs image for the CI., such that the pull request can compile. To merge the pull request, you will also need to rebase on the last master.

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Hello @faverge,

    Thanks you very much for your respond. I am using Jenkins currently and I will be happy to do it in your side.

    Thank you again

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Dear @fpruvost,

    I would like to ask you how can I help you in order to integrate al4san in the hiepacs image for the IC., in order for the pull request to compile.

    Could you please share with me your username on github to give you an access to al4san repository?

    Thank you and I appreciate your support.

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Hello @fpruvost,

    Yes it is a private repository on github. You have an access to it at https://github.com/ecrc/al4san-dev/invitations

    Please give me the Docker image SSH key to add it to the deploy key list of al4san.

    Thank you.

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Hello @fpruvost ,

    I have done adding the ssh key.

    Building Al4san resemble chameleon:

    1- Enable quark using AL4SAN_SCHED_QUARK option. For quark runtime, you need to use quark runtime directory inside al4san package which has exposed one private function for task flag.

    2- Enable starpu using AL4SAN_SCHED_STARPU option. For starpu, you need to use 1.3 branch nightly snapshot.

    3- Enable parsec using AL4SAN_SCHED_PARSEC option. In order for chameleon to work with last commit from parsec the header #include <parsec/insert_function.h> inside chameleon_parsec.h needs to be changed to #include <parsec/interfaces/superscalar/insert_function.h>.

    Thank you very much

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Hello @fpruvost ,

    First, I will give you just a brief of what AL4SAN is and what does it means for Chamaleon. AL4SAN is abstraction layer that Chameleon will use to switch between different runtime systems when it is coming to define the task. Therefore, Chameleon does not need to define codelets for each runtime instead it will use AL4SAN APIs as unified interface to Quark, StarPU, and PaRSEC.

     As you see, in Chameleon in this merge request, there is al4san-codelet folder which contains Cholesky, QR, and Blas3 codelets. These codelets use al4san task api for defining task insert and task unpack and it will enable switching between runtimes without changing the codelet code for each runtime.

    Here it is the steps for enabling chameleon with al4san. You need first to add FindAL4SAN.cmake to cmake_modules/morse_cmake/modules/find/. I have attached FindAL4SAN.cmake module. FindAL4SAN.cmake

    For Quark runtime:

     1.  You need to build, compile and install quark runtime inside al4san repository.  


 2.  Building AL4SAN with Quark runtime as backend:
    cmake .. -DAL4SAN_SCHED_QUARK=ON -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DQUARK_INCDIR=al4san/quark-0.9.0 -DQUARK_LIBDIR=al4san /quark-0.9.0


 3.  Compile, install, and use PKG_CONFIG_PATH environment variable to specify paths in which  pkg-config will search for al4san.pc that built based on Quark.


 4.  Then, you will be able to build, compile, install Chameleon.  Chameleon with Al4san can be enabled by using cmake option -DCHAELEON_USE_AL4SAN=ON:

    cmake .. -DCHAMELEON_SCHED_QUARK=ON -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DCHAMELEON_USE_AL4SAN=ON

     5.  Chameleon now will use al4san with quark as backend.

    For StarPU runtime:

     1.  You need to build, compile and install 1.3 branch nightly snapshot.

 2.  Building AL4SAN with StarPU runtime as backend:

    AL4SAN-StarPU:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DAL4SAN_SCHED_STARPU=ON

    AL4SAN-StarPU-mpi:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DAL4SAN_SCHED_STARPU=ON -DAL4SAN_USE_MPI=ON

    AL3SAN-StarPU-gpu:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DAL4SAN_SCHED_STARPU=ON -DAL4SAN_ENABLE_CUDA=ON -DAL4SAN_USE_CUDA=ON

     3.  Compile, install, and use PKG_CONFIG_PATH environment variable to specify paths in which pkg-config will search for al4san.pc that built based on StarPU.

 4.  Then, you will be able to build, compile, install Chameleon.  Chameleon with Al4san can be enabled by using cmake option -DCHAELEON_USE_AL4SAN=ON:

    *Chameleon -StarPU *:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DCHAMELEON_SCHED_STARPU=ON -DCHAMELEON_USE_ALTANAL=ON

    Chameleon-StarPU-mpi:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DCHAMELEON_SCHED_STARPU=ON -DCHAMELEON_USE_ALTANAL=ON -DCHAMELEON_USE_MPI=ON

    Chameleon-StarPU-gpu:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DCHAMELEON_SCHED_STARPU=ON -DCHAMELEON_ENABLE_CUDA=ON -DCHAMELEON_USE_CUDA=ON -DCHAMELEON_USE_ALTANAL=ON

     5.  Chameleon now  use al4san with starpu as backend.

    For PaRSEC runtime:

     1.   You need to build, compile and install parsec runtime. Moreover, In order for chameleon to work with last commit from parsec the header #include <parsec/insert_function.h> inside chameleon_parsec.h needs to be changed to #include <parsec/interfaces/superscalar/insert_function.h>.


2.  Building AL4SAN with PaRSEC runtime as backend:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DAL4SAN_SCHED_PARSEC=ON -DAL4SAN_USE_MPI=ON

    3.  Compile, install, and use PKG_CONFIG_PATH environment variable to specify paths in which pkg-config will search for al4san.pc that built based on PaRSEC.

 4.  Then, you will be able to build, compile, install Chameleon.  Chameleon with Al4san can be enabled by using cmake option -DCHAELEON_USE_AL4SAN=ON:

    cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DCHAMELEON_SCHED_PARSEC=ON -DCHAMELEON_USE_MPI=ON -DCHAMELEON_USE_ALTANAL=ON

     5.  Chameleon now will use al4san with parsec as backend.

    Please note when CHAMELEON_USE_AL4SAN option is OFF, the chameleon should run as usual.

    I hope these details are clear and let me know if you have any question.

    Edited by Rabab Alomairy
  • Hatem Ltaief
    Hatem Ltaief @ltaief ·
    Developer

    @faverge @fpruvost thanks for your help! Are we facing any showstoppers?

  • Mathieu Faverge
    Mathieu Faverge @faverge ·
    Owner

    Vacations and starting of the school year :)

  • Mathieu Faverge
    Mathieu Faverge @faverge ·
    Owner

    More seriously, it would be good, if we could rely on a public repository revision, or on a release. So we can have the CI that includes only one revision, or an automatically updated one through submodules.

  • Hatem Ltaief
    Hatem Ltaief @ltaief ·
    Developer

    You could at least have mentioned the world cup championship, that would make more sense ;-)

    Are you referring to AL4SAN repo? If yes, we want to get AL4SAN out asap, probably in a couple of weeks or so. But in the meantime, can't we integrate in a dev branch of chameleon using a tarball of Al4SAN? This won't be disruptive and will be a non-intrusive solution. Thoughts?

  • Hatem Ltaief
    Hatem Ltaief @ltaief ·
    Developer

    StarPU and quark are OK. I am not sure about PaRSEC. @Rabab ?

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Hello @fpruvost

    AL4SAN is compatible with:

    1- starpu 1.2.6 the last release

    2- parsec from https://bitbucket.org/mfaverge/parsec.git

    3- Quark from https://github.com/ecrc/quark. There is some slight changes for quark for purpose of our al4san.

    Edited by Rabab Alomairy
  • Hatem Ltaief
    Hatem Ltaief @ltaief ·
    Developer

    The changes on the original Quark source code (which is not supported anymore) are related to exposing internal functions related to task queuing in the library.

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    You need to specify the include path in your cmake.

    cmake .. -DAL4SAN_SCHED_QUARK=ON -DCMAKE_INSTALL_PREFIX=$PWD/installdir -DQUARK_INCDIR=<path to quark dir> -DQUARK_LIBDIR=<path to quark dir>

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Could you please pull the minor changes to al4san?

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Great!! Yes.

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Yes I don't rely on it anymore. If you need have another issue let me know. Thanks

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    Could you please use FindAL4SAN.cmake inside al4san in the following path al4san-dev/cmake_modules/al4san_cmake/modules/find?

  • Rabab Alomairy
    Rabab Alomairy @Rabab ·
    Author Developer

    @fpruvost Have you found FindAL4SAN.cmake or you can find it in the attachment? You need to add FindAL4SAN.cmake to chameleon/cmake_modules/morse_cmake/modules/find

    FindAL4SAN.cmake

    Edited by Rabab Alomairy
  • Rabab Alomairy added 1 commit

    added 1 commit

    • 5b32d9a0 - include_directories for al4san under al4san cmake

    Compare with previous version

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply