Mentions légales du service

Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • v1.0.0
2 results
Name Last commit Last update
benchmarks
figures
non-libre
public
.gitignore
.gitlab-ci.yml
Dockerfile
README.md
Singularity
plot.R
references.bib
study.tex
README.md

test_FEMBEM: Baseline setup

pipeline status

This repository contains a copy of an experimental study relying on the open-source version of the test_FEMBEM solver test suite [6]. In this case, we do not rely on the GNU Guix [1] transactional package manager and literate programming [2] to ensure reproducibility of the research study. We use either the combination of native system package manager and manual builds or a container solution to manage the software environment and simple comments or README files in an attempt to provide a documentation hopefully allowing others to reproduce the study and its experiments. We refer to this version of the study as to Baseline setup.

Where to find the original study?

The study is available online as a PDF document and the latest benchmark results as a CSV file.

How to reproduce the study?

Software environment

To enter the software environment required to run experiments, post-process results and produce the PDF of the study, one can either rely on system package manager and manual builds or use pre-built Singularity [3] software container.

Manual build

Manual build instructions are provided for a personal machine with superuser privileges and running a Debian GNU/Linux 11 installation as well as for the PlaFRIM [4] high-performance computing platform using modules to provide software packages.

Installing pre-built dependencies

On Debian GNU/Linux 11, we can install pre-built dependencies using the system package manager apt. Note that in order to be able to install the Intel(R) MKL library, we need to add an extra package source to apt. To do this, we follow the instructions in the 'Pre-installation Steps' section of the related official vendor's site.

Then, with the command line below we install all the pre-built dependencies.

sudo apt install -y build-essential pkg-config python3 python3-dev \
  python3-pygments flex bison r-base r-base-dev r-cran-ggplot2 r-cran-svglite \
  inkscape texlive-full libhwloc15 libhwloc-dev cmake openmpi-bin \
  openmpi-common openmpi-doc libopenmpi3 libopenmpi-dev intel-oneapi-mkl \
  intel-oneapi-mkl-devel

Once the installation completes, we can setup Intel(R)-related environment variables for all users as follows. A re-login is required for this setting to take effect.

sudo bash -c 'echo "source /opt/intel/oneapi/setvars.sh" > /etc/profile.d/intel.sh'

On the PlaFRIM platform using modules, we load the pre-built dependencies like this:

module load build/cmake/3.15.3 mpi/openmpi/4.1.1 linalg/mkl/2022.0.2 \
       compiler/gcc/10.3.0 hardware/hwloc/2.7.0
Building the other dependencies

We use the folders $HOME/src and $HOME/bin to store sources and builds, respectively. We can create these folders with:

mkdir -p $HOME/src
mkdir -p $HOME/bin

Then, we download all the sources into $HOME/src.

cd $HOME/src
wget https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2
wget https://files.inria.fr/starpu/starpu-1.3.9/starpu-1.3.9.tar.gz
wget https://gitlab.inria.fr/solverstack/chameleon/uploads/b299d6037d7636c6be16108c89bc2aab/chameleon-1.1.0.tar.gz
git clone https://github.com/jeromerobert/hmat-oss
git clone https://gitlab.inria.fr/solverstack/test_fembem

We are now ready to proceed with the build of test_FEMBEM dependencies and, at the end, test_FEMBEM itself.

  1. jemalloc 
    cd $HOME/src
mkdir jemalloc
tar -xvf jemalloc-4.5.0.tar.bz2 -C jemalloc --strip-components 1
cd $HOME/src/jemalloc
./configure --prefix=$HOME/bin/jemalloc
make -j8 install
  2. StarPU 
    cd $HOME/src
mkdir starpu
tar -xvzf starpu-1.3.9.tar.gz -C starpu --strip-components 1
cd $HOME/src/starpu
./configure --enable-fast --enable-shared --disable-static --disable-cuda \
            --disable-opencl --enable-maxcpus=48 --enable-maxbuffers=4 \
            --disable-build-examples --disable-build-doc --enable-mpi \
            --disable-starpu-top --disable-gcc-extensions --disable-fortran \
            --disable-mlr --prefix=$HOME/bin/starpu
make -j8 install
  3. Chameleon 
    cd $HOME/src
mkdir chameleon
tar -xvf chameleon-1.1.0.tar.gz -C chameleon --strip-components 1
cd $HOME/src/chameleon
mkdir build
cd build
cmake -DCHAMELEON_USE_MPI=ON -DBLA_VENDOR=Intel10_64lp \
      -DBUILD_SHARED_LIBS=ON -DCMAKE_PREFIX_PATH="$HOME/bin/starpu" \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/chameleon ..
make -j8 install
  4. HMAT-OSS 
    cd $HOME/src/hmat-oss
mkdir build
cd build
cmake -DHMAT_JEMALLOC=ON -DHMAT_EXPORT_BUILD_DATE=ON \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/hmat-oss \
      -DCMAKE_PREFIX_PATH="$HOME/bin/jemalloc;$HOME/bin/starpu" ..
make -j8 install
  5. test_FEMBEM 
    cd $HOME/src/test_fembem
mkdir build
cd build
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/bin/starpu/lib"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/bin/chameleon/lib"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/bin/hmat-oss/lib"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$LIBRARY_PATH"
cmake -DCMAKE_INSTALL_PREFIX=$HOME/bin/test_fembem \
      -DCMAKE_PREFIX_PATH="$HOME/bin/starpu;$HOME/bin/chameleon;$HOME/bin/hmat-oss" \
      ..
make -j8 install

At the end of the process, we modify our .bashrc as follows in order to make the dependency libraries and the test_FEMBEM executable available in the environment permanently.

echo "export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:$LD_LIBRARY_PATH" \
     >> $HOME/.bashrc
echo 'export PATH="$PATH:$HOME/bin/test_fembem/bin"' >> $HOME/.bashrc
source $HOME/.bashrc

Container solution

A pre-built Singularity container with all the software packages required to run experiments, post-process results and produce the PDF of the study is available in the Singularity container library and can be downloaded using the command below.

singularity pull test-fembem-container.sif \
                 library://mfelsoci/test-fembem/container:latest

The recipe container.def used to build the container is present in the root of this repository. See [5] for further information on building Singularity containers.

Running the experiments

We need to clone this repository first.

git clone \
  https://gitlab.inria.fr/tuto-techno-guix-hpc/test_fembem/baseline-setup

Then, we navigate to the repository, and create an empty folder named results under ./benchmarks which will contain the results of a new benchmark campaign.

cd baseline-setup/benchmarks
mkdir -p results

At this point, we can launch the new benchmark campaing using the dedicated run.sh shell script. The parameters for the different test cases that shall be run are defined in ./benchmarks/definitions.csv. See the comments in ./benchmarks/run.sh for more details about this file.

./run.sh -d definitions.csv -o results

Note that to run the benchmarks within the Singularity container, we would need to use this command instead (assuming that the container is located in the current working directory).

singularity exec --cleanenv container.sif bash ./benchmarks/run.sh \
                                          -d ./benchmarks/definitions.csv \
                                          -o ./benchmarks/results

Post-processing

Once the benchmarks have finished running, we can post-processes the results in ./benchmarks/results/results.csv using the plot.R R script.

We generate the figures from within the root of the repository.

Rscript plot.R ./benchmarks/results/results.csv

At the end, we produce the PDF document of the study.

pdflatex --shell-escape study
bibtex study
pdflatex --shell-escape study
pdflatex --shell-escape study

Et voilà !

References

  1. GNU Guix software distribution and transactional package manager https://guix.gnu.org.
  2. Literate Programming, Donald E. Knuth, 1984 https://doi.org/10.1093/comjnl/27.2.97.
  3. SingularityCE, Sylabs https://sylabs.io/singularity/.
  4. Plateforme fédérative pour la recherche en informatique et en mathématiques (PlaFRIM) https://www.plafrim.fr/.
  5. Build a Container, Singularity 3.0 container documentation https://sylabs.io/guides/3.0/user-guide/build_a_container.html.
  6. test_FEMBEM, a simple application for testing dense and sparse solvers with pseudo-FEM or pseudo-BEM matrices https://gitlab.inria.fr/solverstack/test_fembem.