configuration.texi 13 KB
Newer Older
1 2 3
@c -*-texinfo-*-

@c This file is part of the MORSE Handbook.
4
@c Copyright (C) 2017 Inria
5 6
@c Copyright (C) 2014 The University of Tennessee
@c Copyright (C) 2014 King Abdullah University of Science and Technology
7
@c See the file ../chameleon.texi for copying conditions.
8 9 10 11 12 13 14

@menu
* Compilation configuration::
* Dependencies detection::
@c * Dependencies compilation::
* Use FxT profiling through StarPU::
* Use simulation mode with StarPU-SimGrid::
15
* Use out of core support with StarPU::
16 17 18 19 20 21 22 23 24 25 26 27
@end menu

@c @code{} @option{}
@c @table @code
@c @item truc
@c @item muche
@c @item et zut
@c @c @end table

@node Compilation configuration
@section Compilation configuration

28
The following arguments can be given to the @command{cmake <path to source
29 30 31 32 33 34 35 36 37
directory>} script.

In this chapter, the following convention is used:
@itemize @bullet
@item
@option{path} is a path in your filesystem,
@item
@option{var} is a string and the correct value or an example will be given,
@item
38
@option{trigger} is an CMake option and the correct value is @code{ON} or
39 40 41 42 43 44
@code{OFF}.
@end itemize

Using CMake there are several ways to give options:
@enumerate
@item directly as CMake command line arguments
45 46 47
@item invoque @command{cmake <path to source directory>} once and then use
@command{ccmake <path to source directory>} to edit options through a
minimalist gui (required
48
@samp{cmake-curses-gui} installed on a Linux system)
49 50 51
@item invoque @command{cmake-gui} command and fill information about the
location of the sources and where to build the project, then you have
access to options through a user-friendly Qt interface (required
52 53 54
@samp{cmake-qt-gui} installed on a Linux system)
@end enumerate

55
Example of configuration using the command line
56
@example
PRUVOST Florent's avatar
PRUVOST Florent committed
57 58 59 60 61
cmake ~/chameleon/ -DCMAKE_BUILD_TYPE=Debug          \
                   -DCMAKE_INSTALL_PREFIX=~/install  \
                   -DCHAMELEON_USE_CUDA=ON           \
                   -DCHAMELEON_USE_MPI=ON            \
                   -DBLA_VENDOR=Intel10_64lp         \
62 63
                   -DSTARPU_DIR=~/install/starpu-1.1 \
                   -DCHAMELEON_ENABLE_TRACING=ON
64 65
@end example

66
You can get the full list of options with @option{-L[A][H]} options of
67 68 69 70 71 72 73
@command{cmake} command:
@example
cmake -LH <path to source directory>
@end example

@menu
* General CMake options::
74
* CHAMELEON options::
75 76 77 78 79 80 81 82
@end menu

@node General CMake options
@subsection General CMake options

@table @code

@item -DCMAKE_INSTALL_PREFIX=@option{path} (default:@option{path=/usr/local})
83
Install directory used by @code{make install} where some headers and libraries
84
will be copied.
85
Permissions have to be granted to write onto @option{path} during @code{make
86 87 88 89
install} step.

@item -DCMAKE_BUILD_TYPE=@option{var} (default: @option{Release})
Define the build type and the compiler optimization level.
90
The possible values for @option{var} are:
91 92 93 94 95 96 97 98 99
@table @code
@item empty
@item Debug
@item Release
@item RelWithDebInfo
@item MinSizeRel
@end table

@item -DBUILD_SHARED_LIBS=@option{trigger} (default:@option{OFF})
100
Indicate wether or not CMake has to build CHAMELEON static (@option{OFF}) or
101 102 103 104
shared (@option{ON}) libraries.

@end table

105 106
@node CHAMELEON options
@subsection CHAMELEON options
107

108
List of CHAMELEON options that can be enabled/disabled (value=@code{ON}
109 110 111
or @code{OFF}):
@table @code

112
@item @option{-DCHAMELEON_SCHED_STARPU}=@option{trigger} (default: @code{ON})
113 114
to link with StarPU library (runtime system)

115
@item @option{-DCHAMELEON_SCHED_QUARK}=@option{trigger} (default: @code{OFF})
116 117
to link with QUARK library (runtime system)

118
@item @option{-DCHAMELEON_USE_CUDA}=@option{trigger} (default: @code{OFF})
119 120
to link with CUDA runtime (implementation paradigm for accelerated codes on
GPUs) and cuBLAS library (optimized BLAS kernels on GPUs), can only be used with
121 122
StarPU

123
@item @option{-DCHAMELEON_USE_MPI}=@option{trigger} (default: @code{OFF})
124
to link with MPI library (message passing implementation for use of multiple
125 126
nodes with distributed memory), can only be used with StarPU

127 128 129
@item @option{-DCHAMELEON_ENABLE_TRACING}=@option{trigger} (default: @code{OFF})
to enable trace generation during execution of timing drivers.
It requires StarPU to be linked with FxT library (trace execution of kernels on workers).
130

131
@item @option{-DCHAMELEON_SIMULATION=trigger} (default: @code{OFF})
132 133 134 135
to enable simulation mode, means CHAMELEON will not really execute tasks,
see details in section @ref{Use simulation mode with StarPU-SimGrid}.
This option must be used with StarPU compiled with
@uref{http://simgrid.gforge.inria.fr/, SimGrid} allowing to guess the
136
execution time on any architecture.
137
This feature should be used to make experiments on the scheduler behaviors and
138 139
performances not to produce solutions of linear systems.

140
@item @option{-DCHAMELEON_ENABLE_DOCS=trigger} (default: @code{ON})
141
to control build of the documentation contained in @file{docs/} sub-directory
142
@item @option{-DCHAMELEON_ENABLE_EXAMPLE=trigger} (default: @code{ON})
143
to control build of the examples executables (API usage)
144
contained in @file{example/} sub-directory
145
@item @option{-DCHAMELEON_ENABLE_TESTING=trigger} (default: @code{ON})
146
to control build of testing executables (numerical check) contained in
147
@file{testing/} sub-directory
148
@item @option{-DCHAMELEON_ENABLE_TIMING=trigger} (default: @code{ON})
149
to control build of timing executables (performances check) contained in
150 151
@file{timing/} sub-directory

152
@item @option{-DCHAMELEON_PREC_S=trigger} (default: @code{ON})
153
to enable the support of simple arithmetic precision (float in C)
154
@item @option{-DCHAMELEON_PREC_D=trigger} (default: @code{ON})
155
to enable the support of double arithmetic precision (double in C)
156
@item @option{-DCHAMELEON_PREC_C=trigger} (default: @code{ON})
157
to enable the support of complex arithmetic precision (complex in C)
158
@item @option{-DCHAMELEON_PREC_Z=trigger} (default: @code{ON})
159
to enable the support of double complex arithmetic precision (double complex
160 161 162 163 164
in C)

@item @option{-DBLAS_VERBOSE=trigger} (default: @code{OFF})
to make BLAS library discovery verbose
@item @option{-DLAPACK_VERBOSE=trigger} (default: @code{OFF})
165
to make LAPACK library discovery verbose (automatically enabled if
166 167 168
@option{BLAS_VERBOSE=@code{ON}})
@end table

169
List of CHAMELEON options that needs a specific value:
170 171 172 173 174 175 176 177 178 179 180 181 182
@table @code
@item @option{-DBLA_VENDOR=@option{var}} (default: @option{empty})
The possible values for @option{var} are:
@table @code
@item empty
@item all
@item Intel10_64lp
@item Intel10_64lp_seq
@item ACML
@item Apple
@item Generic
@item ...
@end table
183
to force CMake to find a specific BLAS library, see the full list of BLA_VENDOR
184
in @file{FindBLAS.cmake} in @file{cmake_modules/morse/find}.
185
By default @option{BLA_VENDOR} is empty so that CMake tries to detect all
186 187 188
possible BLAS vendor with a preference for Intel MKL.
@end table

189
List of CHAMELEON options which requires to give a path:
190 191 192 193 194 195 196 197
@table @code
@item @option{-DLIBNAME_DIR=@option{path}} (default: empty)
root directory of the LIBNAME library installation
@item @option{-DLIBNAME_INCDIR=@option{path}} (default: empty)
directory of the LIBNAME library headers installation
@item @option{-DLIBNAME_LIBDIR=@option{path}} (default: empty)
directory of the LIBNAME libraries (.so, .a, .dylib, etc) installation
@end table
198
LIBNAME can be one of the following: BLAS - CBLAS - FXT - HWLOC -
199
LAPACK - LAPACKE - QUARK - STARPU - TMG.
200 201
See paragraph about @ref{Dependencies detection} for details.

202
Libraries detected with an official CMake module (see module files in
203 204 205 206 207 208 209
@file{CMAKE_ROOT/Modules/}):
@itemize @bullet
@item CUDA
@item MPI
@item Threads
@end itemize

210
Libraries detected with CHAMELEON cmake modules (see module files in
211
@file{cmake_modules/morse/find/} directory of CHAMELEON sources):
212 213 214 215 216 217 218 219 220
@itemize @bullet
@item BLAS
@item CBLAS
@item FXT
@item HWLOC
@item LAPACK
@item LAPACKE
@item QUARK
@item STARPU
221
@item TMG
222 223 224 225 226
@end itemize


@node Dependencies detection
@section Dependencies detection
227 228 229
You have different choices to detect dependencies on your system, either by
setting some environment variables containing paths to the libs and headers or
by specifying them directly at cmake configure.
230 231
Different cases :
@enumerate
232
@item detection of dependencies through environment variables:
233
  @itemize @bullet
234 235
  @item @env{LD_LIBRARY_PATH} environment variable should contain the list of
paths
236
where to find the libraries:
237
    @example
238 239
    export @env{LD_LIBRARY_PATH}=$@env{LD_LIBRARY_PATH}:path/to/your/libs
    @end example
240
  @item @env{INCLUDE} environment variable should contain the list of paths
241 242 243 244 245 246 247 248
where to find the header files of libraries
    @example
    export @env{INCLUDE}=$@env{INCLUDE}:path/to/your/headers
    @end example
  @end itemize

@item detection with user's given paths:
  @itemize @bullet
249 250 251
  @item you can specify the path at cmake configure by invoking
  @example
  cmake <path to SOURCE_DIR> -DLIBNAME_DIR=path/to/your/lib
252 253 254 255 256 257
  @end example
  where LIB stands for the name of the lib to look for, example
  @example
  cmake <path to SOURCE_DIR> -DSTARPU_DIR=path/to/starpudir \
                             -DCBLAS_DIR= ...
  @end example
258
  @item it is also possible to specify headers and library directories
259 260
separately, example
  @example
PRUVOST Florent's avatar
PRUVOST Florent committed
261
  cmake <path to SOURCE_DIR>                           \
262 263 264
  -DSTARPU_INCDIR=path/to/libstarpu/include/starpu/1.1 \
  -DSTARPU_LIBDIR=path/to/libstarpu/lib
  @end example
265 266
  @item Note BLAS and LAPACK detection can be tedious so that we provide a
verbose mode. Use @option{-DBLAS_VERBOSE=ON} or @option{-DLAPACK_VERBOSE=ON} to
267 268
enable it.
  @end itemize
269

270 271 272 273 274 275 276 277 278
@end enumerate


@c @node Dependencies compilation
@c @section Dependencies compilation

@node Use FxT profiling through StarPU
@section Use FxT profiling through StarPU

279 280 281 282 283 284 285 286
StarPU can generate its own trace log files by compiling it with the
@option{--with-fxt}
option at the configure step (you can have to specify the directory where you
installed FxT by giving @option{--with-fxt=...} instead of @option{--with-fxt}
alone).
By doing so, traces are generated after each execution of a program which uses
StarPU in the directory pointed by the @env{STARPU_FXT_PREFIX} environment
variable. Example:
287
@example
PRUVOST Florent's avatar
PRUVOST Florent committed
288
export @env{STARPU_FXT_PREFIX}=/home/yourname/fxt_files/
289 290
@end example

291 292 293
When executing a @command{./timing/...} CHAMELEON program, if it has been
enabled (StarPU compiled with FxT and @option{-DCHAMELEON_ENABLE_TRACING=ON}), you
can give the option @option{--trace} to tell the program to generate trace log
294 295
files.

296 297 298 299 300
Finally, to generate the trace file which can be opened with
@uref{http://vite.gforge.inria.fr/, Vite} program, you have to use the
@command{starpu_fxt_tool} executable of StarPU.
This tool should be in @file{path/to/your/install/starpu/bin}.
You can use it to generate the trace file like this:
301 302 303 304 305
@itemize @bullet
@item @command{path/to/your/install/starpu/bin/starpu_fxt_tool -i prof_filename}

There is one file per mpi processus (prof_filename_0, prof_filename_1 ...).
To generate a trace of mpi programs you can call it like this:
306
@item @command{path/to/your/install/starpu/bin/starpu_fxt_tool -i
307 308
prof_filename*}

309
The trace file will be named paje.trace (use -o option to specify an output
310
name).
311
@end itemize
312

313 314
Alternatively, one can also generate directly .paje trace files after the execution
by setting @env{STARPU_GENERATE_TRACE=1}.
315 316 317 318

@node Use simulation mode with StarPU-SimGrid
@section Use simulation mode with StarPU-SimGrid

319
Simulation mode can be enabled by setting the cmake option
320
@option{-DCHAMELEON_SIMULATION=ON}.
321
This mode allows you to simulate execution of algorithms with StarPU compiled
322
with @uref{http://simgrid.gforge.inria.fr/, SimGrid}.
323
To do so, we provide some perfmodels in the @file{simucore/perfmodels/}
324
directory of CHAMELEON sources.
325 326 327 328 329 330
To use these perfmodels, please set the following
@itemize @bullet
@item @env{STARPU_HOME} environment variable to:
  @example
  @code{<path to SOURCE_DIR>/simucore/perfmodels}
  @end example
331
@item @env{STARPU_HOSTNAME} environment variable to the name of the machine to
332 333 334 335
simulate. For example, on our platform (PlaFRIM) with GPUs at Inria Bordeaux
  @example
  @env{STARPU_HOSTNAME}=mirage
  @end example
336
Note that only POTRF kernels with block sizes of 320 or 960 (simple and double
337 338 339
precision) on mirage machine are available for now.
Database of models is subject to change, it should be enrich in a near future.
@end itemize
340

THIBAULT Samuel's avatar
THIBAULT Samuel committed
341 342 343 344
@node Use out of core support with StarPU
@section Use out of core support with StarPU

If the matrix can not fit in the main memory, StarPU can automatically evict
345 346 347 348 349
tiles to the disk.  The descriptors for the matrices which can not fit in the
main memory need to be created with @code{MORSE_Desc_Create_OOC}, so that MORSE
does not force StarPU to keep it in the main memory.

The following variables then need to be set:
THIBAULT Samuel's avatar
THIBAULT Samuel committed
350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366
@itemize @bullet
@item @env{STARPU_DISK_SWAP} environment variable to a place where to store
evicted tiles, for example:
  @example
  @env{STARPU_DISK_SWAP}=/tmp
  @end example
@item @env{STARPU_DISK_SWAP_BACKEND} environment variable to the I/O method,
for example:
  @example
  @env{STARPU_DISK_SWAP_BACKEND}=unistd_o_direct
  @end example
@item @env{STARPU_LIMIT_CPU_MEM} environment variable to the amount of memory
that can be used in MBytes, for example:
  @example
  @env{STARPU_LIMIT_CPU_MEM}=1000
  @end example
@end itemize