configuration.texi 12.7 KB
Newer Older
1 2 3
@c -*-texinfo-*-

@c This file is part of the MORSE Handbook.
4
@c Copyright (C) 2017 Inria
5 6
@c Copyright (C) 2014 The University of Tennessee
@c Copyright (C) 2014 King Abdullah University of Science and Technology
7
@c See the file ../chameleon.texi for copying conditions.
8 9 10 11 12 13 14

@menu
* Compilation configuration::
* Dependencies detection::
@c * Dependencies compilation::
* Use FxT profiling through StarPU::
* Use simulation mode with StarPU-SimGrid::
15
* Use out of core support with StarPU::
16 17 18 19 20 21 22 23 24 25 26 27
@end menu

@c @code{} @option{}
@c @table @code
@c @item truc
@c @item muche
@c @item et zut
@c @c @end table

@node Compilation configuration
@section Compilation configuration

28
The following arguments can be given to the @command{cmake <path to source
29 30 31 32 33 34 35 36 37
directory>} script.

In this chapter, the following convention is used:
@itemize @bullet
@item
@option{path} is a path in your filesystem,
@item
@option{var} is a string and the correct value or an example will be given,
@item
38
@option{trigger} is an CMake option and the correct value is @code{ON} or
39 40 41 42 43 44
@code{OFF}.
@end itemize

Using CMake there are several ways to give options:
@enumerate
@item directly as CMake command line arguments
45 46 47
@item invoque @command{cmake <path to source directory>} once and then use
@command{ccmake <path to source directory>} to edit options through a
minimalist gui (required
48
@samp{cmake-curses-gui} installed on a Linux system)
49 50 51
@item invoque @command{cmake-gui} command and fill information about the
location of the sources and where to build the project, then you have
access to options through a user-friendly Qt interface (required
52 53 54
@samp{cmake-qt-gui} installed on a Linux system)
@end enumerate

55
Example of configuration using the command line
56
@example
PRUVOST Florent's avatar
PRUVOST Florent committed
57 58 59 60 61
cmake ~/chameleon/ -DCMAKE_BUILD_TYPE=Debug          \
                   -DCMAKE_INSTALL_PREFIX=~/install  \
                   -DCHAMELEON_USE_CUDA=ON           \
                   -DCHAMELEON_USE_MPI=ON            \
                   -DBLA_VENDOR=Intel10_64lp         \
62 63
                   -DSTARPU_DIR=~/install/starpu-1.1 \
                   -DCHAMELEON_ENABLE_TRACING=ON
64 65
@end example

66
You can get the full list of options with @option{-L[A][H]} options of
67 68 69 70 71 72 73
@command{cmake} command:
@example
cmake -LH <path to source directory>
@end example

@menu
* General CMake options::
74
* CHAMELEON options::
75 76 77 78 79 80 81 82
@end menu

@node General CMake options
@subsection General CMake options

@table @code

@item -DCMAKE_INSTALL_PREFIX=@option{path} (default:@option{path=/usr/local})
83
Install directory used by @code{make install} where some headers and libraries
84
will be copied.
85
Permissions have to be granted to write onto @option{path} during @code{make
86 87 88 89
install} step.

@item -DCMAKE_BUILD_TYPE=@option{var} (default: @option{Release})
Define the build type and the compiler optimization level.
90
The possible values for @option{var} are:
91 92 93 94 95 96 97 98 99
@table @code
@item empty
@item Debug
@item Release
@item RelWithDebInfo
@item MinSizeRel
@end table

@item -DBUILD_SHARED_LIBS=@option{trigger} (default:@option{OFF})
100
Indicate wether or not CMake has to build CHAMELEON static (@option{OFF}) or
101 102 103 104
shared (@option{ON}) libraries.

@end table

105 106
@node CHAMELEON options
@subsection CHAMELEON options
107

108
List of CHAMELEON options that can be enabled/disabled (value=@code{ON}
109 110 111
or @code{OFF}):
@table @code

112
@item @option{-DCHAMELEON_SCHED_STARPU}=@option{trigger} (default: @code{ON})
113 114
to link with StarPU library (runtime system)

115
@item @option{-DCHAMELEON_SCHED_QUARK}=@option{trigger} (default: @code{OFF})
116 117
to link with QUARK library (runtime system)

118
@item @option{-DCHAMELEON_USE_CUDA}=@option{trigger} (default: @code{OFF})
119 120
to link with CUDA runtime (implementation paradigm for accelerated codes on
GPUs) and cuBLAS library (optimized BLAS kernels on GPUs), can only be used with
121 122
StarPU

123
@item @option{-DCHAMELEON_USE_MPI}=@option{trigger} (default: @code{OFF})
124
to link with MPI library (message passing implementation for use of multiple
125 126
nodes with distributed memory), can only be used with StarPU

127 128 129
@item @option{-DCHAMELEON_ENABLE_TRACING}=@option{trigger} (default: @code{OFF})
to enable trace generation during execution of timing drivers.
It requires StarPU to be linked with FxT library (trace execution of kernels on workers).
130

131
@item @option{-DCHAMELEON_SIMULATION=trigger} (default: @code{OFF})
132 133 134 135
to enable simulation mode, means CHAMELEON will not really execute tasks,
see details in section @ref{Use simulation mode with StarPU-SimGrid}.
This option must be used with StarPU compiled with
@uref{http://simgrid.gforge.inria.fr/, SimGrid} allowing to guess the
136
execution time on any architecture.
137
This feature should be used to make experiments on the scheduler behaviors and
138 139
performances not to produce solutions of linear systems.

140 141
@item @option{-DCHAMELEON_ENABLE_DOCS=trigger} (default: @code{ON})
@item @option{-DCHAMELEON_ENABLE_EXAMPLE=trigger} (default: @code{ON})
142
to control build of the examples executables (API usage)
143
@item @option{-DCHAMELEON_ENABLE_TESTING=trigger} (default: @code{ON})
144
to control build of testing executables (numerical check) contained in
145
@item @option{-DCHAMELEON_ENABLE_TIMING=trigger} (default: @code{ON})
146
to control build of timing executables (performances check) contained in
147

148
@item @option{-DCHAMELEON_PREC_S=trigger} (default: @code{ON})
149
to enable the support of simple arithmetic precision (float in C)
150
@item @option{-DCHAMELEON_PREC_D=trigger} (default: @code{ON})
151
to enable the support of double arithmetic precision (double in C)
152
@item @option{-DCHAMELEON_PREC_C=trigger} (default: @code{ON})
153
to enable the support of complex arithmetic precision (complex in C)
154
@item @option{-DCHAMELEON_PREC_Z=trigger} (default: @code{ON})
155
to enable the support of double complex arithmetic precision (double complex
156 157 158 159 160
in C)

@item @option{-DBLAS_VERBOSE=trigger} (default: @code{OFF})
to make BLAS library discovery verbose
@item @option{-DLAPACK_VERBOSE=trigger} (default: @code{OFF})
161
to make LAPACK library discovery verbose (automatically enabled if
162 163 164
@option{BLAS_VERBOSE=@code{ON}})
@end table

165
List of CHAMELEON options that needs a specific value:
166 167 168 169 170 171 172 173 174 175 176 177 178
@table @code
@item @option{-DBLA_VENDOR=@option{var}} (default: @option{empty})
The possible values for @option{var} are:
@table @code
@item empty
@item all
@item Intel10_64lp
@item Intel10_64lp_seq
@item ACML
@item Apple
@item Generic
@item ...
@end table
179 180
to force CMake to find a specific BLAS library, see the full list of BLA_VENDOR
By default @option{BLA_VENDOR} is empty so that CMake tries to detect all
181 182 183
possible BLAS vendor with a preference for Intel MKL.
@end table

184
List of CHAMELEON options which requires to give a path:
185 186 187 188 189 190 191 192
@table @code
@item @option{-DLIBNAME_DIR=@option{path}} (default: empty)
root directory of the LIBNAME library installation
@item @option{-DLIBNAME_INCDIR=@option{path}} (default: empty)
directory of the LIBNAME library headers installation
@item @option{-DLIBNAME_LIBDIR=@option{path}} (default: empty)
directory of the LIBNAME libraries (.so, .a, .dylib, etc) installation
@end table
193
LIBNAME can be one of the following: BLAS - CBLAS - FXT - HWLOC -
194
LAPACK - LAPACKE - QUARK - STARPU - TMG.
195 196
See paragraph about @ref{Dependencies detection} for details.

197
Libraries detected with an official CMake module (see module files in
198 199 200 201 202 203
@itemize @bullet
@item CUDA
@item MPI
@item Threads
@end itemize

204
Libraries detected with CHAMELEON cmake modules (see module files in
205 206 207 208 209 210 211 212 213
@itemize @bullet
@item BLAS
@item CBLAS
@item FXT
@item HWLOC
@item LAPACK
@item LAPACKE
@item QUARK
@item STARPU
214
@item TMG
215 216 217 218 219
@end itemize


@node Dependencies detection
@section Dependencies detection
220 221 222
You have different choices to detect dependencies on your system, either by
setting some environment variables containing paths to the libs and headers or
by specifying them directly at cmake configure.
223 224
Different cases :
@enumerate
225
@item detection of dependencies through environment variables:
226
  @itemize @bullet
227 228
  @item @env{LD_LIBRARY_PATH} environment variable should contain the list of
paths
229
where to find the libraries:
230
    @example
231 232
    export @env{LD_LIBRARY_PATH}=$@env{LD_LIBRARY_PATH}:path/to/your/libs
    @end example
233
  @item @env{INCLUDE} environment variable should contain the list of paths
234 235 236 237 238 239 240 241
where to find the header files of libraries
    @example
    export @env{INCLUDE}=$@env{INCLUDE}:path/to/your/headers
    @end example
  @end itemize

@item detection with user's given paths:
  @itemize @bullet
242 243 244
  @item you can specify the path at cmake configure by invoking
  @example
  cmake <path to SOURCE_DIR> -DLIBNAME_DIR=path/to/your/lib
245 246 247 248 249 250
  @end example
  where LIB stands for the name of the lib to look for, example
  @example
  cmake <path to SOURCE_DIR> -DSTARPU_DIR=path/to/starpudir \
                             -DCBLAS_DIR= ...
  @end example
251
  @item it is also possible to specify headers and library directories
252 253
separately, example
  @example
PRUVOST Florent's avatar
PRUVOST Florent committed
254
  cmake <path to SOURCE_DIR>                           \
255 256 257
  -DSTARPU_INCDIR=path/to/libstarpu/include/starpu/1.1 \
  -DSTARPU_LIBDIR=path/to/libstarpu/lib
  @end example
258 259
  @item Note BLAS and LAPACK detection can be tedious so that we provide a
verbose mode. Use @option{-DBLAS_VERBOSE=ON} or @option{-DLAPACK_VERBOSE=ON} to
260 261
enable it.
  @end itemize
262

263 264 265 266 267 268 269 270 271
@end enumerate


@c @node Dependencies compilation
@c @section Dependencies compilation

@node Use FxT profiling through StarPU
@section Use FxT profiling through StarPU

272 273 274 275 276 277 278 279
StarPU can generate its own trace log files by compiling it with the
@option{--with-fxt}
option at the configure step (you can have to specify the directory where you
installed FxT by giving @option{--with-fxt=...} instead of @option{--with-fxt}
alone).
By doing so, traces are generated after each execution of a program which uses
StarPU in the directory pointed by the @env{STARPU_FXT_PREFIX} environment
variable. Example:
280
@example
PRUVOST Florent's avatar
PRUVOST Florent committed
281
export @env{STARPU_FXT_PREFIX}=/home/yourname/fxt_files/
282 283
@end example

284 285 286
When executing a @command{./timing/...} CHAMELEON program, if it has been
enabled (StarPU compiled with FxT and @option{-DCHAMELEON_ENABLE_TRACING=ON}), you
can give the option @option{--trace} to tell the program to generate trace log
287 288
files.

289 290 291 292
Finally, to generate the trace file which can be opened with
@uref{http://vite.gforge.inria.fr/, Vite} program, you have to use the
@command{starpu_fxt_tool} executable of StarPU.
You can use it to generate the trace file like this:
293 294 295 296 297
@itemize @bullet
@item @command{path/to/your/install/starpu/bin/starpu_fxt_tool -i prof_filename}

There is one file per mpi processus (prof_filename_0, prof_filename_1 ...).
To generate a trace of mpi programs you can call it like this:
298
@item @command{path/to/your/install/starpu/bin/starpu_fxt_tool -i
299 300
prof_filename*}

301
The trace file will be named paje.trace (use -o option to specify an output
302
name).
303
@end itemize
304

305 306
Alternatively, one can also generate directly .paje trace files after the execution
by setting @env{STARPU_GENERATE_TRACE=1}.
307 308 309 310

@node Use simulation mode with StarPU-SimGrid
@section Use simulation mode with StarPU-SimGrid

311
Simulation mode can be enabled by setting the cmake option
312
@option{-DCHAMELEON_SIMULATION=ON}.
313
This mode allows you to simulate execution of algorithms with StarPU compiled
314
with @uref{http://simgrid.gforge.inria.fr/, SimGrid}.
315
directory of CHAMELEON sources.
316 317 318 319 320 321
To use these perfmodels, please set the following
@itemize @bullet
@item @env{STARPU_HOME} environment variable to:
  @example
  @code{<path to SOURCE_DIR>/simucore/perfmodels}
  @end example
322
@item @env{STARPU_HOSTNAME} environment variable to the name of the machine to
323 324 325 326
simulate. For example, on our platform (PlaFRIM) with GPUs at Inria Bordeaux
  @example
  @env{STARPU_HOSTNAME}=mirage
  @end example
327
Note that only POTRF kernels with block sizes of 320 or 960 (simple and double
328 329 330
precision) on mirage machine are available for now.
Database of models is subject to change, it should be enrich in a near future.
@end itemize
331

THIBAULT Samuel's avatar
THIBAULT Samuel committed
332 333 334 335
@node Use out of core support with StarPU
@section Use out of core support with StarPU

If the matrix can not fit in the main memory, StarPU can automatically evict
336 337 338 339 340
tiles to the disk.  The descriptors for the matrices which can not fit in the
main memory need to be created with @code{MORSE_Desc_Create_OOC}, so that MORSE
does not force StarPU to keep it in the main memory.

The following variables then need to be set:
THIBAULT Samuel's avatar
THIBAULT Samuel committed
341 342 343 344 345 346 347 348 349 350 351
@itemize @bullet
@item @env{STARPU_DISK_SWAP} environment variable to a place where to store
evicted tiles, for example:
  @example
  @env{STARPU_DISK_SWAP}=/tmp
  @end example
@item @env{STARPU_DISK_SWAP_BACKEND} environment variable to the I/O method,
for example:
  @example
  @env{STARPU_DISK_SWAP_BACKEND}=unistd_o_direct
  @end example
352 353 354
This will create a hierarchy of directory to store one file per tile. If that
poses problems, one can use the hdf5 I/O method which stores all tiles in a
single file.
THIBAULT Samuel's avatar
THIBAULT Samuel committed
355 356 357 358 359 360
@item @env{STARPU_LIMIT_CPU_MEM} environment variable to the amount of memory
that can be used in MBytes, for example:
  @example
  @env{STARPU_LIMIT_CPU_MEM}=1000
  @end example
@end itemize