GitLab

- add a second thread to separate OTF file operations and trace operations : not as much parallelism as in Paje as file operations are more efficient for OTF (as for Paje parsing, flag MT_PARSING)

- multithreaded loading of serialized files : moves loading from the main thread to multiple threads, as for serialization.

- beginning of the work with MPI : work can now be distributed and several intervals loaded from several machines and displayed there. 

How to use: the flag USE_MPI must be set, links  -lboost_mpi -lmpi -lmpi_cxx must be added in src.pro

mpicc and mpic++ have to be used instead of gcc and g++. This can be set manually in src/makefile, but will be reseted by the global makefile. Another solution is to add

unix{
QMAKESPEC = mpicc
QMAKE_CXX = mpic++
}

to the src.pro file.

To launch on a single machine :
mpirun -np nprocess vite path/to/file.vite2/configure

on several machines with separated screens and a machinefile to list them :
- allow ssh authentification without password (by key)
- allow display on each distant machine with "xhost +"
- have a split folder in a folder accessible with the same path for each machine (nfs or local)
- have vite in the path for each machine
- use mpirun -np nprocess -hostfile machinefile -mca orte_rsh_agent "ssh -X" -xDISPLAY=:0.0 vite -tInterval path/to/file.vite

This will split the given interval in nprocess parts and send them for display to the various nodes described, displaying on their local display

Summary :

- trace can be dumped on disk while parsing
- data can be restored, loading only in memory the part of trace we want to display (by time and by containers).
- a light preview version of the whole trace can be displayed, allowing to chose the interval and actually load data from it


How it works :

- allow Serialization of IntervalOfContainers to the disk while parsing. Each finished IntervalOFContainer (containing 10000 states or links) can be dumped to the disk, using Boost serialization library, in a separate file, then its memory is freed, allowing to parse huge files (tested with 8Gb). Each type, container, linked in the IntervalOfContainer is assigned a unique id to avoid serializing too much data (file Serializer.hpp). If Boost with gzip is used, the resulting data is compressed. This is handled by Several SerializerWriter threads, and the assignment to each thread is done by a Singleton object, the SerializerDispatcher. The number of threads used is the number of CPUs found in the machine.

At the end of parsing all remaining loaded intervalOfContainers are dumped. File naming is "Unique ID of the container"_"IntervalOfContainer index in the container". They are saved in a folder named after the trace file, without extension.  

At the end of dumping, we have a folder containing many files. A file called "name of the trace".vite is created in this folder, which handles all containers, types, with their unique IDs. For each IntervalOFContainer of each Container, the beginning and end timings are also saved. This file will be used to correlate data from the multiple IntervalOfContainers files. It stores also the sum of all the times of all StateTypes encountered in each intervalOFContainers.

- we can now open this .vite file. A ParserSplitted is then used, allowing to restore the structure of the trace and all types. 

   - If the -t option is specified with a time interval, data is directly reloaded from the serialized files, loading in memory only the intervalofcontainers in the time interval.
   - If the -t option was not specified, we load the preview version of the trace, contained in the .vite file.

The preview version only stores states for the moment. When browsing the preview version, user can select a zone and press ctrl. This opens a new vite window, with the same zoom, but the data is then loaded from the serialized files.



How to use : 

needed libraries : libboost_serialization, libboost_thread, and libboost_iostreams . These libraries are in the standard boost package. In linux, they include the gzip library needed for compression and bindings. In windows, this library is not included and has to be included after, and boost recompiled.

- cmake : activate the option VITE_ENABLE_SERIALIZATION in order to check for boost libraries, and to add corresponding files 
- configure :  add the flag --use_boost if libraries are in /usr/lib, --boost_libdir=$dir else.
- by hand in the src.pro file :  add needed libraries ( -lboost_serialization -lboost_thread -lboost_iostreams ) and flags USE_ITC, BOOST_SERIALIZE, and BOOST_GZIP to activate everything



todo: 
- make preview + -c option work together ( -c and -t work together for the moment, -c and preview also, but not when loading actual data from disk)
- add other data to the preview (links events and variables)
- check if using lots of threads to compress is really useful
- better balance between those threads, without rebinding qt signal/slots each time
- tests, tests and tests.
- documentation and comments.

Change the version of the trunk. Pass on 1.2. In Tools.cpp add a condition on the time (because does not exist with mingw, need to find the good function for it). Fix a bug with the OTF parser. Remove set_depth method of container because useless (depth is set in the constructor using its father one). ParserViTE fixed.

Merge trunk in the unified render branch which will be renamed to be 
another branch for new features.

to QList length()), Privatisation of constructors.

errors* .
Improvement of the file loading in parser and trace structures.

- DO NOT COMPILE!

- Split core and interface: create a 'core' directory and put in it the interface_console class. (rename Core class)
- Update help menu.
- Update with trunk. (rev 821 to 867)
- Change some shortcuts. (double clic to display event information)

of 
otf traces. Modification of the parsers to have the otf one, beginning 
of it (containers are printed), adding a new statistics on counters.

New macros in the main_ressource.hpp for the ViTE version (major, minor, 
and a string one).

Modification of the configure script to handle all the changes, you 
should have to restart the script to be able to compile (make distclean 
&& ./configure && make should work).

console et graphique)

Debuggage de Vite.

qui tourne...

problemes avec Qt pour afficher des QObject dans le thread.

compilent et fonctionnent.

Deplacement des fichiers Errors.* dans le dossier Message afin qu'ils 
gerent toutes les erreurs.

partie raccourcis dans le manuel user

Debut d'ajout des messages du parser lors de bugues de parsage dans la 
fenetre graphique.
Les messages s'affichent de haut en bas et pas le contraire.

Reorganisation du dossier source : etape 5 ajout du src.pro et d'un makefile rudimentaire         le projet compile avec un seul makefile

Reorganisation du dossier source : etape 4 supression de dossiers residuels et changement des paths dans les includes

reorganisation du dossier source : etape 3 remonte des sources de message, trace et parser et deplacement des tests de interface, parser et message

Déboggage