- improved documentation evaluation/test generation

html/cli.html

@@ -13,17 +13,20 @@
 The implementation offers various command line tools available in
 the <tt>fr.inrialpes.exmo.align.cli</tt> package:
 <dl>
-<dt>ParserPrinter</dt><dd></dd>
+<dt>Procalign</dt><dd>Runs a matcher on a pair of ontologies</dd>
+<dt>ParserPrinter</dt><dd>Parses and alignment and displays it in
+    various formats.</dd>
 <!--dt>DiffAlign</dt><dd>TODO</dd-->
-<dt>Procalign</dt><dd></dd>
-<dt>EvalAlign</dt><dd></dd>
-<dt>TestGen</dt><dd></dd>
-<dt>GroupAlign</dt><dd></dd>
-<dt>GroupEval</dt><dd></dd>
-<dt>ExtGroupEval</dt><dd></dd>
-<dt>WGroupEval</dt><dd></dd>
-<dt>GroupOutput</dt><dd></dd>
-<dt>GenPlot</dt><dd></dd>
+<dt>EvalAlign</dt><dd>Evaluates and alignment with respect to a
+    reference alignment.</dd>
+<dt>TestGen</dt><dd>Generates tests for evaluating matchers.</dd>
+<dt>GroupAlign</dt><dd>Runs a matcher against several different test cases.</dd>
+<dt>GroupEval</dt><dd>Evaluates several matcher results in several different test cases.</dd>
+<dt>ExtGroupEval</dt><dd>Same as above with extended evaluators</dd>
+<dt>WGroupEval</dt><dd>Same as above with weighted evaluators.</dd>
+<dt>GenPlot</dt><dd>Generates various pliots from matcher resulst
+    (ROC curves or precision/recall graphs).</dd>
+<dt>GroupOutput</dt><dd>Generates other representations of matcher results.</dd>
 </dl>
 </p>
 <p>
@@ -38,8 +41,7 @@ These command line operations usually share switch conventions:
 <p>
 Another such utility (<tt>ParserPrinter</tt>) allows to read an
 alignment, to manipulate it and to output it without processing
-the <tt>align</tt> method. Other utilities dealing with evaluation are
-presented in <a href="eval.html">the evaluation page</a>.
+the <tt>align</tt> method.
 </p>
 
 <h2>Matching</h2>
@@ -113,8 +115,8 @@ in the <a href="testgen.html">test generation page</a>.
 
 <p>
 Tools for evaluating (<tt>EvalAlign</tt>) and batch evaluating
-alignments  (<tt>GroupEval</tt>) and displaying
-results (<tt>GenPlot</tt>) are described in the <a href="eval.html">evaluation page</a>.
+alignments  (<tt>GroupEval</tt>, <tt>ExtGroupEval</tt>, <tt>WGroupEval</tt>) and displaying
+results (<tt>GenPlot</tt>, <tt>GroupOutput</tt>) are described in the <a href="eval.html">evaluation page</a>.
 </p>
 
 <address>

html/eval.html

@@ -335,6 +335,33 @@ $ java -cp ../lib/procalign.jar fr.inrialpes.exmo.align.cli.GroupEval -t triangl
   better.</b>
 </center>
 
+<p>
+The display below synthesises the results of a matchers by aggregating
+in one cell a group of tests in which the ontologies to compare share
+a common set of characteristics (namely that the same set of features
+has been altered). 
+Each diamond shaped cell corresponds to the
+availability of some features in the test (l=label and comments,
+p=properties, i=instances, h=hierarchy).
+Each cell is presented with a color representing the
+average of the F-measure in each of these tests. The darker the cell,
+the better the algorithm. These diagrams are topologically correct,
+i.e., if a cell touches another cell on one side, they differ from
+only one 'feature', but not complete, i.e., it is not possible to
+present all these connections in a planar way.
+This graphic representation is obtained through the <tt>GroupOutput</tt>
+class by generating a Latex file corresponding in the following figure.
+<div class="terminal">
+$ java -cp ../lib/procalign.jar fr.inrialpes.exmo.align.cli.GroupOutput -v -c -m f -l "refalign,matcher1,matcher2,..." > losanges.tex
+</div>
+</p>
+
+<center>
+<img src="img/losanges.png" /><br />
+<b>Matcher performance display by <tt>GroupOutput</tt> on a grid preserving as much as possible
+the topology of tests.</b>
+</center>
+
 <h2>Implementing and extending the API</h2>
 
 <p>

html/license.html

@@ -13,8 +13,8 @@
   the <a href="credits.html">credit page</a>.</p>
 
 <p>The alignment API and its implementation is distributed under the
-  GNU <a href="../distrib/LICENSE.TXT">Lesser General Public
-  License</a>.</p>
+  GNU <a href="http://www.gnu.org/licenses/lgpl-2.1.html">Lesser General Public
+  License</a> 2.1 or above.</p>
 
 <p>Other terms may be granted on demand.</p>
 

html/testgen.html

@@ -98,40 +98,144 @@ alignment. Following the schema of the figure.
 <h3>Predefined alterators</h3>
 
 <p>
-The available predefined alterators are:
+We provide a description of the alterators currently implemented in the Alignment API. 
+</p>
+<p>
+Their functional description is given at the level of OWL, however, the actual implementation largely differs by taking advantage of different internal structures referring to an actual ontology model. Hence, sometimes modifications are applied on the OWL structure, and sometimes they are applied at the level of triples. However, the final model and alignments are generated at the end of the process. This is only at the end of the process that the entities are assigned a final URI.
+</p>
+<p>
+Parameters are key-value pairs defining the degree or modalities of alteration.
+The priority is expressed by a level such that modifications of lower level must be applied before modifications of higher level.
+</p>
 <dl>
-<dt>EmptyModification []</dt>
-<dd>generates a test in which the initial ontology is not
-  altered. This test is used in test sets (see below), for the root test.</dd>
-<!--dt>AddClassLevel [addClassLevel=<i>p.n</i>]</dt>
-<dd>adds at level <i>p</i> (from the top) of the class hierarchy </dd-->
-<dt>AddClasses [addClasses=<i>p</i>]</dt>
-<dd>adds <i>p</i>% classes.</dd>
-<dt>AddProperties [addProperties=<i>p</i>]</dt>
-<dd>adds <i>p</i>% properties.</dd>
-<dt>FlattenLevel [levelFlattened=<i>n</i>]</dt>
-<dd>removes all classes of level <i>n</i> (their subclasses are
-  directly connected to their superclasses).</dd>
-<!--dt>RemoveClassLevel []</dt>
-<dd></dd-->
-<dt>RemoveClasses [removeClasses=<i>p</i>]</dt>
-<dd>removes <i>p</i>% of classes.</dd>
-<dt>RemoveComments [removeComments=<i>p</i>]</dt>
-<dd>removes <i>p</i>% of comments.</dd>
-<dt>RemoveIndividuals [removeIndividuals=<i>p</i>]</dt>
-<dd>removes <i>p</i>% of individuals.</dd>
-<dt>RemoveProperties [removeProperties=<i>p</i>]</dt>
-<dd>removes <i>p</i>% of properties.</dd>
-<dt>RemoveRestrictions [removeRestrictions=<i>p</i>]</dt>
-<dd>removes <i>p</i>% of restriction.</dd>
-<dt>RenameClasses [renameClasses=<i>p</i>]</dt>
-<dd>renames <i>p</i>% of classes in various ways [default: random]</dd>
-<dt>RenameProperties [renameProperties=<i>p</i>]</dt>
-<dd>renames <i>p</i>% of properties in various ways [default: random]</dd>
-<dt>SuppressHierarchy [noHierarchy]</dt>
-<dd>suppresses all subclasses relations (but to owl:Thing)</dd>
+<dt>EmptyModification</dt><dd>
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.EmptyModification</tt>
+<br /><b>No parameters</b>
+<br /><b>Result:</b> generates a test in which the initial ontology is not altered. This test is used in test sets, for generating as initial identity alignment.
+<br /><b>Functional description:</b> For each class and each property in the input ontology, creates a correspondence with equivalence between this entity and itself.
+<br /><b>Priority:</b> 0
+
+</dd><dt>RemoveClasses</dt><dd>
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RemoveClasses</tt>
+<br /><b>Parameters:</b> removeClasses=<i>p</i>
+<br /><b>Result:</b> removes <i>p</i>% of classes.
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of classes to suppress from the ontology.
+For each of these classes, 
+% removeClass
+	attaches, directly, its subclasses to its superclasses (including constraints, i.e., anonymous <tt>Restriction</tt> classes);
+	attaches, directly, its instances to its superclasses;
+	converts all restrictions involving it (<tt>allValuesFrom</tt>, <tt>someValueFrom</tt>) with its superclasses; 
+% changeDomainRange
+	converts all property restrictions involving it (<tt>domain</tt>, <tt>range</tt>) with its superclasses;
+	remove the class.
+% alignment.remove
+Updates the alignment by suppressing all correspondences involving the suppressed classes.
+<br /><b>Priority:</b> 1
+
+</dd><dt>RemoveComments</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RemoveComments</tt>
+<br /><b>Parameters:</b> removeComments=<i>p</i>
+<br /><b>Result:</b> removes <i>p</i>% of comments.
+<br /><b>Functional description:</b> Selects randomly and independently from the ontology <i>p</i>% of classes, properties and individuals whose comments will be suppressed. Removes the comments of each selected entity.
+The alignment is not modified.
+<br /><b>Priority:</b> 2
+
+</dd><dt>RemoveIndividuals</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RemoveIndividuals</tt>
+<br /><b>Parameters:</b> removeIndividuals=<i>p</i>
+<br /><b>Result:</b> removes <i>p</i>% of individuals.
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of individuals to suppress from the ontology.
+Suppresses from the ontology model all statements using these individuals, either as subject or object.
+The alignment is not modified, because it does not contain individuals.
+<br /><b>Priority:</b> 2
+
+</dd><dt>RemoveProperties</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RemoveProperties</tt>
+<br /><b>Parameters:</b> removeProperties=<i>p</i>
+<br /><b>Result:</b> removes <i>p</i>% of properties.
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of properties to suppress from the ontology.
+For each of these properties,
+	remove all restrictions (<tt>allValuesFrom</tt>, <tt>someValuesFrom</tt>) involving it,
+	remove all assertions involving it.
+% alignment.remove
+Updates the alignment by suppressing all correspondences involving the suppressed properties.
+<br /><b>Priority:</b> 2
+
+</dd><dt>RemoveRestrictions</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RemoveRestrictions</tt>
+<br /><b>Parameters:</b> removeRestrictions=<i>p</i>
+<br /><b>Result:</b> removes <i>p</i>% of restriction.
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of properties restrictions to suppress from the ontology.
+For each of these restrictions, remove it from the model.
+The alignment is not modified.
+<br /><b>Priority:</b> 3
+
+</dd><dt>RenameClasses</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RenameClasses</tt>
+<br /><b>Parameters:</b> renameClasses=<i>p</i>, renameMethod=<i>m</i>
+<br /><b>Result:</b> renames <i>p</i>% of classes depending on the requested method <i>m</i> (random strings, translation, synonyms) [default: random]
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of classes from the ontology to rename.
+For each of these classes, create a new name depending on the selected <i>m</i> and replace its <tt>rdfs:label</tt> by the new name.
+For each triple of the model, all URI fragments are replaced if necessary by the corresponding new name.
+Each correspondence of the alignment involving a renamed class is replaced by the corresponding new URI.
+<br /><b>Priority:</b> 4
+
+</dd><dt>RenameProperties</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.RenameProperties</tt>
+<br /><b>Parameters:</b> renameProperties=<i>p</i>, renameMethod=<i>m</i>
+<br /><b>Result:</b> renames <i>p</i>% of properties depending on the requested method <i>m</i> (random strings, translation, synonyms) [default: random]
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of properties from the ontology to rename.
+For each of these properties, create a new name depending on the selected <i>m</i>.
+For each triple of the model, all URI fragments are replaced if necessary by the corresponding new name.
+Each correspondence of the alignment involving a renamed property is replaced by the corresponding new URI.
+<br /><b>Priority:</b> 4
+
+</dd><dt>AddClasses</dt><dd>
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.AddClasses</tt>
+<br /><b>Parameters:</b> addClasses=<i>p</i>
+<br /><b>Result:</b> adds <i>p</i>% classes.
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of classes from the ontology to add a subclass to.
+For each of these classes, generates randomly a new name; creates a new subclass of the selected class with this name.
+The alignment is not modified.
+<br /><b>Priority:</b> 5
+
+</dd><dt>AddProperties</dt><dd>
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.AddProperties</tt>
+<br /><b>Parameters:</b> addProperties=<i>p</i>
+<br /><b>Result:</b> adds <i>p</i>% properties.
+<br /><b>Functional description:</b> Selects randomly <i>p</i>% of properties from the ontology to add. 
+For each of these properties, generates randomly a new name. 
+For half of these new properties, creates an <tt>ObjectProperty</tt>: selects randomly a <tt>domain</tt> and <tt>range</tt> among the classes and assign them to this property.
+For the other half of these new properties, creates a <tt>DatatypeProperty</tt>: selects randomly a <tt>domain</tt> among the classes and assign it to this property. <tt>range</tt> is set to <tt>xsd:string</tt>.
+The alignment is not modified.
+<br /><b>Priority:</b> 5
+
+</dd><dt>FlattenLevel</dt><dd>
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.FlattenLevel</tt>
+<br /><b>Parameters:</b> levelFlattened=<i>n</i>
+<br /><b>Result:</b> removes all classes of level <i>n</i> (their subclasses are directly connected to their superclasses).
+<br /><b>Functional description:</b>
+% This code should be fully rewrited!
+For each class of that level,
+	attaches, directly, its subclasses to its superclasses (including constraints, i.e., anonymous <tt>Restriction</tt> classes);
+	attaches, directly, its instances to its superclasses;
+% checkClassesRestrictions
+	converts all restrictions involving it (<tt>allValuesFrom</tt>, <tt>someValueFrom</tt>) with its superclasses; 
+% changeDomainRange
+	converts all property restrictions involving it (<tt>domain</tt>, <tt>range</tt>) with its superclasses;
+	remove the class.
+Updates the alignment by suppressing all correspondences involving the suppressed classes.
+<br /><b>Priority:</b> 6
+
+</dd><dt>SuppressHierarchy</dt><dd> 
+<b>Class:</b> <tt>fr.inrialpes.exmo.align.gen.alt.SuppressHierarchy</tt>
+<br /><b>No parameter</b>%noHierarchy
+<br /><b>Result:</b> suppresses all subclasses relations (but to <tt>owl:Thing</tt>)
+<br /><b>Functional description:</b> Suppresses all <tt>subClassOf</tt> assertions towards other named classes (assertions towards <tt>Restriction</tt>s remain).
+The alignment is not modified.
+<br /><b>Priority:</b> 7
+</dd>
 </dl>
-</p>
 
 <h3>Adding new alterators</h3>