GitLab

Since this might break some cases relying on opaqueness of
type variables occuring only in return types, opaqueness
annotations are now allowed there as well.

- move common lexing functions to util/lexlib.mll
- move and rename Typing.create_user_tv to Ty.tv_of_string

Will be supported in programs as soon, as we enable lambdas.

Finally, it is simpler to treat "higher-order" applications
in Dterm and Dexpr, since we can analyze the dtypes.

Instead, pass the optional vsymbol representing the result.
Now formulas-under-patterns do not need to be typechecked
separately, so the previous commit is partially reverted.

The (pattern -> fmla) format is used in Dexpr for postconditions.

Also, restructure the mutually recursive calls, with a massive
change of indentation (use "git show -w" to view this commit).

If the function is defined, the correct opacity for fresh type
variables in the result will be computed automatically. If the
function is abstract, then every fresh type variable is opaque
by default, as no information leak is possible anyway.

TODO: impose the same restriction in programs

Also:

- Make [Highord.pred 'a] an alias for [Highord.func 'a bool],
rename [Highorg.(@!)] to [(@)], remove [Highorg.(@?)], remove
the quantifiers [\!] and [\?] and only leave [\] which is the
only true lambda now;

- Allow mixing bool and Prop in logic, Dterm will introduce
coercions where necessary (trying to minimize the number of
if-then-else in the term context).

We store in every lsymbol a new integer field ls_constr,
equal to zero if the lsymbol is not a constructor, and equal
to the number of constructors of the lsymbol's type otherwise.
It is allowed to declare or define an lsymbol with ls_constr > 0
as an ordinary function (otherwise algebraic type elimination
wouldn't work - though we might still check this in theories),
but it is forbidden to use a wrong ls_constr in algebraic type
definitions.

The ghostness of a match expression is now determined as follows:

If at least one branch expression is ghost,
  then the match is ghost;
else if there is only one branch,
  then the match is not ghost;
else if the matched expression is ghost,
  then the match is ghost;
else if at least one pattern matches a ghost field
        against a constructor with ls_constr > 1
  then the match is ghost;
else
  the match is not ghost.

We do just enough to recognize obvious non-ghost cases, and
make no attempt to handle redundant matches or to detect
exhaustive or-patterns in subpatterns.

In programs, but also in pure theories, it is not safe to compare
arbitrary types. For example, if we have a record with ghost fields,
a comparison may produce different results before and after ghost
code elimination. Even for pure types like 'map' or 'set', it is
unlikely that the result of logical equality will be the same as
the result of OCaml structural equality on the implemented type.

This commit makes the first step towards fixing this issue.
We proceed in the following way:

1. Every lsymbol (pure function or predicate symbol) carries
   a subset of type variables of its signature, called "opaque
   type variables". By marking a type variable 'a opaque in an
   lsymbol's signature, the user guarantees that this lsymbol
   can be implemented without ever comparing values of type 'a.
   In other words, this is a promise not to break into a type
   variable.

   The corresponding syntax is: "predicate safe (x y : ~'a)".

   All type variables in undefined symbols are non-opaque,
   unless annotated otherwise. Non-opaque is the default
   to keep the change conservative.

   Opacity of type variables in defined symbols is inferred
   from the definition. If the definition violates a given
   opacity annotation, an exception is raised. Notice that
   we only check definitions in _theory_ declarations. One
   can define an lsymbol in a _task_ in a way that violates
   opacity. We cannot forbid it, because various elimination
   transformations would replace safe operations (such as
   matching) with equalities. This is not a problem, since in
   the pure logical realm of provers opacity is not required
   One exception would be Coq, whose transformation chain must
   never perform such operations.

   All type variables in inductive predicates are non-opaque.
   Indeed, we can redefine equality via an inductive predicate.
   [TODO: find safe forms of inductive definitions and implement
   more reasonable restrictions.]

   All type variables in constructors and field symbols are opaque.

   It is forbidden to instantiate an opacity-preserving symbol
   with an opacity-breaking one in a clone substitution.

2. Similar type variable tracking is implemented for program symbols.
   Type variables in the signature of a "val" are non-opaque unless
   annotated otherwise. Opacity of type variables in defined symbols
   is inferred from the definition, and an exception is raised, if
   a given annotation is violated.

   The internal mechanism of tracking is different: the "eff_compar"
   field in effects contains the type variables that occur under
   equality or any other opacity-breaking operation. In this respect,
   our API is inconsistent between lsymbols and psymbols: the former
   asks for the opaque tvsymbols, the latter requires us to fill the
   spec with "comparison effects" for the non-opaque ones. [TODO:
   add the "~opaque" argument to create_psymbol and make the WhyML
   core fill the effect under the hood.]

   Every time an lsymbol or a psymbol is applied in a program,
   we check the substitution into its signature's type variables.
   If a non-opaque type variable is instantiated with a program type,
   an exception is raised. [TODO: be more precise and reject only
   types with ghost and model components - being mutable, private,
   or carrying an invariant doesn't conflict with equality.]

   Notice that we do not allow to compare program types even in
   the ghost code. This is not needed if we only consider the
   problems of the code extraction, but _might_ be necessary,
   if we also want to protect Coq realisations (see below).

This commit fixes the immediate problem of breaking the ghost
guarantees when equality or some other opacity-breaking lsymbol
is applied in a program to a type with ghost or "model" parts.

This leaves the problem of code extraction for programs that
compare complex types such as maps or sets (Coq driver is
affected by this, too, I guess). The next step is to provide
annotations for problematic type constructors. A declaration
"type ~map 'a 'b" would mean "logical equality on this type
is likely to be different from the structural equality on any
implementation of this type - therefore do not apply equality
to it: neither in programs (because this can't be implemented),
nor in pure functions (because they are extracted, too, and
because this can't be realized with Leibniz equality in Coq)."
[TODO: discuss and implement.]

[TODO: mb choose better term for "opaque" and notation for ~'a.]

Also, ghost fields in algerbraic types are now accepted in programs.

As of now, "function", "predicate", "inductive", "val", "let", "fun",
and constructors in algebraic type declarations all accept the same
syntax for parameters. In "function", "predicate", "inductive", "val",
and constructors in algebraic type declarations, binders without
colons are treated as type expressions:

    function f int (list bool) (int,real) (ghost bool)

is syntactic sugar for

    function f (_: int) (_: list bool) (_: (int,real)) (ghost _: bool)

In "let" and "fun", single unqualified idents are treated as parameter
names whose types are to be inferred, other binders without colons are
treated as type expressions:

    let f int (list bool) (int,real) (ghost bool)

is syntactic sugar for

    let f (int: ?) (_: list bool) (_: (int,real)) (ghost bool: ?)

Anonymous binders ("_") are accepted only if their type is specified.