Commit ad9297c0 authored by POTTIER Francois's avatar POTTIER Francois

Release 20200211.

parent 056b435c

Too many changes to show.

To preserve performance only 1000 of 1000+ files are displayed.
# Developer Guide
This guide is intended for new Menhir developers, and is supposed explain how
things work.
## How Menhir is Built
Menhir is built by `dune`. The build process takes place in several
* The library MenhirLib, whose source files reside in `lib`, is built.
This is the runtime support library for the parsers generated by
Menhir in `--table` mode.
* The library MenhirSdk, whose source files reside in `sdk`, is built.
This library allows reading automaton description (`.cmly`) files.
* A preliminary version of Menhir, referred to as "stage 1", is built.
In order to build Menhir's parser, `ocamlyacc` is used.
The parser and driver in `src/stage1` are used.
* The final version of Menhir, referred to as "stage 2", is built.
In order to build Menhir's parser, (the stage 1 version of) Menhir itself is used.
The parser and driver in `src/stage2` are used.
* As a sanity check, another version of Menhir,
referred to as "stage 3", is built.
In order to build Menhir's parser, the stage-2 Menhir is used.
The parser and driver in `src/stage2` are used again.
We check that a fixed point is reached, that is,
the stage-2 and stage-3 versions of Menhir
behave in the same way when applied to Menhir's grammar.
## Testing
To run all tests except the speed test, type `make test` in the root
To run the speed test, type `make speed` in the root directory.
The tests in the directory `test/static` test that Menhir seems to run properly,
but do not test the generated parser.
* The subdirectory `test/static/good` contains a number of correct `.mly` files.
We check that Menhir accepts these files and
compare the output of `menhir --only-preprocess` against an expected output.
We do not check that Menhir actually produces a working parser.
* The subdirectory `test/static/bad` contains a number of incorrect `.mly` files.
We checks that Menhir rejects these files
and produces the expected error message.
The tests in the directory `test/dynamic` test the generated parsers.
See the [README](test/dynamic/ file there.
The demos in the directory `demos` also contain tests.
## Releasing
Prior to making a release, please take the following actions:
* Make sure `` has been properly updated.
* Make sure that you are on the `master` branch and have
committed everything. No uncommitted files should remain.
* Run `make test` and `make versions` to make sure that Menhir can be compiled
and passes all tests under all supported versions of OCaml.
* Run `make speed` and have a look at the performance figures to
make sure that they are still in the right ballpark.
* Test the `opam` package by running `make pin`
(possibly in a dedicated switch, so as to avoid clobbering your
current installation of Menhir).
If `make pin` succeeds, then its effect can be undone by `make unpin`.
To create a release, run `make release`. This creates a release and commits it
to a fresh release branch. It also commits a copy of the documentation to the
master branch. The effects of this command are local; nothing is pushed or
If you are happy with the outcome of `make release`, you can then proceed to
the following three steps:
* Run `make publish` to push the new release to ``.
* Run `make export` to upload the documentation to Menhir's home page.
* Run `make opam` to create and publish a new `opam` package.
If you are not happy with the outcome of `make release`, you can undo it
by typing `make undo`.
## About the Module Ordering
Some toplevel modules have side effects and must be executed in the
following order:
| Module | Task |
| --------------------- | ---- |
| Settings | parses the command line |
| PreFront | reads the grammar description files |
| TokenType | deals with `--only-tokens` and exits |
| Front | deals with `--depend`, `--infer`, `--only-preprocess`, and exits |
| Grammar | performs a number of analyses of the grammar |
| Lr0 | constructs the LR(0) automaton |
| Slr | determines whether the grammar is SLR |
| Lr1 | constructs the LR(1) automaton |
| Conflict | performs default conflict resolution and explains conflicts |
| Invariant | performs a number of analyses of the automaton |
| Interpret | deals with `--interpret` and exits |
| Back | produces the output and exits |
A few artificial dependencies have been added in the code in order
to ensure that this ordering is respected by `ocamlbuild`.
# -------------------------------------------------------------------------
# Compilation and installation rules.
.PHONY: all install uninstall clean
@ dune build @install
# note: @install is smaller than @all,
# as it does not include the tests nor the stage3 executable.
@ dune uninstall
@ dune clean
install: all
@ dune install
# -------------------------------------------------------------------------
# The rest of this Makefile helps perform tests and prepare releases.
# These commands are intended to be used by Menhir's developers.
# Require bash.
SHELL := bash
# Prevent the built-in bash cd from displaying information.
export CDPATH=
# -------------------------------------------------------------------------
# Testing.
# [make test] runs the tests found in the test/ and demos/ directories.
.PHONY: test
@ dune build --display short @test
# [make data] extracts statistics and performance data out of the files
# produced by [make test]. Be careful: the timing data is definitely not high
# quality, because every test is run only once and because the tests are run
# in parallel on a heavily-loaded machine. Furthermore, this data is
# machine-dependent. The data is written to analysis/data.csv.
# A test that caused a TIMEOUT is excluded from the data.
# On MacOS, we require gsed instead of sed.
SED=$(shell if [[ "$$OSTYPE" == "darwin"* ]] ; then echo gsed ; else echo sed ; fi)
.PHONY: data
data: test
@ echo "Collecting data (using $(SED))..." && \
echo "name,mode,terminals,nonterminals,lr0states,lr1states,lr1time" > analysis/data.csv && \
directory=_build/default/test/static/src && \
successful=0 && timedout=0 && \
for timings in $$directory/*.out.timings ; do \
name=$${timings%.out.timings} ; \
out=$$name.out ; \
name=`basename $$name` ; \
if grep --quiet "TIMEOUT after" $$out ; then \
((timedout++)) ; \
else \
((successful++)) ; \
mode=`$(SED) -n -e "s/^The construction mode is \([a-z\-]\+\)./\1/p" $$out` ; \
terminals=`$(SED) -n -e "s/^Grammar has \([0-9]\+\) terminal symbols./\1/p" $$out` ; \
nonterminals=`$(SED) -n -e "s/^Grammar has \([0-9]\+\) nonterminal symbols, among which [0-9]\+ start symbols./\1/p" $$out` ; \
lr0states=`$(SED) -n -e "s/^Built an LR(0) automaton with \([0-9]\+\) states./\1/p" $$out` ; \
lr1states=`$(SED) -n -e "s/^Built an LR(1) automaton with \([0-9]\+\) states./\1/p" $$out` ; \
lr1time=`$(SED) -n -e "s/^Construction of the LR(1) automaton: \(.*\)s/\1/p" $$timings` ; \
echo "$$name,$$mode,$$terminals,$$nonterminals,$$lr0states,$$lr1states,$$lr1time" >> analysis/data.csv ; \
fi \