Mentions légales du service

Skip to content
Snippets Groups Projects

Use Alcotest for the test suite.

Merged MARTINOT Olivier requested to merge omartino/inferno:alcotest into master
1 unresolved thread

Merge request reports

Merge request pipeline #344339 passed

Merge request pipeline passed for d6cadeab

Approval is optional

Merged by SCHERER GabrielSCHERER Gabriel 3 years ago (Nov 16, 2021 8:08pm UTC)

Merge details

  • Changes merged into master with 0cb6d6ea.
  • Deleted the source branch.

Pipeline #344833 passed

Pipeline passed for 0cb6d6ea on master

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • The motivation for this change are as follows:

    • To get rid of the need for manual logging in the testing code (Alcotest captures stdout by itself, so tests are slightly simpler to write)
    • (superficial) to get nicer-looking test output

    The MR does get rid of the Log module, but the "captures stdout" part only works half-nicely, as Alcotest output capture does not work too well with QCheck-based tests (it will print the output of all random attempts for a failed random test). The output is still nicer, and we think that the use of a standard test framework is a reasonable choice (we don't want to have to think too much when extending the testsuite), so I am planning to review the MR for implementation quality and then merge.

353 352 arb
354 353 prop
355 354 ) pairs in
356 QCheck_runner.set_seed 0;
357 let debug_shrink = None (* use (Some stdout) for a shrinking trace *) in
358 let _ = QCheck_runner.run_tests ~colors:true ~verbose:true ~debug_shrink tests in
359 Printf.printf
360 "In total, %d out of %d terms (%.1f%%) were considered well-typed.\n%!"
361 !count_success !count_total
362 (float !count_success /. float !count_total *. 100.);
363 Printf.printf "No problem detected.\n%!"
355
356 (* Note that we need to initialize the random state on each test. *)
  • @omartino is it possible to check, for example by introducing a bug in the code or in one of the random tests, that the random seed is correctly set? (As in: the behavior of running the random testsuite several time should be exactly the same, with the same tests generated).

  • Author Maintainer

    I tried to add a "russian roulette" bug relying on the random seed, and it stopped the random test suite after exactly the same amount of tests. Is there a particular reason why you worry about it ?

  • Please register or sign in to reply
  • added 1 commit

    • 47f44961 - Use Alcotest for the test suite.

    Compare with previous version

  • The CI now fails with a real error:

    File "client/test/dune", line 3, characters 29-44:
    3 |   (libraries client alcotest qcheck-alcotest)
                                     ^^^^^^^^^^^^^^^
    Error: Library "qcheck-alcotest" not found.
  • added 1 commit

    • d6cadeab - Use Alcotest for the test suite.

    Compare with previous version

  • SCHERER Gabriel mentioned in commit 0cb6d6ea

    mentioned in commit 0cb6d6ea

  • This is now merged, thanks!

  • Please register or sign in to reply
    Loading