Code coverage

Here we get a taste of some dynamics analysis. After the tests are executed, the code coverage tools (mainly gcov) show which lines of code have been involved in the execution.

We are going to see how to improve the pipeline as follows :

graph LR;
  subgraph build
  B(Build);
  end
  subgraph test
  C(Unit tests);
  D(Warnings tests);
  end
  subgraph coverage
  E(Coverage);
  end
  B-->C
  B-->D
  C-->E
  D-->E

gcov and lcov

Coverage with gcov needs the GNU C++ compiler and the --coverage compilation option.

gcov generates raw output. We are going to use the lcov utility for the post- processing and the generation of html pages and the gcovr utility in order to present parseable results to GitLab CI / CD.

A CMake function add_test_with_coverage is provided in the cxx/cmake/utils.cmake module to simplify the whole setup.

This function, when used in place of the standard CMake function add_test function, provides coverage support through a new make coverage target.

Remark: You can see all make targets using : make help.

Exercise

1. Add the coverage flag --coverage with the add_cxx_compiler_flag CMake function in the tests subdirectory CMakeLists.txt.
2. Use the CMake function add_test_with_coverage.
3. Run a test with make coverage.
4. Open the file index.html under cxx/TestSphere__testVolume directory with a web browser to see the results.

make coverage output shall look like :

Scanning dependencies of target check
Running tests...
Test project /home/user/gitlabciintroduction/cxx/build
    Start 1: TestVolume
1/1 Test #1: TestVolume .......................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.00 sec
[  0%] Built target check
Scanning dependencies of target TestSphere__testVolume.info
[100%] Coverage report is generated in folder /home/user/gitlabciintroduction/cxx/TestSphere__testVolume
Capturing coverage data from /home/user/gitlabciintroduction/cxx/build
Found gcov version: 7.3.0
Scanning /home/user/gitlabciintroduction/cxx/build for .gcda files ...
Found 2 data files in /home/user/gitlabciintroduction/cxx/build
Processing tptest.dir/TestSphere.cpp.gcda
Processing tptest.dir/testMain.cpp.gcda
  ignoring data for ...
Finished .info-file creation
Reading data file TestSphere__testVolume.info
Found 4 entries.
Found common filename prefix "/home/user/gitlabciintroduction"
Writing .css and .png files.
Generating output.
Processing file cxx/Sphere.hpp
Processing file cxx/tests/testMain.cpp
Processing file cxx/tests/TestSphere.hpp
Processing file cxx/tests/TestSphere.cpp
Writing directory view page.
Overall coverage rate:
  lines......: 87.0% (40 of 46 lines)
  functions..: 100.0% (16 of 16 functions)
[100%] Built target TestSphere__testVolume.info
Scanning dependencies of target coverage
[100%] Built target coverage

GitLab CI coverage

GitLab CI allows coverage

1. Add a coverage stage in the .gitlab-ci.yml file.
2. Add a job_coverage that make coverage.
3. Do not forget the dependencies on job_build.
4. Add an artifact on cxx/TestSphere__testVolume directory.
5. Add the following regex :

  coverage: '/lines\.+:\s(\d+.\d+\%)/'

The regex will capture 87.0 from the output:

  lines......: 87.0% (40 of 46 lines)

Commit and push the .gitlab-ci.yml file and check the job_coverage succeds. Check on the GitLab web site, in CI / CD, Jobs that the coverage percentage is appearing (coverage is not reported in Pipelines section). Check you can download the artifact and check in a web browser the coverage.

Beware: The artifacts require to be ignored by git for not to face conflicts between jobs. Check cxx/coverage/ folder is present in the .gitignore file.

Home | C++ Home | << C++ Previous - GitLab CI pipelines | >> C++ Next - Merge code to cover