Commit 4bd68fa9 authored by GILLES Sebastien's avatar GILLES Sebastien
Browse files

#1360 - #1393 Add the guideline to use macOS machine (and provide Yaml to skip...

#1360 - #1393 Add the guideline to use macOS machine (and provide Yaml to skip it if no credentials to Nostromo). Add the part explaining how to set up Yaml to use in gitlab.
parent c48a46c5
Pipeline #70593 passed with stages
in 26 minutes and 7 seconds
<!-- toc -->
- [Introduction](#introduction)
- [Basic set up of the VM](#basic-set-up-of-the-vm)
* [VM choice on CI platform](#vm-choice-on-ci-platform)
* [Update Ubuntu](#update-ubuntu)
* [Connect the additional disk](#connect-the-additional-disk)
* [Install and set-up Docker](#install-and-set-up-docker)
* [Install and set-up gitlab-runner](#install-and-set-up-gitlab-runner)
* [Fix communication error](#fix-communication-error)
- [Set up the Yaml file to use](#set-up-the-yaml-file-to-use)
- [Runners in Gitlab interface](#runners-in-gitlab-interface)
* [Runner with docker executor](#runner-with-docker-executor)
+ [Introduction](#introduction-1)
+ [Setting up your Docker executors](#setting-up-your-docker-executors)
- [Activate specific runner](#activate-specific-runner)
- [Create new Docker runners on MoReFEM VMs](#create-new-docker-runners-on-morefem-vms)
* [Example: on VM Docker1](#example-on-vm-docker1)
* [The 5 others VMs](#the-5-others-vms)
* [Runner with shell executor (on macOS)](#runner-with-shell-executor-on-macos)
<!-- tocstop -->
# Introduction
Gitlab-CI relies on a relatively simple configuration:
......@@ -24,6 +29,14 @@ So I have defined very basic VMs (with the procedure described [here](Internal/S
As you shall see, the procedure is rather fast: you should have 7 (Docker) runners defined in around 10 minutes
# Set up the Yaml file to use
By default, if CI is activated for your projects Gitlab assumes it is at the root of the projet and is named .gitlab-ci.yml. In MoReFEM I chose to put it elsewhere (so that they appear more explicitly in the project and several may be defined) so you need to specify the path:
* Go in _Settings_ / _CI/CD_ / _General pipelines_.
* In _Custom CI config path_, choose the one you wish.
The most usual is _ExternalTools/Gitlab-CI/gitlab-ci.yml_, but if you do not have access to a macOS machine _ExternalTools/Gitlab-CI/gitlab-ci-no-macos.yml_ is more fitting.
# Runners in Gitlab interface
......@@ -88,7 +101,7 @@ Then choose:
- Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
Copy/password paste the choice on ![Gitlab page](Images/RunnerCredential.png).
Copy/paste password the choice on ![Gitlab page](Images/RunnerCredential.png).
- Please enter the gitlab-ci token for this runner:
......@@ -134,8 +147,36 @@ ssh ci@docker6.ci
## Runner with shell executor (on macOS)
macOS can't run in a Docker container, so we will need to use shell executors on machines. None are currently set up (there is currently an issue at hand on CI platform for macOS VMs), but it will be added in the next MoReFEM tag (probably using M3DISIM MacPro).
macOS can't run in a Docker container, so we will need to use shell executors on machines. A shell executor simply means the job will run directly on the machine; so you need to connect yourself regularly on this machine to update it.
There is currently only one such executor on M3DISIM machine _Nostromo_; you need to be part of the team to be able to use it. If you don't have access to it, you may use the _gitlab-ci-no-macos.yml_ provided which skips the related job in your Gitlab project (see [here](#set-up-the-yaml-file-to-use)).
If you have credential to _Nostromo_:
* Log in with your account.
* Change account and choose CI (same credential as for Linux VM).
* Run _gitlab-runner_ (without _sudo_) and choose:
- Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
Copy/paste password just as for the Linux case.
- Please enter the gitlab-ci token for this runner:
Same as Linux case.
- Please enter the gitlab-ci description for this runner:
You may just repeat the name of the machine... or really put whatever you want (you may change it afterwards anyway).
- Please enter the gitlab-ci tags for this runner (comma separated):
Choose _macos_.
- Please enter the executor:
Choose _shell_.
A shell executor simply means the job will run directly on the machine; so you need to connect yourself regularly on this machine to update it.
stages:
- build_and_test
- check_warnings
# - analysis
# - documentation
# - deploy
# Todo: Doxygen doc should be published with pages.
.build_template_linux: &build_template_linux
stage: build_and_test
image: registry.gitlab.inria.fr/morefem/thirdpartycompilationfactory/${OS}-${COMPILER}-${MODE}
cache:
key: "cache_${CI_COMMIT_REF_SLUG}_${OS}-${COMPILER}-${MODE}-${LIB_NATURE}-${IS_ONLY_ONE_LIB}"
untracked: true
paths:
- build
artifacts:
name: "artifact_${CI_COMMIT_REF_SLUG}_${OS}-${COMPILER}-${MODE}-${LIB_NATURE}-${IS_ONLY_ONE_LIB}"
expire_in: 8 hours
paths:
- build/compilation.log
script:
- mkdir -p build
- cd build && python ../cmake/Scripts/configure_cmake.py --cache_file=../cmake/PreCache/linux.cmake --cmake_args="-G Ninja" --third_party_directory=/opt --mode=${MODE} --library_type=${LIB_NATURE} --morefem_as_single_library=${IS_ONLY_ONE_LIB}
- ninja |& tee compilation.log
- ninja test # as artifact are limited in size, I can't unfortunately use a separate stage...
.build_template_macos: &build_template_macos
stage: build_and_test
tags:
- macos
cache:
key: "cache_${CI_COMMIT_REF_SLUG}_${OS}-${COMPILER}-${MODE}-${LIB_NATURE}-${IS_ONLY_ONE_LIB}"
untracked: true
paths:
- build
artifacts:
name: "artifact_${CI_COMMIT_REF_SLUG}_${OS}-${COMPILER}-${MODE}-${LIB_NATURE}-${IS_ONLY_ONE_LIB}"
expire_in: 8 hours
paths:
- build/compilation.log
script:
- mkdir -p build
- source activate Python3
- cd build && python ../cmake/Scripts/configure_cmake.py --cache_file=../cmake/PreCache/macos_apple_clang.cmake --cmake_args="-G Ninja" --third_party_directory=/Volumes/Data/ci/opt/${MODE} --mode=${MODE} --library_type=${LIB_NATURE} --morefem_as_single_library=${IS_ONLY_ONE_LIB}
- ninja 2>&1 | tee compilation.log
- ninja test # as artifact are limited in size, I can't unfortunately use a separate stage...
.run_doxygen_template: &run_doxygen_template
stage: build_and_test
image: ubuntu:latest
artifacts:
name: "artifact_run_doxygen_${CI_COMMIT_REF_SLUG}_${DOX}"
expire_in: 2 hours
paths:
- Documentation/Doxygen/${OUTPUT_FILE}
before_script:
- apt-get update -y
- apt-get install -y doxygen graphviz libjs-mathjax
script:
- cd Documentation/Doxygen
- doxygen ${DOX}
.check_compilation_warning_template: &check_compilation_warning_template
stage: check_warnings
image: alpine:latest
before_script:
- apk add --update python python-dev && rm -rf /var/cache/apk/*
# dependencies: in the instantiations!
script:
- python Scripts/Tools/find_warning_in_compilation_log.py --log build/compilation.log
allow_failure: true
.check_doxygen_warning_template: &check_doxygen_warning_template
stage: check_warnings
image: alpine:latest
artifacts:
name: "artifact_check_warning_doxygen_${CI_COMMIT_REF_SLUG}_${DOXYGEN_OUTPUT_FILE}"
when: on_failure
expire_in: 2 hours
paths:
- filtered_doxygen_log.txt
before_script:
- apk add --update python python-dev && rm -rf /var/cache/apk/*
# dependencies: in the instantiations!
script:
- python Scripts/Tools/find_warning_in_doxygen_log.py --log Documentation/Doxygen/${DOXYGEN_OUTPUT_FILE}
allow_failure: true
# .test_template: &test_template
# stage: check_warnings
# image: registry.gitlab.inria.fr/morefem/thirdpartycompilationfactory/${OS}-${COMPILER}-${MODE}:latest
# cache:
# key: "${CI_COMMIT_REF_SLUG}_${OS}-${COMPILER}-${MODE}-${LIB_NATURE}-UniqueLib_${IS_ONLY_ONE_LIB}"
# untracked: true
# paths:
# - build
# script:
# - cd build && ctest
build-Ubuntu-gcc-debug-shared-several_libs:
<<: *build_template_linux
variables:
OS: "ubuntu"
COMPILER: "gcc"
MODE: "debug"
LIB_NATURE: "shared"
IS_ONLY_ONE_LIB: "False"
check_warning_Ubuntu-gcc-debug-shared-several_libs:
<<: *check_compilation_warning_template
dependencies:
- build-Ubuntu-gcc-debug-shared-several_libs
variables:
OS: "ubuntu"
COMPILER: "gcc"
MODE: "debug"
LIB_NATURE: "shared"
IS_ONLY_ONE_LIB: "False"
build-Ubuntu-gcc-release-static-one_lib:
<<: *build_template_linux
variables:
OS: "ubuntu"
COMPILER: "gcc"
MODE: "release"
LIB_NATURE: "static"
IS_ONLY_ONE_LIB: "True"
check_warning_Ubuntu-gcc-release-static-one_lib:
<<: *check_compilation_warning_template
dependencies:
- build-Ubuntu-gcc-release-static-one_lib
variables:
OS: "ubuntu"
COMPILER: "gcc"
MODE: "release"
LIB_NATURE: "static"
IS_ONLY_ONE_LIB: "True"
build-Fedora-clang-debug-static-one_lib:
<<: *build_template_linux
variables:
OS: "fedora"
COMPILER: "clang"
MODE: "debug"
LIB_NATURE: "static"
IS_ONLY_ONE_LIB: "True"
check_warning_Fedora-clang-debug-static-one_lib:
<<: *check_compilation_warning_template
dependencies:
- build-Fedora-clang-debug-static-one_lib
variables:
OS: "fedora"
COMPILER: "clang"
MODE: "debug"
LIB_NATURE: "static"
IS_ONLY_ONE_LIB: "True"
build-Fedora-clang-release-shared-one_lib:
<<: *build_template_linux
variables:
OS: "fedora"
COMPILER: "clang"
MODE: "release"
LIB_NATURE: "shared"
IS_ONLY_ONE_LIB: "False"
check_warning_Fedora-clang-release-shared-one_lib:
<<: *check_compilation_warning_template
dependencies:
- build-Fedora-clang-release-shared-one_lib
variables:
OS: "fedora"
COMPILER: "clang"
MODE: "release"
LIB_NATURE: "shared"
IS_ONLY_ONE_LIB: "False"
build-Fedora-gcc-debug-static-one_lib:
<<: *build_template_linux
variables:
OS: "fedora"
COMPILER: "gcc"
MODE: "debug"
LIB_NATURE: "static"
IS_ONLY_ONE_LIB: "True"
check_warning_Fedora-gcc-debug-static-one_lib:
<<: *check_compilation_warning_template
dependencies:
- build-Fedora-gcc-debug-static-one_lib
variables:
OS: "fedora"
COMPILER: "gcc"
MODE: "debug"
LIB_NATURE: "static"
IS_ONLY_ONE_LIB: "True"
complete_doxygen:
<<: *run_doxygen_template
variables:
DOX: "complete_dev_guide.dox"
OUTPUT_FILE: "complete.log"
check_warning_complete_doxygen:
<<: *check_doxygen_warning_template
dependencies:
- complete_doxygen
variables:
DOXYGEN_OUTPUT_FILE: "complete.log"
# Pages doesn't work as invoked from a subgroup (not supported yet in gitlab)
# pages:
# stage: deploy
# cache:
# key: "$CI_COMMIT_REF_SLUG_doxygen"
# untracked: true
# paths:
# - Documentation/Doxygen
# before_script:
# - mkdir -p public
# tags:
# - linux
# - debug
# script:
# - cd Documentation/Doxygen
# - mv Complete/* ../../public/
# artifacts:
# paths:
# - public
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment