... | ... | @@ -36,9 +36,7 @@ First of all to avoid having to type login/password each type, the best is to us |
|
|
On your local computer:
|
|
|
|
|
|
* git relies on ssh for the connexion to the gitlab's git server
|
|
|
|
|
|
* ssh uses your ssh private key to authenticate to the gitlab's git server
|
|
|
|
|
|
* your ssh private key is stored in your home, on your local computer's hard disk. This ssh private needs to be crypted, so that if your local computer is breached or stollen, the key cannot be used. This is the reason why it is crypted with a so-called "passphrase". This passphrase is actually a password, but you are encouraged to use something longer and more complex than a password (beacuse if this passphrase is "cracked", it can lead to a breach on many servers, so from the security point of view, it is critical)
|
|
|
|
|
|
Up to now, everything works correctly, but you have to type your ssh passphrase each time you connect by ssh, thus at each git operation involving the remote git server (push, pulls, ...)
|
... | ... | @@ -50,56 +48,59 @@ This is the ssh-agent that allows not having to enter the passphrase at each ssh |
|
|
By default ssh client is installed with a command line ssh-agent, but modern operating systems / graphical environments come with integrated "keyrings" which can act as ssh-agents (and can as well manage other kind of secrets, such as passwords, gpg keys, etc.):
|
|
|
|
|
|
* for OS X: Keychain
|
|
|
|
|
|
* for linux / gnome: gnome-keyring
|
|
|
|
|
|
* for linux / kde: kde-wallet
|
|
|
|
|
|
* for other linux desktop environments, or for windows: ?
|
|
|
|
|
|
These "keyrings" work as follows: When you login on your local computer, your login password is automatically used to unlock the secrets stored in the keyring, of which the ssh keys are part.
|
|
|
These "keyrings" work as follows: When you login on your local computer, your login password is automatically used to unlock the secrets stored in the keyring, of which the ssh keys are part.
|
|
|
|
|
|
Thus to be able to perform connected git operations (push, pull, ...) without having to type any password you need:
|
|
|
|
|
|
* to use git over ssh
|
|
|
|
|
|
* to have a ssh keypair protected by a passphrase
|
|
|
|
|
|
* to use a command line or graphical ssh-agent
|
|
|
|
|
|
Note: it is also possible to do passwordless operations with git over http by using a personal access token:
|
|
|
|
|
|
* create a personal access token in your gitlab user settings, with permissions "read_repository" (only these permissions).
|
|
|
|
|
|
* when cloning the repository, use customize the URL as follows: https://<username>:<token>@gitlab.inria.fr/<end_of_url>
|
|
|
* when cloning the repository, use customize the URL as follows: https://:@gitlab.inria.fr/\<end_of_url\>
|
|
|
|
|
|
## How to create/manage external (non-Inria) user accounts
|
|
|
|
|
|
Gitlab is a service which may be made available for external users.
|
|
|
Their account is managed via the dedicated Inria external account portal.
|
|
|
Gitlab is a service which may be made available for external users. Their account is managed via the dedicated Inria external account portal.
|
|
|
|
|
|
The procedure and detailed information are provided [here](https://doc-si.inria.fr/pages/viewpage.action?pageId=1873251).
|
|
|
|
|
|
## How to convert a user account from internal to external
|
|
|
When a user is leaving Inria, the gitlab account cannot be used anymore. Two situations can occur:
|
|
|
|
|
|
When a user is leaving Inria, the gitlab account cannot be used anymore. Two situations can occur. In both cases, information needed in the ticket is :
|
|
|
|
|
|
* the name/firstname/login of the person leaving Inria,
|
|
|
* the external email that will be used as identifier and for password recovery,
|
|
|
* the name of the sponsor at Inria and
|
|
|
* until when the account is needed (maximum is 3 years).
|
|
|
|
|
|
### The user is still at Inria
|
|
|
|
|
|
Before leaving Inria:
|
|
|
* open a ticket requesting to convert the account, providing the external email that will be used as identifier and for password recovery, the name of the sponsor at Inria and until when the account is needed (maximum is 3 years).
|
|
|
|
|
|
* open a ticket requesting to convert the account, providing the information mentioned above.
|
|
|
* the new account will be available when the Inria account is deactivated, which occurs when the Human Resources service closes the Inria account, approximately 3 months after the departure.
|
|
|
* then, when the account has been converted, the user can ask for password recovery using the new identifier and provide a new password.
|
|
|
|
|
|
### The user has already left Inria
|
|
|
|
|
|
* Only an Inria administrator of a project the user was contributing to can ask for the account conversion.
|
|
|
* The administrator open a ticket requesting to convert the account, providing the external email that will be used as identifier and for password recovery, the name of the sponsor at Inria and until when the account is needed (maximum is 3 years).
|
|
|
* The administrator open a ticket requesting to convert the account, providing the information mentioned above.
|
|
|
* the new account will be available when the Inria account is deactivated, which occurs when the Human Resources service ends the Inria account, approximately 3 months after the departure.
|
|
|
* then, when the account has been converted, the user can ask for password recovery using the new identifier and provide a new password.
|
|
|
|
|
|
|
|
|
## How to convert a user account from external to internal
|
|
|
External users can one day be hired by Inria and ask for an internal gitlab account. In this situation the external account can be converted into an internal one with proper inria email and iLDAP parameters. To do so the user must open a ticket (https://helpdesk.inria.fr/, section gitlab) requesting to convert the existing external account into an internal one, providing:
|
|
|
* the current external account login (e.g. x-Jdoe),
|
|
|
* the inria email
|
|
|
|
|
|
External users can one day be hired by Inria and ask for an internal gitlab account. In this situation the external account can be converted into an internal one with proper inria email and iLDAP parameters. To do so the user must open a ticket (https://helpdesk.inria.fr/, section gitlab) requesting to convert the existing external account into an internal one, providing:
|
|
|
|
|
|
* the current external account login (e.g. x-Jdoe),
|
|
|
* the inria email
|
|
|
* and the iLDAP login.
|
|
|
|
|
|
## How to activate a commit email hook
|
... | ... | @@ -114,38 +115,40 @@ As documented in https://gitlab.inria.fr/help/user/permissions project members w |
|
|
|
|
|
## How to add users of an INRIA ildap group (eg. team members) to a gitlab group
|
|
|
|
|
|
Open a ticket, specifying the INRIA ildap group name, the gitlab group name, and the access level (GUEST, REPORTER, DEVELOPER, MASTER, OWNER) that you wish to give to added members. We will run a script which adds *existing gitlab users* matching the ildap group members to the gitlab group. Be aware that inria members need to connect at least one time to gitlab for their account to be created.
|
|
|
Open a ticket, specifying the INRIA ildap group name, the gitlab group name, and the access level (GUEST, REPORTER, DEVELOPER, MASTER, OWNER) that you wish to give to added members. We will run a script which adds _existing gitlab users_ matching the ildap group members to the gitlab group. Be aware that inria members need to connect at least one time to gitlab for their account to be created.
|
|
|
|
|
|
Provided you are in the INRIA network and have access to ldap://ildap.inria.fr, you can search for a groupname with a command such as:
|
|
|
|
|
|
$ ldapsearch -x -H "ldap://ildap.inria.fr" -b "ou=groups,dc=inria,dc=fr" | grep -i <team name>
|
|
|
```plaintext
|
|
|
$ ldapsearch -x -H "ldap://ildap.inria.fr" -b "ou=groups,dc=inria,dc=fr" | grep -i <team name>
|
|
|
```
|
|
|
|
|
|
For verification purposes you can check the list of ildap group members with a command such as:
|
|
|
|
|
|
$ ldapsearch -x -H "ldap://ildap.inria.fr" -b "ou=People,dc=inria,dc=fr" "(inriaGroupMemberOf=cn=<ldap group name>,ou=groups,dc=inria,dc=fr)"
|
|
|
```plaintext
|
|
|
$ ldapsearch -x -H "ldap://ildap.inria.fr" -b "ou=People,dc=inria,dc=fr" "(inriaGroupMemberOf=cn=<ldap group name>,ou=groups,dc=inria,dc=fr)"
|
|
|
```
|
|
|
|
|
|
## How to use the continuous integration (CI) service?
|
|
|
|
|
|
Besides using Inria's Continuous Integration platform [ci.inria.fr](https://ci.inria.fr), you can use gitlab's integrated CI pipelines, see the [official documentation](https://docs.gitlab.com/ee/ci/) and the complete [gitlab-ci yaml reference](https://docs.gitlab.com/ee/ci/yaml/).
|
|
|
Besides using Inria's Continuous Integration platform ci.inria.fr, you can use gitlab's integrated CI pipelines, see the [official documentation](https://docs.gitlab.com/ee/ci/) and the complete [gitlab-ci yaml reference](https://docs.gitlab.com/ee/ci/yaml/).
|
|
|
|
|
|
Concerning the machines setup to run the tests you can either use existing "shared runners" (to be used with docker images), see section _Use existing shared runners_ hereafter, or you will need to install [gitlab-runner](https://docs.gitlab.com/runner/install/) on your specific machine and then register the runner with your project, see section _Installing your own specific runners_.
|
|
|
|
|
|
In addition, if you look for "real life" gitlab-ci examples, please visit this dedicated group [gitlabci_gallery](https://gitlab.inria.fr/gitlabci_gallery). It contains several subgroups and git repositories showing some interesting key features of gitlab-ci and possible integrations with external tools/platforms (ci.inria.fr, terraform, github, a supercomputer, etc).
|
|
|
In addition, if you look for "real life" gitlab-ci examples, please visit this dedicated group [gitlabci_gallery](https://gitlab.inria.fr/gitlabci_gallery). It contains several subgroups and git repositories showing some interesting key features of gitlab-ci and possible integrations with external tools/platforms (ci.inria.fr, terraform, github, a supercomputer, etc).
|
|
|
|
|
|
### Enabling CI on a gitlab project
|
|
|
|
|
|
* Go to your project's settings
|
|
|
* In the *General* *Permissions* tab, search for "CI/CD", enable and do not forget to click on Save changes
|
|
|
* Then a *CI / CD* tab should now appears in the settings
|
|
|
* In this *CI / CD* tab, you will find the url and the registration token you will need later, when adding a **specific** runner.
|
|
|
* In the _General_ _Permissions_ tab, search for "CI/CD", enable and do not forget to click on Save changes
|
|
|
* Then a _CI / CD_ tab should now appears in the settings
|
|
|
* In this _CI / CD_ tab, you will find the url and the registration token you will need later, when adding a **specific** runner.
|
|
|
|
|
|
### Use existing shared runners
|
|
|
|
|
|
If you don't want to manage runners yourself you can use some existing ones provided by ci.inria.fr where jobs can be executed in docker containers.
|
|
|
You will see the shared runners available in the *Settings* *CI/CD* *Runners* *Shared runners* (right column).
|
|
|
Please read the available [documentation](https://ci.inria.fr/doc/page/gitlab/) for more information.
|
|
|
The shared runners require to use [tags](https://docs.gitlab.com/ee/ci/yaml/#tags), e.g.
|
|
|
```
|
|
|
If you don't want to manage runners yourself you can use some existing ones provided by ci.inria.fr where jobs can be executed in docker containers. You will see the shared runners available in the _Settings_ _CI/CD_ _Runners_ _Shared runners_ (right column). Please read the available [documentation](https://ci.inria.fr/doc/page/gitlab/) for more information. The shared runners require to use [tags](https://docs.gitlab.com/ee/ci/yaml/#tags), e.g.
|
|
|
|
|
|
```plaintext
|
|
|
myjob:
|
|
|
tags: ['ci.inria.fr', 'small']
|
|
|
image: docker:19.03.12
|
... | ... | @@ -155,29 +158,28 @@ myjob: |
|
|
|
|
|
### Installing your own specific runners
|
|
|
|
|
|
To perform the actual build, you need to install GitLab Runner. Runners can run anywhere : you can use a virtual machine on [ci.inria.fr](https://ci.inria.fr) to host your GitLab runner.
|
|
|
To perform the actual build, you need to install GitLab Runner. Runners can run anywhere : you can use a virtual machine on ci.inria.fr to host your GitLab runner.
|
|
|
|
|
|
See :
|
|
|
|
|
|
* <https://inria-ci.gitlabpages.inria.fr/doc/page/web_portal_tutorial/> to create a slave on Inria's CI platform and access it (you can ignore the jenkins related parts)
|
|
|
* <https://docs.gitlab.com/runner/install/> for the official documentation to install the runner on the vm(s) you've created.
|
|
|
|
|
|
* https://inria-ci.gitlabpages.inria.fr/doc/page/web_portal_tutorial/ to create a slave on Inria's CI platform and access it (you can ignore the jenkins related parts)
|
|
|
* https://docs.gitlab.com/runner/install/ for the official documentation to install the runner on the vm(s) you've created.
|
|
|
|
|
|
#### Installation example on a GNU/linux slave from ci.inria.fr ("ci" user account)
|
|
|
```
|
|
|
|
|
|
```plaintext
|
|
|
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"
|
|
|
sudo chmod +x /usr/local/bin/gitlab-runner
|
|
|
sudo gitlab-runner install --user=ci --working-directory=/builds
|
|
|
sudo gitlab-runner start
|
|
|
sudo gitlab-runner status # should return "service is running"
|
|
|
|
|
|
```
|
|
|
|
|
|
#### Installation example on a macOS slave from ci.inria.fr ("ci" user account)
|
|
|
|
|
|
**[Limitations on macOS](https://docs.gitlab.com/runner/install/osx.html#limitations-on-macos)** : The service needs to be installed from a Terminal window logged in as your current user (i.e., "ci" user account).
|
|
|
[**Limitations on macOS**](https://docs.gitlab.com/runner/install/osx.html#limitations-on-macos) : The service needs to be installed from a Terminal window logged in as your current user (i.e., "ci" user account).
|
|
|
|
|
|
```
|
|
|
```plaintext
|
|
|
sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64
|
|
|
sudo chmod +x /usr/local/bin/gitlab-runner
|
|
|
# Run the following commands as the "ci" user
|
... | ... | @@ -187,7 +189,8 @@ gitlab-runner status # should return "service is running" |
|
|
```
|
|
|
|
|
|
#### Installation example on a Windows slave from ci.inria.fr ("ci" user account)
|
|
|
```
|
|
|
|
|
|
```plaintext
|
|
|
# Follow https://docs.gitlab.com/runner/install/windows.html then when installing the service prefer the following to use the existing "ci" user account
|
|
|
gitlab-runner install -u ".\ci" -p "ci" -d "C:\Users\ci"
|
|
|
gitlab-runner start
|
... | ... | @@ -198,55 +201,60 @@ gitlab-runner status # should return "service is running" |
|
|
|
|
|
On the virtual machine where you installed the gitlab CI runner, run (as root or sudo if the gitlab-runner program has been installed with sudo, remove sudo from the following if not) :
|
|
|
|
|
|
```
|
|
|
```plaintext
|
|
|
$ sudo gitlab-runner register
|
|
|
```
|
|
|
Use the gitlab URL and the registration token found in the "Pipelines" page when asked.
|
|
|
|
|
|
In order to answer when asked
|
|
|
Use the gitlab URL and the registration token found in the "Pipelines" page when asked.
|
|
|
|
|
|
Please enter the gitlab-ci tags for this runner (comma separated):
|
|
|
In order to answer when asked
|
|
|
|
|
|
it is important to know that "gitlab-ci tags" are NOT git tags. The tags you enter here are useful to specify conditional execution in the configuration file .gitlab-ci.yml.
|
|
|
```plaintext
|
|
|
Please enter the gitlab-ci tags for this runner (comma separated):
|
|
|
```
|
|
|
|
|
|
it is important to know that "gitlab-ci tags" are NOT git tags. The tags you enter here are useful to specify conditional execution in the configuration file .gitlab-ci.yml.\
|
|
|
See :
|
|
|
|
|
|
* <https://docs.gitlab.com/ee/ci/yaml/#tags>
|
|
|
* and <https://docs.gitlab.com/ee/ci/yaml/#only--except>
|
|
|
* https://docs.gitlab.com/ee/ci/yaml/#tags
|
|
|
* and https://docs.gitlab.com/ee/ci/yaml/#only--except
|
|
|
|
|
|
for more details.
|
|
|
|
|
|
The next question
|
|
|
The next question
|
|
|
|
|
|
Whether to run untagged builds [true/false]:
|
|
|
```plaintext
|
|
|
Whether to run untagged builds [true/false]:
|
|
|
```
|
|
|
|
|
|
becomes clear when the "gitlab-ci tag" concept is understood.
|
|
|
|
|
|
When asked
|
|
|
When asked
|
|
|
|
|
|
```plaintext
|
|
|
Please enter the executor:
|
|
|
```
|
|
|
|
|
|
Please enter the executor:
|
|
|
you should answer `shell` (or `docker` but then see the related subsection below)
|
|
|
|
|
|
At the end of this step, your runner should appear in the *Settings > Pipelines* tab of your project.
|
|
|
At the end of this step, your runner should appear in the _Settings \> Pipelines_ tab of your project.
|
|
|
|
|
|
### Configurating CI tasks
|
|
|
|
|
|
Finally, configure the tasks to run by creating a .gitlab-ci.yml file at the root of your project.
|
|
|
Follow the official documentation at <https://docs.gitlab.com/ce/ci/yaml/> to create this file.
|
|
|
Finally, configure the tasks to run by creating a .gitlab-ci.yml file at the root of your project.\
|
|
|
Follow the official documentation at https://docs.gitlab.com/ce/ci/yaml/ to create this file.
|
|
|
|
|
|
### Troubleshooting gitlab runners
|
|
|
|
|
|
A few logging-related issues were found in older gitlab-runner versions. When using a dedicated (non-shared) runner,
|
|
|
you might want to ensure that your gitlab-runner version is up to date
|
|
|
A few logging-related issues were found in older gitlab-runner versions. When using a dedicated (non-shared) runner, you might want to ensure that your gitlab-runner version is up to date
|
|
|
|
|
|
If you have any issue with gitlab runner (for example, an online runner but no debug logs) during your pipeline execution,
|
|
|
the following [official documentation](https://docs.gitlab.com/runner/faq/) may help you.
|
|
|
For example, it may show you how to enable debug logging on your runner.
|
|
|
If you have any issue with gitlab runner (for example, an online runner but no debug logs) during your pipeline execution, the following [official documentation](https://docs.gitlab.com/runner/faq/) may help you. For example, it may show you how to enable debug logging on your runner.
|
|
|
|
|
|
### Using a docker executor
|
|
|
|
|
|
It has been reported that there are DNS issues with docker running on the INRIA CI's VMs (due to bad interaction between dnsmasq and docker, see https://stackoverflow.com/questions/49998099/dns-not-working-within-docker-containers-when-host-uses-dnsmasq-and-googles-dns). This can be solved by adding `network_mode = "host"` in the configuration. Eg. `/etc/gitlab-runner/config.toml`:
|
|
|
|
|
|
```
|
|
|
```plaintext
|
|
|
concurrent = 1
|
|
|
check_interval = 0
|
|
|
|
... | ... | @@ -268,25 +276,22 @@ shm_size = 0 |
|
|
|
|
|
### Cleaning the pipelines (artifacts, logs)
|
|
|
|
|
|
In the course of time the size of data generated for gitlab-ci pipelines may grow quickly and sometimes reaches dozens of GB.
|
|
|
The quantity of data stored on the gitlab's server for pipelines can be checked in the Settings -> Usage Quotas panel of the project, or in Build -> Artifacts.
|
|
|
For projects with more than 10GB of artifacts there is certainly something to do to reduce the disk storage:
|
|
|
In the course of time the size of data generated for gitlab-ci pipelines may grow quickly and sometimes reaches dozens of GB. The quantity of data stored on the gitlab's server for pipelines can be checked in the Settings -\> Usage Quotas panel of the project, or in Build -\> Artifacts. For projects with more than 10GB of artifacts there is certainly something to do to reduce the disk storage:
|
|
|
|
|
|
1) By cleaning old artifacts.
|
|
|
2) By disabling the "Keep artifacts from most recent successful jobs" in the Settings -> CI/CD -> Artifacts if not necessary because it keeps all job's artifacts (build, test, etc) of all the git refs (branches, tags, merge requests, ...) and this can cost a lot.
|
|
|
2) By disabling the "Keep artifacts from most recent successful jobs" in the Settings -\> CI/CD -\> Artifacts if not necessary because it keeps all job's artifacts (build, test, etc) of all the git refs (branches, tags, merge requests, ...) and this can cost a lot.
|
|
|
3) By changing the gitlab-ci jobs definitions, for example:
|
|
|
|
|
|
- use the [expire_in](https://docs.gitlab.com/ee/ci/yaml/#artifactsexpire_in) keyword to reduce the expiration time (30 days by default) of artifacts,
|
|
|
- use [cache](https://docs.gitlab.com/ee/ci/yaml/#cache) instead of artifacts when possible when jobs with dependencies can be executed on the same machine,
|
|
|
- generate fewer archives and lighter archives, keeping only what is strictly necessary.
|
|
|
|
|
|
To view and delete artifacts and logs there is for example this tool [gitlab-storage-analyzer](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-storage-analyzer).
|
|
|
This tool use the Gitlab's REST API to execute tasks on the project, thus you must get:
|
|
|
To view and delete artifacts and logs there is for example this tool [gitlab-storage-analyzer](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-storage-analyzer). This tool use the Gitlab's REST API to execute tasks on the project, thus you must get:
|
|
|
|
|
|
- a [Personal Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with __api__ access,
|
|
|
- the gitlab's project __id__, see the number in front of Project ID on the main page of your project.
|
|
|
- a [Personal Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with **api** access,
|
|
|
- the gitlab's project **id**, see the number in front of Project ID on the main page of your project.
|
|
|
|
|
|
Then you can clone it locally, install python-gitlab with pip
|
|
|
|
|
|
```sh
|
|
|
git clone https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-storage-analyzer.git
|
|
|
cd gitlab-storage-analyzer/
|
... | ... | @@ -295,37 +300,44 @@ pip install python-gitlab |
|
|
```
|
|
|
|
|
|
and start playing.
|
|
|
|
|
|
```sh
|
|
|
export GL_SERVER=https://gitlab.inria.fr
|
|
|
export GL_TOKEN=
|
|
|
export GL_PROJECT_ID=
|
|
|
python gitlab_storage_analyzer.py
|
|
|
```
|
|
|
|
|
|
This will list all the artifacts.
|
|
|
|
|
|
To only look for archives and exclude logs (called trace here) use `GL_EXCLUDE_FILE_TYPE`
|
|
|
|
|
|
```sh
|
|
|
export GL_EXCLUDE_FILE_TYPE="trace,metadata"
|
|
|
```
|
|
|
|
|
|
To only look for logs
|
|
|
|
|
|
```sh
|
|
|
export GL_EXCLUDE_FILE_TYPE="archive,metadata"
|
|
|
```
|
|
|
|
|
|
To only look for artifacts older than a number of seconds use `GL_THRESHOLD_AGE_SEC`
|
|
|
|
|
|
```sh
|
|
|
export GL_THRESHOLD_AGE_SEC=604800 # i.e. older than 1 week
|
|
|
```
|
|
|
|
|
|
Then to actually delete the filtered artifacts use `GL_DELETE_MODE`
|
|
|
|
|
|
```sh
|
|
|
export GL_DELETE_MODE=threshold # none by default
|
|
|
```
|
|
|
|
|
|
Unfortunately this script cannot be used to delete logs (only archives) associated with pipelines and which, yet, can be responsible for a large amount of the storage quota.
|
|
|
|
|
|
To remove logs of a pipeline there is no other way than deleting the pipeline.
|
|
|
The script [cleanup-gitlab-pipelines.sh](https://gitlab.inria.fr/siteadmin/doc/-/snippets/897) allows to delete 100 (max pagination size) pipelines older than a certain date:
|
|
|
To remove logs of a pipeline there is no other way than deleting the pipeline. The script [cleanup-gitlab-pipelines.sh](https://gitlab.inria.fr/siteadmin/doc/-/snippets/897 "cleanup-gitlab-pipelines") allows to delete 100 (max pagination size) pipelines older than a certain date:
|
|
|
|
|
|
```sh
|
|
|
export GL_SERVER=https://gitlab.inria.fr
|
|
|
export GL_TOKEN=
|
... | ... | @@ -333,10 +345,10 @@ export GL_PROJECT_ID= |
|
|
export GL_DELETE_BEFORE="2023-01-01"
|
|
|
./cleanup-gitlab-pipelines.sh
|
|
|
```
|
|
|
To delete more than 100 pipelines one has to repeat this operation several times.
|
|
|
|
|
|
After having doing that the storage quota can be recalculated on the page Settings -> Usage Quotas, button "Recalculate repository usage" and wait several minutes to get a refreshed page up-to-date (the result is not instantaneous).
|
|
|
To delete more than 100 pipelines one has to repeat this operation several times.
|
|
|
|
|
|
After having doing that the storage quota can be recalculated on the page Settings -\> Usage Quotas, button "Recalculate repository usage" and wait several minutes to get a refreshed page up-to-date (the result is not instantaneous).
|
|
|
|
|
|
## Git-LFS is not activated
|
|
|
|
... | ... | @@ -350,29 +362,32 @@ You are welcome to tell us if git-lfs is a need for your usage. Your requests wi |
|
|
|
|
|
This Inria Gitlab instance offers accounts both to Inria members (who login using their Inria LDAP credentials) and to external users (who can be invited/sponsored by Inria members through the external account portal: https://external-account.inria.fr ). The external (non-inria) users are limited to join existing projects, but cannot create projects on their own.
|
|
|
|
|
|
This limitation for external users has an important result: from the Gitlab point of view, forking a project *is* creating a project, thus external users cannot fork projects.
|
|
|
This limitation for external users has an important result: from the Gitlab point of view, forking a project _is_ creating a project, thus external users cannot fork projects.
|
|
|
|
|
|
As a result, the well-known MR (Merge Request) development model cannot be used directly with external users, but some workarounds can be used in some situations.
|
|
|
|
|
|
The MR development model works as follows:
|
|
|
|
|
|
- A restricted core development team is member of a project and has commit access to the development tree
|
|
|
- Some contributor starts contributing by forking the project ("fork" button). When their contribution is ready, they open a new merge request (in the "new" menu).
|
|
|
- Someone from the core development team can then accept (or refuse) the Merge Request to be merged into the main development tree
|
|
|
|
|
|
This development model has several benefits:
|
|
|
|
|
|
- It limits the access to the main development tree to a small group of trusted core developers.
|
|
|
- It allows the core development team to control finely which contributions to merge or not.
|
|
|
- In particular, it shields the project and forces all contributions to pass through a validation workflow, such as ensuring that the Continuous Integration passes, or that all intellectual property constraints are satisfied, before merging some development.
|
|
|
- With tools such as Github or Gitlab, the web user interface for merge requests is very user friendly.
|
|
|
|
|
|
So, here are some possible workarounds to this development model when using the Inria Gitlab with external contributors:
|
|
|
|
|
|
- Use pull-requests mechanisms at the git level, rather than at the Gitlab level. This is the way that the Linux kernel has been developed for a long time now. Instead of submitting a MR with the Gitlab interface, contributors send patches by email.
|
|
|
- For projects with few known external contributors, it is possible for Inria members to create forks on behalf of the external contributors, then transfer ownership of these forks to the contributors. Here is a possible worklow:
|
|
|
- create a new group to handle the forks of your external collaborators
|
|
|
- take care to remain the only owner/master of the group in order to control the creation/destruction of forks
|
|
|
- create a fork from the original project with as destination the newly created group. By default the name of the project will be the same as the original project, e.g. `https://gitlab.inria.fr/ForksOfFoo/bar`
|
|
|
- then rename this forked project through Settings -> General -> Advanced Settings, e.g. `https://gitlab.inria.fr/ForksOfFoo/bar-jdoe`
|
|
|
- finally add your collaborator master on that fork
|
|
|
- create a new group to handle the forks of your external collaborators
|
|
|
- take care to remain the only owner/master of the group in order to control the creation/destruction of forks
|
|
|
- create a fork from the original project with as destination the newly created group. By default the name of the project will be the same as the original project, e.g. `https://gitlab.inria.fr/ForksOfFoo/bar`
|
|
|
- then rename this forked project through Settings -\> General -\> Advanced Settings, e.g. `https://gitlab.inria.fr/ForksOfFoo/bar-jdoe`
|
|
|
- finally add your collaborator master on that fork
|
|
|
- For projects with few known and trusted external contributors, these contributors can be added to the project but develop in their own branches. There are some basic mechanisms in Gitlab to ensure they don't override their permissions: if these users have role developer, they won't be allowed to push to protected branches (master, by default). Details on Gitlab permissions can be found here: https://gitlab.inria.fr/help/user/permissions.md.
|
|
|
|
|
|
If these workarounds do not fit well with what you need and you absolutely want to use the standard MR development model with external contributors, the only solution is to use another tool than the Inria Gitlab.
|
... | ... | @@ -384,28 +399,35 @@ If these workarounds do not fit well with what you need and you absolutely want |
|
|
- There is a basic gitlab-pages example project here: https://gitlab.inria.fr/siteadmin/pages-example The generated web page is here: https://siteadmin.gitlabpages.inria.fr/pages-example/
|
|
|
- You can also have a look at https://gitlab.inria.fr/help/user/project/pages/getting_started/new_or_existing_website.md note that there are templates for many website generators.
|
|
|
- Inria gitlab does not support custom domains and certificates. They are required if you want to have a custom domain that you own point to a gitlab pages generated site (optionnaly in https) but Inria has explicitely forbidden this functionnality.
|
|
|
- The first time Gitlab-Pages are activated, a question is asked: *"Authorize GitLab Pages to use your account? An application called GitLab Pages is requesting access to your GitLab account. Please note that this application is not provided by GitLab and you should verify its authenticity before allowing access. This application will be able to: Access the authenticated user's API, Grants complete read/write access to the API, including all groups and projects, the container registry, and the package registry."*. This is strange, but normal. The Gitlab-Pages server is a separate server, and for situation where pages are restricted to project members, the Gitlab-Pages server must use the authentication infrastructure of the Gitlab server. So you need to answer yes.
|
|
|
- The first time Gitlab-Pages are activated, a question is asked: _"Authorize GitLab Pages to use your account? An application called GitLab Pages is requesting access to your GitLab account. Please note that this application is not provided by GitLab and you should verify its authenticity before allowing access. This application will be able to: Access the authenticated user's API, Grants complete read/write access to the API, including all groups and projects, the container registry, and the package registry."_. This is strange, but normal. The Gitlab-Pages server is a separate server, and for situation where pages are restricted to project members, the Gitlab-Pages server must use the authentication infrastructure of the Gitlab server. So you need to answer yes.
|
|
|
|
|
|
## Broken Links with Uploaded Files in a Project Wiki
|
|
|
|
|
|
The symptom is that when clicking on links to uploaded resources, an error 404 is thrown. The link follows the pattern `(/uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf)`.
|
|
|
In that case, check that the file `uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf` is not in the git repository of the wiki by:
|
|
|
The symptom is that when clicking on links to uploaded resources, an error 404 is thrown. The link follows the pattern `(/uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf)`. In that case, check that the file `uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf` is not in the git repository of the wiki by:
|
|
|
|
|
|
1. cloning the wiki repository (button at the upper right of the wiki);
|
|
|
2. checking with `ls uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf` whether the file is stored in the repository. If it not, then you are in the situation addressed by this FAQ entry.
|
|
|
|
|
|
### Explanation
|
|
|
The way the linked files are stored in the wiki has changed (cf. https://docs.gitlab.com/ee/user/project/wiki/#attachment-storage): previously to gitlab 11.3 release, the linked files of all the wikis were uploaded in a global /uploads directory. Since 11.3, they are uploaded to a uploads directory pertaining to the git repository of the wiki.
|
|
|
|
|
|
The way the linked files are stored in the wiki has changed (cf. https://docs.gitlab.com/ee/user/project/wiki/#attachment-storage): previously to gitlab 11.3 release, the linked files of all the wikis were uploaded in a global /uploads directory. Since 11.3, they are uploaded to a uploads directory pertaining to the git repository of the wiki.
|
|
|
|
|
|
But the files that were uploaded previously to 11.3 have not been moved to the wiki repositories, causing:
|
|
|
|
|
|
* 404 errors since it appears the global repository is not used anymore when gitlab follows the links inside the wiki;
|
|
|
* the files are not present when cloning the wiki repository, making the clone incomplete.
|
|
|
|
|
|
### Workarounds
|
|
|
|
|
|
If the file is not stored in the git repository of the wiki, two solutions exist:
|
|
|
|
|
|
#### Quick and Dirty Workaround
|
|
|
Change the `(/uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf)` links to `(https://gitlab.inria.fr/uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf)`.
|
|
|
|
|
|
Change the `(/uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf)` links to `(https://gitlab.inria.fr/uploads/4d0b0eecbe6f0bd95a96f3b90cf64fe3/my_file.pdf)`.
|
|
|
|
|
|
#### Long-term Solution
|
|
|
|
|
|
It requires to download and the upload again each linked file:
|
|
|
|
|
|
* the file can be downloaded using the link given in the previous section;
|
|
|
* then upload the file using the "attach a file" button at the bottom right of the wiki editor. |
|
|
\ No newline at end of file |