From 0bdb75c1766fa86ad9c81cc73da192f3cff1a6d7 Mon Sep 17 00:00:00 2001 From: Emanuele Aina <emanuele.aina@collabora.com> Date: Thu, 27 Jan 2022 17:07:20 +0100 Subject: [PATCH] README: Actually document what this repository is about Add some proper documentation: * add a short general introduction * describe the mapping to OBS projects, which has been a common topic of discussion recently * describe the extension points Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com> --- README.md | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 70 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 5c8bcaf7..617f7d5b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,65 @@ -# The document is now available [here](https://www.apertis.org/guides/component_guide/) +# Packaging CI/CD workflow + +This repository contains the pipeline definition that handles the packaging workflow. +[`ci-package-builder.yml`](ci-package-builder.yml). + +This pipeline handles the main packaging workflow: +1. converting the contents in git to source packages (`.dsc` and linked files) +2. uploading the source packages to OBS +3. monitor the build on OBS and relaying the build logs and results + +It also deals with pulling updates from the upstream distribution: +1. import upstream packages to the matching git branch, for instance `debian/bullseye-security` +2. merge the changes into the appropriate branch, for instance `apertis/v2022-security` +3. scan the sources for inappropriate licenses, for instance complaining when + trying to import GPL-3 files in a package to be landed in the `target` component + +The [workflow guide](https://www.apertis.org/guides/component_guide/) documents +the most common operation. + +To use this workflow: +* ensure that CI/CD is enabled on your packaging project +* set a few CI/CD variables at the project or group level: + * `GITLAB_CI_USER`, `GITLAB_CI_PASSWORD`: credentials with push access to the + git repositories to push tags and packaged sources on the `pristine-lfs-source` branch + * `OSC_CONFIG`: settings and credentials to upload sources to OBS (legacy runner) + * `OBS_SERVER`, `OBS_USER`, and `OBS_PASSWORD`: settings and credentials to upload sources to OBS (custom runner) +* ensure your git repository follows the [expected structure](https://www.apertis.org/guides/component_structure/) +* set the configuration file path to +* `ci-package-builder.yml@infrastructure/ci-package-builder` in the CI/CD + settings of your packaging project + +## Where the pipeline is going to upload sources + +The pipeline uses the details below to define on which OBS project the sources +are going to be uploaded: + +* the `OBS_PREFIX` CI/CD variable, for instance `foo` +* the branch name, for instance `bar/baz-quux`, which get split in a branch prefix, `bar` and a suffix, `baz-quux` +* the component name from the `debian/apertis/component` file, for instance `frob` + +With that, the pipeline will upload to the `foo:bar:baz:quux:frob` OBS project. + +Customizing `bar` also requires to set the `OSNAME=bar` CI/CD variable. + +> âš ï¸ Warning! +> +> The GitLab issue [#35438](https://gitlab.com/gitlab-org/gitlab/-/issues/35438) +> "Regexp CI/CD variables are evaluated as strings" actually prevents this from +> working as intended and forking this repository is needed to customize +> the osname. The issue is expected to be fixed in GitLab 14.8. + +## Extension points + +The pipeline can be extended with child pipelines by adding the pipeline +definitions in your package: + +* `debian/apertis/local-pre-source-build-gitlab-ci.yml` runs before attempting + to build the source package from the content in git, providing a good place + to run unit tests or other checks +* `debian/apertis/local-post-source-build-gitlab-ci.yml` runs after the source + package has been built, before it gets uploaded to OBS, providing a good + place to run [Lintian](https://wiki.debian.org/Lintian) checks # Injecting an external APT repository to CI Build environment @@ -43,9 +104,14 @@ Below are some screenshots on how these variables and their values are fed into   -# Updating Existing Components +## Low-level manual operations + +Some special cases that are not handled by the automation need to be dealt with +some manual operations, described below. + +Most users can safely ignore this section. -## Manually Updating Components from Debian +### Manually Updating Components from Debian Upstream updates are usually handled automatically by the [`ci-package-builder.yml`](https://gitlab.apertis.org/infrastructure/ci-package-builder/) @@ -133,7 +199,7 @@ operations. Follow the instructions provided by the above command to create a merge request -## Manually Updating Components from a Newer Debian Release +### Manually updating components from a newer Debian release There are circumstances, when we deviate from the default upstream. This usually happens when: -- GitLab