Set up a test repository and run the ci-package-builder pipeline on it
by committing new changes, creating merge requests and monitoring the
resulting pipelines.

The test currently checks:
* submitting non-release changes and landing them
* submitting release commits and landing them
* blocking commits to frozen stable branches

This pipeline needs some extra setup:
* a tests/dash> repository, forked from pkg/target/dash>, where we
  force-push changes, create merge requests and monitor pipelines
* the `GITLAB_CI_USER`, `GITLAB_CI_PASSWORD`, `OSC_USERNAME` and
  `OSC_PASSWORD` CI variables to be set on tests/dash>, matching what it
  is used on the pkg/ projects
* the `TEST_GITLAB_AUTH_TOKEN` CI variable to be set on this repository
  to access the GitLab APIs used to issue MRs and monitor pipelines, and
  to push changes via git to the tests/dash> repository
* the `TEST_OSCRC` CI variable to be set on this repository to prepare
  the branch projects where the upload jobs will be tested

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Introduce the `$OBS_PREFIX` variable to upload to alternative OBS
projects from the pipeline.

This is useful when forking a packaging project as uploads can then be
directed to a user home project branched from one of the main Apertis
projects, like `home:em:branches:apertis:v2021dev3:target`: set the
`OBS_PREFIX` variable to `home:em:branches:` in the project's CI/CD
settings panel and the pipeline will pick it up.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Rather than hardcoding `apertis` as the first part of the OBS project
names, derive it from the branch name.

For instance, packages built on the `apertis/v2020-security` branch will
get uploaded to `apertis:v2020:security:$COMPONENT'.

This simplifies customization by product teams since the upload
destination is now implied by the name of the branches on which they
have enabled the upload jobs.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Back out commit dea8d068 "Make the packaging branch customizable"
since we're facing issues with the way GitLab does variables matching in
the `rules:` conditions when storing the regex in a variable as well.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit dea8d068 switched the regex for the packaging branches from a
literal to a variable to ease customization.

Accordingly to the documentation the behaviour should be the same:
https://docs.gitlab.com/ee/ci/variables/README.html#storing-regular-expressions-in-variables



Unfortunately this seems to hit a bug when the variable to be matched is
empty: when checking against a literal the check fails as expected, when
checking against a variable it succeeds.

That is, if you have the rules below, `only-on-mrs` is executed on
pipelines not tied to a MR, with `$CI_MERGE_REQUEST_TARGET_BRANCH_NAME`
being empty.

```
only-on-mrs:
  script:
    - echo hello $CI_MERGE_REQUEST_TARGET_BRANCH_NAME!
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ $BRANCH_REGEX
      when: always
    - when: never

only-on-mrs-but-with-a-literal-regex:
  script:
    - echo hello $CI_MERGE_REQUEST_TARGET_BRANCH_NAME!
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^master$/
      when: always
    - when: never
```

Work around this by explicitly checking for a merge request.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit 3be093c0 missed some nuance about the differences between
`rules:` and `only:` which resulted in the pipeline being run twice on
MRs for no good reason.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit a293f819 broke the pipeline since it was not ignoring the
error code when checking for an existing tag.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit 3be093c0 ended up breaking everything: while switching from
the `only:` keyword I dind't introduce the actual `rules:` keyword so
the `if:` ended up directly attached to the job, causing much
unhappiness in the CI.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

What `tally-revision` does can be replaced by a single `git describe`
invocation as other places of the pipeline already do.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Make it easier for downstreams and product teams to customize the
packaging branch.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `rules:` directive gives us more flexibility which will be useful
later to make the packaging branch customizable for downstreams.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

We can have scenario where we have packages pulled in from:
* Debian Testing/Unstable and others
* Debian's git development repositories

This MR attemps to document and clarify the steps involed for such
special cases.

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Emanuele Aina authored 4 years ago and Frederic Danis committed 4 years ago

Make a step toward making the pipeline work out of the box on downstream
infrastructures by picking the address of the current GitLab server
instance from the GitLab CI/CD predefined variables.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 4 years ago and Frederic Danis committed 4 years ago

Make a step toward making the pipeline work out of the box on downstream
infrastructures by picking the address of the Docker image registry from
the GitLab CI/CD predefined variables.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Because the gitlab rules cannot be applied on an empty git repository.
Similarly, the gitlab rules make an assumption about default branches,
ci configuration file, protected branches and such; for which we need to
populate the git repository with the actual content

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

When merging large numbers of merge requests some builds get stuck in
the `scheduled` state for a while, waiting for a OBS builder to be free.

Gracefully handle that case by polling as we do for states that are
marked dirty, waiting to be recomputed.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Fix a missing `%` in the format string used to report the final build
result when monitoring OBS builds.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

To determine whether the build has been successful we check the code
printed by `osc result` and `grep` for `succeeded`.

However, this means that if the result is not `succeeded` the pipeline
gives no indication about what was the actual code.

To address that, show the full information from `osc results` and check
it only later.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use the CSV output format of `osc results` to ignore values that are
marked "dirty" because the internal OBS queues have not processed the
update.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Stream the build log and propagate the build results from OBS, so that
developers get integrated feeback directly on their GitLab pipeline.

The approach chosen has the drawback of keeping a full worker slot busy
doing almost nothing, simply streaming the log from OBS and propagating
the result.

Builds for larger packages can take more than 7h, and it is not
acceptable to keep a normal runner slot busy for that.

The best way to handle such issue is to deploy a separate runner that
only runs these monitoring jobs using the appropriate tags.
Since each job only consumes a negligible amount of resources a large
amount of slots can be configured, in a way that long builds do not
block other jobs.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This reverts commit 87ef0c7c.

The changes introduced with this commit added the feature to link OBS
builds to Gitlab CI.

In the change, we have:

obs:
  stage: OBS
  needs:
    - upload
  trigger:
    strategy: depend
    include:
      - artifact: obs.yml
        job: upload

which "needs" upload. Now, as per the limitation of "needs" usage,
it is bound to fail if it points to a job not instantiated

See: https://gitlab.com/help/ci/yaml/README#requirements-and-limitations


for details

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Stream the build log and propagate the build results from OBS, so that
developers get integrated feeback directly on their GitLab pipeline.

The approach chosen has the drawback of keeping a full worker slot busy
doing almost nothing, simply streaming the log from OBS and propagating
the result.

Builds for larger packages can take more than 7h, and it is not
acceptable to keep a normal runner slot busy for that.

The best way to handle such issue is to deploy a separate runner that
only runs these monitoring jobs using the appropriate tags.
Since each job only consumes a negligible amount of resources a large
amount of slots can be configured, in a way that long builds do not
block other jobs.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Expand the `ci-obs-upload` script directly in the job.

This will make it easier re-use the information it computes like the
final OBS project, for instance to monitor the build results.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

To properly handle the `linux` package build, the `debian/control` file
needs to be re-generated on the fly to reflect our customizations like
disabling some build flavors.

So far it shipped a custom pipeline, but that's problematic as it can
go out of sync as it indeed did with the recent changes in this
repository, yielding errors due to the sequence having changed, stages
no longer matching, and artifacts being saved in different locations.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Allow submitting merges to stable branches if they come from an
`apertis/*-{updates,security}` branch: for instance allow a merge to
`apertis/v2020` if it comes from `apertis/v2020-updates`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Add small comments describing the intent of the rules for the job
freezing MRs to stable branches.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `tag-release` got confused when adding a commit that was not
changing `debian/changelog` and then manually triggering a pipeline.

Triggering a pipeline manually means that `only: changes:` is ignored,
so `tag-release` is run: since we're not exactly on a tagged commit
the job will try to create a new tag only to discover that the tag
already exists, pointing to previous commit.

To avoid that, rework the job to:

1. skip on UNRELEASED as it did before
2. fail on PLEASE SUMMARIZE as it did before
3. fetch a pre-made build if we're sitting on a tagged commit and return
4. if the changelog points to an existing tag, skip
5. run the actual build and tag if none of the previous checks matched

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit 31726673 "Automatically tag and upload if the changelog does
not say UNRELEASED" broke non-release uploads since the merged upload
job only sourced artifacts from the `tag-release` job.

To fix that, add the `build-source` in the `dependencies:` with
`tag-release` and disambiguate the `_build` directories from the two
stages to avoid confusing `ci-obs-upload` by having two `.dsc` files.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Currently the pipeline builds and uploads sources to the :snapshots
OBS project, and then the tagging and upload to the actual OBS
project needs to be triggered manually by the developer.

Now that things seem to work quite reliably we can let the pipeline
do the tag and final upload step on its own unless the top entry
in the `debian/changelog` file has `UNRELEASED` as the target
distribution.

Before this change, `tally-revision` skipped snapshot uploads for
already-tagged versions.
This was to avoid submitting everything to snapshots when creating
branches while branching a release.
Since the other steps were manual, they just were left pending.

Now that we have a single upload stage we can't use `tally-revision`
without breaking fast-forward backports, where e.g. version 0.1co1 gets
landed to v2021dev2 and then we want to propagate it unchanged to v2020.
From the pipeline perspective, there's no way to distinguish that case
(where we want to upload) from the case where we branch v2021dev3 from
v2021dev2 (where we want to skip uploading).

In this version what happens when branching is that:
1. stage build get skipped
1. stage release runs, and simply checks out the sources with
   pristine-lfs
1. stage uploads tries to re-upload to OBS what OBS already has with
   `osc dput`, which will likely return an error, marking the pipeline
   failed

This means that we'd do a bit more useless work during branching
compared to now, but not too much.

Ideally branch creation should skip ci, but there's no option in the
API to do so (we use the API to avoid checking out every single
repository):
https://gitlab.com/gitlab-org/gitlab/-/issues/26422#note_314606108



To avoid the failed pipelines we may enhance `ci-obs-upload` in a
followup MR to return success when the upload is a no-op.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

With the previous commit I ended up freezing only the development (like
`apertis/v2021dev2`) or updates (like `apertis/v2020-updates`) branches
and not the stable ones (like `apertis/v2020`), so let's try to fix
that again.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Rework once again the `freeze-stable-branches` job to let people submit
merge requests to secondary branches.

For instance if there's a topic branch named topic/whatever and
a MR is submitted from a wip/user/foo branch we don't want to complain.

We only want to complain once topic/whatever is submitted to merge
to apertis/v2020 rather than going through apertis/v2020-updates.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use `rules:` to make GitLab run the conditional check and skip the
pipeline completely when there's no reason to complain.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Sjoerd Simons authored 5 years ago and Andrej Shadura committed 5 years ago

gbp uses git-archive during build and potentially other places;
git-archive can do some substitute in files during archiving (e.g.
setting the tag or commit id). Which is not what we want when exporting
pristine source, so disable it globally

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

Ritesh Raj Sarraf authored 5 years ago and Andrej Shadura committed 5 years ago

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Let MRs to be issued against `$RELEASE-updates` and `$RELEASE-security`
branches, the current regex was too strict.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit bd9e1697 "Automatically freeze stable branches" uses the
wrong YAML multiline syntax and since it trims newlines it produces an
invalid shell snippet, breaking merge pipelines with this error:

    $ if echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep -Ev '(dev[0-9]|pre)$' then # collapsed multi-line command
    /bin/bash: eval: line 124: syntax error near unexpected token `fi'

Switch from `>` to `|` to preserve newlines in the snippet.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>