Dylan Aïssi authored 6 months ago and Walter Lozano committed 3 months ago

This is especially useful during a rebase where the copyright
tooling is updated and the copyrght file needs to be refreshed.
Some packages don't have any update in the next Debian release,
but the pipeline fails only because we have to manually refresh
the copyright file, so let's it create a MR to save us time.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

In preparation for the rebase, make our pipeline more flexible by tuning
the subset of packages to run license scan. In this way, in case of many
unexpected failures we can easily disable the scanning temporarily
to unblock the rebase.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Since debug packages can be useful for developers without
access to OBS, an option is available to re-enable the
capture.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This ensures v2025 will continue to follow Bookworm when
we will start the rebase on Trixie.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This happens when creating merge request to backport changes, like for
instance creating a merge request from apertis/v2025pre to apertis/v2024-updates.
In this case, the tool may cancel a legitimate pipeline running on
apertis/v2024-updates.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This is only a workaround for missing features in GitLab to disable
some unneeded pipelines.

Since this tool requires packages (apertis-dev-tools) and dependencies
(python-gitlab) only available in an Apertis docker image and not
in a Debian image, the image used for prepare-build-env is now
an hardcoded Apertis image (because we cannot use DOCKER_IMAGE
at this step since the processing to compute the value of this
variable is done during this job).

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 7 months ago and Walter Lozano committed 6 months ago

OBS commits take place in two steps:

    uploading the files
    actually creating the commit

so if it's cancelled before (1) completes, then (2) never happens.

So, it seems safe to make the upload job interruptible on non-release branches.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 7 months ago and Walter Lozano committed 6 months ago

build-source: allow build-source to be interruptible only for pipelines running on a non-release branches

build-source was set to non interruptible to avoid incomplete pushes
and uploads which are done only for release builds. Having this job
non interruptible for non-release builds can lead to a pipeline not
cancelled when scan-licenses push a copyright update. Thus, we have
two pipelines running at the same time.

This issue does not always happen, but is more frequent for big
packages like linux or mesa.

 https://docs.gitlab.com/ee/ci/yaml/#rulesinterruptible



Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

When a changelog has the message "PLEASE SUMMARIZE", it also has UNRELEASED
for tageted distribution that means it triggers the previous checks and
the message saying:
"


 debian/changelog says PLEASE SUMMARIZE
Add a summary of the remaining downstream changes
"
is never displayed, so move it first to get a more accurate error message.

Fix: f5901ebe

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 9 months ago and Walter Lozano committed 7 months ago

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 9 months ago and Walter Lozano committed 7 months ago

v2022 is EOL and keeping it here lead to creation
of useless MRs agasint v2022 when importing packages
from Bullseye for v2023.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

During the folding MR are created but they don't trigger builds as they
have little value and can be very time consuming. Since the are no
binaries to check, skip the job.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Otherwise, the script fails and complains about an already existing folder.

This change is only useful for local runs for instance for debugging
purposes.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This generates a warning and will be removed in Python 3.13:

./tests/test_packaging_pipelines.py:355: DeprecationWarning: pathlib.Path.__enter__() is deprecated and scheduled for removal in Python 3.13; Path objects as a context manager is a no-op

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

See: https://github.com/python-gitlab/python-gitlab/commit/c4f5ec6c615e9f83d533a7be0ec19314233e1ea0



Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This was enabled in fd3ce686 to allow linux updating
jobs to run on "upstream/" branches.
Unfortunately, they never worked as expected since the
jobs definition is in a local yaml file which is not
available in "upstream/" branches.
Those jobs require need rethinking.

Meanwhile, disable pipelines on those branches since
there is no value.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

In order to avoid wasting resources, move the PLEASE SUMMARIZE check
earlier, to the build env stage, where other changelog checks are
performed.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

To make the workflow cleaner and avoid wasting resources move the check
for UNRELEASED changelog earlier, to the build env stage, when a similar
check is done for the main branch.

In the process also, enforce this check only MR targeting main branches
allowing developers to build packages in other circumstances.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Dylan Aïssi authored 9 months ago and Walter Lozano committed 8 months ago

This is a common mistake and require more work afterwards since
we need to create a new MR with distribution correctly set.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Vignesh Raman authored 2 years ago and Dylan Aïssi committed 9 months ago

If the changelog contains an incorrect date, the lintian pipeline will
pass on a wip or proposed branch. However, when the pipeline runs on
an Apertis branch, it will fail due to a debian-changelog-has-wrong-day-of-week
lintian warning.

When the pipeline runs on a wip or proposed branch, two entries are
added to debian/changelog: a snapshot entry with a correct date
generated by dch, and an OBS build entry with the same date as the
snapshot entry. When the pipeline runs on an Apertis branch, only the
OBS build entry is added with an incorrect date, which triggers a
debian-changelog-has-wrong-day-of-week lintian warning. Since lintian
checks only the latest entry in the changelog, this warning is emitted
only on Apertis branches.

To ensure that lintian complains for wip or proposed branches, reuse
the wrong date from git commit for the snapshot changelog entry.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>

Koushik Tm authored 1 year ago and Walter Lozano committed 11 months ago

When running integration tests on Merge Request a pipeline is triggered
which requires a token. Currently a special token is defined for this use
case, but this is not really required as Gitlab provides short live token
CI_JOB_TOKEN. The only restriction is that the user who runs the job
have enough permission to run the image pipeline.

In the future this implementation will change to build custom images
but it the meantime it simplifies the configuration.

Signed-off-by: Koushik Tm <external.koushik.tm@in.bosch.com>
Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Following the rationale in c562a835 Rename stages and jobs rename also
the test metadata to avoid confusions.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

After adding additional test support during merge requests confusions can
arise. In an effort to mitigate this issue, rename stages and jobs
to follow a common pattern.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Now that v2024.0 has been released it is time to mark it as stable in order
to run all the required checks when processing packages updates.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Apertis doesn’t have yq, so it’s easier to use Debian rather than port
yq to all supported releases.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Dylan Aïssi authored 1 year ago and Walter Lozano committed 1 year ago

Now, we have two releases based on Bookworm: v2024 and v2025*

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 1 year ago and Walter Lozano committed 1 year ago

Otherwise apertis/v2022 is processed BEFORE apertis/v2023,
that means the pipeline creates MR against v2022 instead of
v2023. In this case, we have to manually change the targeted
branch from v2022 to v2023 and then do the backport from v2023
to v2022.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Since downstreams most likely will have a differnet osname pass it as
argument to the script that creates the jobs.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Downstreams can have different instances for apertis-test-cases with different
image repositories and supported boards. Thus, a package maintainer should be
able to select which instance is needed to test its package.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

pull-updates: check CI_COMMIT_BRANCH for passing --backports or --proposed-updates to apertis-pkg-pull-updates

Previously, apertis-pkg-pull-updates pulled by default versions
from backports or proposed-updates repositories. Since it was
more confusing than heplful because not part of our usual workflow,
these repo are now optional. This change allows to control which
repo needs to be checked by running the job on a
'debian/*-backports' or 'debian/*-proposed-updates' branch.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Dylan Aïssi authored 1 year ago and Andrej Shadura committed 1 year ago

 
For packages that are not built on all architectures, obs-capture captures empty directories. 
It’s creating issues with our lintian job, and since we don’t need empty folders anyway, 
we can drop them here.

Emanuele Aina authored 1 year ago and Andrej Shadura committed 1 year ago

Due to the way `osc getbinaries` works, we're currently capturing the
package sources once for every architectures.

Since the `build-source` job is already capturing the source files,
let's drop them from here.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>