Use of `git restore` has been introduced in commit e954d2be
"ci-buildpackage: use wrap-and-sort as a workaround for odd packages".

It works on the Bullseye-based v2022dev2, but on older Buster-based
release `git restore` is not available.

Replace it with the old-style `git checkout`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Make interpolation work in a printed message by turning a plain string
to a proper f-string.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This reverts commit b26ad3fa "Explicitly set TAG to latest"
since it is causing every image that gets built, even the ones on
secondary wip branches, to be tagged as `latest` and thus affecting
production everywhere.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Andrej Shadura authored 4 years ago and Martyn Welch committed 4 years ago

This can happen if the source package has been renamed while still
generating a binary package with the same name. For example, Ruby
bundler is no longer a separate source package, but is shipped in
the rubygems package. Madison returns this for bundler:

---
bundler:
  2.2.5-2:
    bullseye:
      - all

The "source" pseudoarchitecture is missing from the list since it’s not
a source package in bullseye anymore.

Apertis: T7811

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Under early testing of ci-license-scan with docker image apertis/v2022dev2
the copyright.new file generated was missformated, putting the header Format
in the last line instead of the first one.

In order to fix the issue, force line buffering when the file is generated.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

If `--auto-merge` is not set it results in:

    Traceback (most recent call last):
      File "/usr/bin/apertis-pkg-merge-upstream-to-downstreams", line 146, in <module>
        main()
      File "/usr/bin/apertis-pkg-merge-upstream-to-downstreams", line 92, in main
        auto_merge = args.auto_merge.split(":")
    AttributeError: 'NoneType' object has no attribute 'split'

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `--auto-merge` flag was not meant to be mandatory.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Avoid the underscore in `--auto_merge` and use `--auto-merge` instead.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In order to avoid manual work, enable merge when pipeline succeeds for
trivial changes, such those on d/apertis/* or d/changelog if this feature
is enabled via command line.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Arnaud Ferraris authored 4 years ago and Martyn Welch committed 4 years ago

Building packages into `_build` causes issues with ocaml packages: those
too build into this directory by default, which is therefore removed
during the `clean` stage.

This has been observed for ounit, pcre-ocaml and ocaml-result, and
probably affects all ocaml packages.

Using a less common name for the build directory solves the issue
without having to patch individual packages.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

A lot of `node-*` packages have their Build-Depends formatted in a way
apt can't parse, causing CI failures.

Using wrap-and-sort beforehand in order to make sure the file format is
correct will help working around these issues.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

This reverts commit cf5f4c2d.

Building packages into `_build` causes issues with ocaml packages: those
too build into this directory by default, which is therefore removed
during the `clean` stage.

This has been observed for ounit, pcre-ocaml and ocaml-result, and
probably affects all ocaml packages.

Using a less common name for the build directory solves the issue
without having to patch individual packages.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

`gbp import-dsc` puts all the new changelog entries in the commit
message, generating big walls of text when, for instance, importing the
version from Bullseye on top of the Buster one.

GitLab then puts the whole log message in the `CI_COMMIT_MESSAGE` env
var, which is passed on the docker command line, resulting in a error:

    standard_init_linux.go:219: exec user process caused: argument list too long

See https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26624#note_529234097



To avoid that, trim the message to only keep the first line.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Create a MR even when the local attempt to merge failed. In that case
let the proposed updates branch point to the upstream branch and create
a MR to submit it to the downstream branch.

This makes the failed updates more visbile and easier to track than the
failed pipelines.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Refactor the code to reduce the amount of state carried across loop
iterations and to reduce the nesting of conditionals, so that it's
hopefully easier to understand and modify.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In some cases the current code ensures that a local downstream branch,
for instance `apertis/v2021` or `apertis/v2021-updates`, is available if
a corresponding ref exists in the `origin` remote.

However, in some other cases the local ref is not created, creating an
annoying inconsistency. For instance it is never created for development
branches like `apertis/v2022dev1`. Even for stable branches, when
merging from a security branch like `debian/buster-security` and a
`origin/apertis/v2021-security` remote ref is available, the local
`apertis/v2021-security` ref is not created.

Tweak the code to always ensure there's a local ref for the selected
downstream branch.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Without it, the generated changelog was wrong like below:

```
node-accepts (1.3.7-1{local_suffix}0) apertis; urgency=medium

  * Sync from Debian debian/bullseye.

 -- Apertis CI <devel@lists.apertis.org>  Wed, 10 Mar 2021 14:56:51 +0000
```

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

The RELEASE string is used commonly by multiple parties
We set a custom CUSTERL variable, for this special case, to push in the
special string "bullseyebootstrap"

For the use case of building pure Debian packages, we only need the
package-source-builder. So, lets just disable the rest of the images.

And also add the Debian specific bits to get the debootstrap work

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Ritesh Raj Sarraf authored 4 years ago and Emanuele Aina committed 4 years ago

Switch from coX to apertisX as the default Debian changelog suffix.

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>
Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Downstreams need a different local suffix for their packages and we may
change ours at some point.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 4 years ago and Martyn Welch committed 4 years ago

Avoid failing silently in case of fakemachine boot issues.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

License scan for `pkg/dash` and `tests\dash` should not re-use the scan
results from a previous scan of the other project.

Separate license scan using different group id, which is done using
different users.

Strip off the suffix since all branches related to the upstream
Debian release need to be taken into account when pulling updates.

Without this, when the script runs on e.g. buster-security branch,
it would completely ignore the existence of any other buster branches,
leading to incorrectly applied updates.

Apertis: T7681

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Try to download an update for dash from Buster, given a repository
with a version from Lenny. Don’t try to check when exactly is in the
update, as versions may change, but verify a tarball has been imported.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

This will try to merge an "update" of dash on a completely made-up
package history. There are two fixtures, sharing common parts. They
both:

* Start with an ancient dash version from Lenny, 0.5.4-12.
* This version is imported as 0.5.4-12co0 into an imaginary
  Apertis version.
* A "new" version of dash, 0.5.10.2-1 is imported from a 2018
  cut of Buster.
* This new version is merged into Apertis as 0.5.10.2-1co0

From this point, the "modified" fixture differs:
* An imaginary Apertis developer changes the compat level in
  the imported version from 10 to 11, thus introducing a delta
  which however is not going to conflict with any further version
  of dash in Buster.

apertis-pkg-merge-updates is expected to merge the update into the
"unmodified" repository as 0.5.10.2-5co0 and finalise the changelog.

For the "modified" repository, it should import the update as
0.5.10.2-5co1, add "PLEASE SUMMARIZE remaining Apertis changes" note
to the top entry and leave it UNRELEASED.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

We always add some extra files, at least the Apertis component,
plus copyright scanning results. Since those are intended to end up
in the source package, we should append a version suffix regardless
of any other actual changes we add on top.

Since these changes are an addition of metadata only, co0 seems a
good enough starting version.

Instead of dch, we use a custom changelog modification implementation.
This is because dch insists on launching an interactive editor when
a changelog is finalised.

Apertis: T7556

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

checkout-tarballs is no longer needed since pristine-lfs can do
this directly.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Since pristine-lfs can import tarballs automatically, we should
not use a separate but different implementation of the same
functionality.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Include some extra compression tools from the upstream Debos
dockerfile which may be useful.

Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

The equivs package is used to create dummy Debian packages which can
be useful in development to satisfy dependencies without installing the
real package. Note that this package is not the recommended way of
dealing with broken dependencies: a bug report should be filed instead.

Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

By applying `latest` tags as soon as the relevant image has been built
we end up with a quite large time window where they are all quite out
of sync: in particular, the tag is applied to the `base` image as soon
as it is ready, before all the depending images get built.

On top of that, if depending images fail, the `latest` tags will stay
out of sync.

This can be quite confusing for users, so let's apply the `latest` tag
only after all the previous stages have completed.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

We want to ensure that the jobs updating the `latest` tags always reflect
the actual latest pipeline.

When two pipelines are started at the same time on the same branch
sometimes the earlier one can complete last due to scheduling or any
other reason, thus breaking the previous assumption.

To avoid that, make the whole pipeline interruptible up to the tagging jobs
so that earlier pipelines get killed when a new one is started.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>