This reverts commit e954d2be.

This was a workaround for odd nodes packages that apt was
not able to parse. It has been fixed in apt 2.3.10 (Bookworm)
with https://salsa.debian.org/apt-team/apt/-/merge_requests/167

.

Having this workaround leads to other apt failures
to resolve dependencies (because of conflicts between
rustc and rustc-web).

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Vignesh Raman authored 1 year ago and Dylan Aïssi committed 1 year ago

When building packages, wrap-and-sort fails with the below error,
raise SyntaxOrParseError(
debian._deb822_repro.types.SyntaxOrParseError: Syntax or Parse error on the line: "%if USE_SYSTEM_ZLIB\n"

wrap-and-sort works in bullseye python-debian version 0.1.39 and in
bookworm version 0.1.49 it fails to parse debian/control.in files.

We don't need to use wrap-and-sort on all files in debian/ folder,
but only on debian/control to fix the issue with node-* packages.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Emanuele Aina authored 3 years ago and Martyn Welch committed 3 years ago

We can now drop the `debian/apertis/` cleanup hook since:

1. we no longer use `debian/apertis/gitlab-ci.yml` since we have
   switched to an external reference
2. `debian/apertis/` won't become empty since we always have at least a
   `component` file

This also makes `ci-buildpackage` work with non-Apertis packages which
were previously failing on the `rmdir` invocation no handling well the
lack of the `debian/apertis/` folder.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use of `git restore` has been introduced in commit e954d2be
"ci-buildpackage: use wrap-and-sort as a workaround for odd packages".

It works on the Bullseye-based v2022dev2, but on older Buster-based
release `git restore` is not available.

Replace it with the old-style `git checkout`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Arnaud Ferraris authored 4 years ago and Martyn Welch committed 4 years ago

Building packages into `_build` causes issues with ocaml packages: those
too build into this directory by default, which is therefore removed
during the `clean` stage.

This has been observed for ounit, pcre-ocaml and ocaml-result, and
probably affects all ocaml packages.

Using a less common name for the build directory solves the issue
without having to patch individual packages.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

A lot of `node-*` packages have their Build-Depends formatted in a way
apt can't parse, causing CI failures.

Using wrap-and-sort beforehand in order to make sure the file format is
correct will help working around these issues.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

This reverts commit cf5f4c2d.

Building packages into `_build` causes issues with ocaml packages: those
too build into this directory by default, which is therefore removed
during the `clean` stage.

This has been observed for ounit, pcre-ocaml and ocaml-result, and
probably affects all ocaml packages.

Using a less common name for the build directory solves the issue
without having to patch individual packages.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

checkout-tarballs is no longer needed since pristine-lfs can do
this directly.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Repeating "apertis" everywhere is not particularly useful but is
annoying for downstreams which have to either keep the misleading
names or rename everything and deal with the resulting mess when
pulling updates.

This also makes our life less painful since not having a common prefix
makes shell completion much more useful. :)

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

If for any reason `git-buildpackage` fails to detect the orig tarball
extracted from the `pristine-lfs` branch it tries to generate it using
`pristine-tar` or `git archive`.

This is not desirable, we definitely prefer erroring out in case
something goes wrong as we always want to rely on `pristine-lfs` to
ensure that the orig tarballs are bit-identical to the upstream ones.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Emanuele Aina authored 5 years ago and Andrej Shadura committed 5 years ago

To avoid conflicts with any upstream `gbp.conf` that packages may ship, the
GitLab pipeline currently sets `GBP_CONF_FILES=/dev/null`.

However, having it in the pipeline leads to confusing errors when testing
things locally, as it easy to forget to set the right environment in the
Docker container.

The `apertis-pkg-pull-updates` script already does something similar,
only sourcing `/etc/git-buildpackage/gbp.conf`. We may later look into
aligning them.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Seems http pipeline sometimes causes broken downloads; While this really
needs fixing in apt work around it for now so CI doesn't hit it.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

"build" is a rather common name which has the potential for file name
conflicts, as is an example below.

Processing triggers for libc-bin (2.28-7co1ba1) ...
Processing triggers for man-db (2.8.5-2bv2020dev0b1) ...
+ dpkg-checkbuilddeps
+ dpkg-parsechangelog -S Version
+ dversion=3.2.26+dfsg-3
+ checkout-tarball build
Traceback (most recent call last):
  File "/usr/bin/checkout-tarball", line 49, in <module>
    main()
  File "/usr/bin/checkout-tarball", line 25, in main
    os.makedirs(dest, exist_ok=True)
  File "/usr/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: 'build'
ERROR: Job failed: exit code 1

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

We keep getting hard-to-debug hash mismatches when downloading packages with
APT, like this one in the https://gitlab.apertis.org/pkg/target/ostree/-/jobs/189090
GitLab pipeline:

    Err:94 https://repositories.apertis.org/apertis v2020dev0/target amd64 libsystemd-dev amd64 241-5co3bv2020dev0b1
      Hash Sum mismatch
      Hashes of expected file:
       - SHA256:41d1ccfd064d8e7aafd19d031f0f35e350279dec3e46c2d6323b9a70159a5bd4
       - SHA1:161fff18c277d0b8bda8a80be7a433e8807aca13 [weak]
       - MD5Sum:2ae73e29e95daa715d07ac2b1ca78412 [weak]
       - Filesize:320672 [weak]
      Hashes of received file:
       - SHA256:bba2cd8458b872069ec6697f2ec23ee47ab5e285d6627f96b6d44914d61c0f14
       - SHA1:384222001e9c4a63552e0017da7aadee5894344b [weak]
       - MD5Sum:63b6130dcff030abf9c75a91074b6731 [weak]
       - Filesize:320672 [weak]
      Last modification reported: Thu, 15 Aug 2019 18:46:42 +0000
    Get:95 https://repositories.apertis.org/apertis v2020dev0/target amd64 libyaml-0-2 amd64 0.2.1-1ba1 [49.2 kB]
    Get:96 https://repositories.apertis.org/apertis v2020dev0/development amd64 python3-yaml amd64 3.13-2bv2020dev0b1 [121 kB]
    E: Failed to fetch https://repositories.apertis.org/apertis/pool/target/s/systemd/libsystemd-dev_241-5co3bv2020dev0b1_amd64.deb  Hash Sum mismatch
       Hashes of expected file:
        - SHA256:41d1ccfd064d8e7aafd19d031f0f35e350279dec3e46c2d6323b9a70159a5bd4
        - SHA1:161fff18c277d0b8bda8a80be7a433e8807aca13 [weak]
        - MD5Sum:2ae73e29e95daa715d07ac2b1ca78412 [weak]
        - Filesize:320672 [weak]
       Hashes of received file:
        - SHA256:bba2cd8458b872069ec6697f2ec23ee47ab5e285d6627f96b6d44914d61c0f14
        - SHA1:384222001e9c4a63552e0017da7aadee5894344b [weak]
        - MD5Sum:63b6130dcff030abf9c75a91074b6731 [weak]
        - Filesize:320672 [weak]
       Last modification reported: Thu, 15 Aug 2019 18:46:42 +0000

It also happens on Jenkins and has been intermittently reproduced on developer machines.

Only APT seems to be affected, so enable as much debugging we can to try to catch it.

See https://phabricator.collabora.com/T15071

 for more information.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

When populating the build dir with a tarball gbp will create a symlink
(if both are different). While this works fine for the build it does
mean things get confused when just the build dir is stored as an
artifact.

To fix simply make the tarball and build dirs the same as there is no real
benefit in keeping them seperate.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

ci-buildpackage has two main jobs; it builds the source package and
uploads it to OBS, while also submitting a request iff the build was for
a tag.

This ends up complicating the script as well as taking all policy away
from the gitlab-ci job. Preventing e.g. just running the build for
testing purposes without uploading.

This split ci-buildpackage in a simpler wrapper script around
git-buildpackage which sets up the build environment etc and a new
ci-obs-upload script to dput into obs; Both scripts look at the RELEASE
environment variable for release vs snapshot mode. In snapshot mode
ci-buildpackage adds a deb-git-version-gen based version, while
ci-obs-upload in release mode will also run submitrequest

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

As we have both debian/ tags and apertis/ tags make sure
deb-git-version-gen only considers local hooks.

Patch from:
  https://gitlab.collabora.com/smcv/deb-build-snapshot/merge_requests/2



Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

This will create meaningful (and increasing) version numbers for
building snapshots.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

Drop the check for a git tag for making an upload. This is needed to
get all changes to be built before a release is tagged.

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Adding -i to dpkg-buildpackage invocation adds the common defaults
which are only enabled by default for 3.0 packages.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>