Skip to content

obs: Add a job to capture binaries in the parent pipeline

Emanuele Aina requested to merge wip/em/catpure-obs-artifacts-in-parent into master

The retrieval of binary packages from OBS currently happens on a dynamic child pipeline since the number of builds to be monitored is dynamic as it depends on the number of repositories and the number of architectures configured on OBS, and on the actual package control file.

Unfortunately, making artifacts originating from the child pipeline to the parent one is currently unsupported in GitLab, see https://gitlab.com/gitlab-org/gitlab/-/issues/285100#note_1610649891.

However, we need that for jobs like lintian, abi-checker, and local-post-binary-build to be useful. Alternatively, the post-build steps could be injected in the child pipeline, but this has its own challenges as well, being harder to maintain, annoying to visualize, prone to be affected by other nested pipeline limitation.

By using a Project/Personal Access Token like the one used to push to the pristine-lfs-source branch ($GITLAB_CI_PASSWORD) the retrieval could be open-coded via the GitLab API, but ideally we would like to move away from it as well, see https://gitlab.com/gitlab-org/gitlab/-/issues/389060

CI_JOB_TOKEN does not give access to the bridge jobs API, and only grants access to the artifacts API on Premium instances (the GitLab documentation is misleading, CI_JOB_TOKEN can retrieve artifacts), so it is definitely not suitable for the API approach.

That said, in our specific case we are retrieving artifacts from OBS anyway, so we can add a new job to call osc getbinaries with basically the same caveats of the current approach. In fact, that's what the lintian job is already doing for its own needs, so we can later move it to the newly captured artifacts.

Using CI_JOB_TOKEN to access the generic package registry as shared temporary storage across the child-parent pipelines would be a valid approach, but it seems overkill in our specific case.

Sample job: https://gitlab.apertis.org/tests/dash/-/jobs/2812409/artifacts/browse

Artifacts:

binaries/default_aarch64/_buildenv
binaries/default_aarch64/_statistics
binaries/default_aarch64/ash_0.5.12-2+apertis1.1bv2024preb1_all.deb
binaries/default_aarch64/dash-dbgsym_0.5.12-2+apertis1.1bv2024preb1_arm64.deb
binaries/default_aarch64/dash_0.5.12-2+apertis1.1.debian.tar.xz
binaries/default_aarch64/dash_0.5.12-2+apertis1.1.dsc
binaries/default_aarch64/dash_0.5.12-2+apertis1.1bv2024preb1_arm64.buildinfo
binaries/default_aarch64/dash_0.5.12-2+apertis1.1bv2024preb1_arm64.changes
binaries/default_aarch64/dash_0.5.12-2+apertis1.1bv2024preb1_arm64.deb
binaries/default_aarch64/dash_0.5.12.orig.tar.gz
binaries/default_armv7hl/_buildenv
binaries/default_armv7hl/_statistics
binaries/default_armv7hl/ash_0.5.12-2+apertis1.1bv2024preb1_all.deb
binaries/default_armv7hl/dash-dbgsym_0.5.12-2+apertis1.1bv2024preb1_armhf.deb
binaries/default_armv7hl/dash_0.5.12-2+apertis1.1.debian.tar.xz
binaries/default_armv7hl/dash_0.5.12-2+apertis1.1.dsc
binaries/default_armv7hl/dash_0.5.12-2+apertis1.1bv2024preb1_armhf.buildinfo
binaries/default_armv7hl/dash_0.5.12-2+apertis1.1bv2024preb1_armhf.changes
binaries/default_armv7hl/dash_0.5.12-2+apertis1.1bv2024preb1_armhf.deb
binaries/default_armv7hl/dash_0.5.12.orig.tar.gz
binaries/default_x86_64/_buildenv
binaries/default_x86_64/_statistics
binaries/default_x86_64/dash_0.5.12-2+apertis1.1bv2024preb1_all.deb
binaries/default_x86_64/dash-dbgsym_0.5.12-2+apertis1.1bv2024preb1_amd64.deb
binaries/default_x86_64/dash_0.5.12-2+apertis1.1.debian.tar.xz
binaries/default_x86_64/dash_0.5.12-2+apertis1.1.dsc
binaries/default_x86_64/dash_0.5.12-2+apertis1.1bv2024preb1_amd64.buildinfo
binaries/default_x86_64/dash_0.5.12-2+apertis1.1bv2024preb1_amd64.changes
binaries/default_x86_64/dash_0.5.12-2+apertis1.1bv2024preb1_amd64.deb
binaries/default_x86_64/dash_0.5.12.orig.tar.gz

Task: https://phabricator.apertis.org/T10120

Edited by Emanuele Aina

Merge request reports