Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Sometimes, we need to import and modify packages from Debian.
To make the initial review easier, an apertis/v202* branch is created
from the debian one, then a merge request is created against it adding
debian/apertis/* files. During the review, tha apertis branch
doesn't have any debian/apertis/* files and the dashboard fails with:

  Traceback (most recent call last):
    File "/builds/infrastructure/dashboard/./bin/packaging-check-invariants", line 857, in <module>
      checker.check_git_and_obs_versions(package)
    File "/builds/infrastructure/dashboard/./bin/packaging-check-invariants", line 271, in check_git_and_obs_versions
      component = branch.component
                ^^^^^^^^^^^^^^^^
    File "/builds/infrastructure/dashboard/./bin/packaging-check-invariants", line 79, in __getattr__
      item = super(DictWrapper, self).__getitem__(attr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  KeyError: 'component'
  The above exception was the direct cause of the following exception:
  Traceback (most recent call last):
    File "/builds/infrastructure/dashboard/./bin/packaging-check-invariants", line 867, in <module>
      raise Error(package.name) from e

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Apertis v2022 is no longer supported, so drop its configuration to save
resources and avoid misunderstandings.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

The version of this tool used in Apertis is frozen
due to the upstream license change (to GPL-3), so
no need to run uscan since we know this is outdated version
on purpose.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

For some packages, the watch file use the "git" mode to check git
repository, in that case git is required otherwise it fails with:

uscan issue: Command 'git' not found in /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin at /usr/share/perl5/Devscripts/Utils.pm line 24.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

When uscan reports a package is up to date, its return code is 1,
which is interpreted as an issue with uscan/watch by the dashboard.
So, before returning an issue for this package, check if the
output of uscan contains "Package is up to date".

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 1 year ago and Walter Lozano committed 1 year ago

Sometimes, we need to backport a package version to a previous
Apertis release. But, a fast-forward backport is not always
possible as some changes may have occurred that are not suitable
for previous Apertis releases (like a change in debian/apertis/component
or debian/apertis/copyright needs to be refreshed with a different
version or the toolchain). In order to keep the git history linear,
while doing the backport, unsuitable changes are reverted and a
suffix like "~v2023" is used for versioning.

Because this unsual workflow is a bit tricky, it's not possible
to teach the dashboard about this suffix which makes it complaning
for "Mismatch between OBS XXX~v2023 and Git XXX".

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This mapping was included from the beginning in the dashboard, but it seems
that is not really needed right now. Having this mapping causes dashboard
to complain with

gnome-settings-daemon apt error Not published on APT
gnome-settings-daemon-data git error GitLab project missing but found on OBS:

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Whitelists may not be useful for downstreams, so it should not
be mandatory.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Both scripts packaging-updates and packaging-check-invariants
contain quite long lists of packages to ignore. This makes both
scripts and lists a bit difficult to read. Instead move the lists
into a specific file at data/whitelists.yaml. This should make
maintenance easier.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 1 year ago and Walter Lozano committed 1 year ago

Due to the new anchor added in 3c55dc09 for v2024,
the indentation was wrong. Because of this wrong
indentation, the source field for v2025dev*
is filled with bullseye values instead of bookworm.
This completely confuses the dashboard which compares
Apertis bullseye-based releases to the bookworm-bases
releases.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

For most of these packages Apertis is upstream,
so no need to run uscan.
For some others, the version used in Apertis is frozen
due to the upstream license change (to GPL-3), so again
no need to run uscan since we know this is outdated version
on purpose.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Make the past an exception instead of the future.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dashboard goes through all the packages in GitLab and reports
an error if they are not published in apt repositories. Unfortunately,
this behavior does not take into account the fact that packages can
be dropped in a release, so they are no longer published.
To fix that, don't report missing packages on apt for packages no
longer in an active Apertis release.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Make the past an exception instead of the future.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

These definitions are inherited through anchors.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

No need to keep a separate definition without non-free.
"-security", "-updates" and "-backports" may contain
non-free packages that have been imported in Apertis.

We need to make sure we stay in sync with all sources
even for the non-free component.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Since Bookworm, some Debian packages moves to non-free-firmware instead
of non-free:
- https://lists.debian.org/debian-devel/2022/12/msg00176.html
- https://lists.debian.org/debian-project/2023/01/msg00018.html



We need to track this component to ensure some of our packages
are up to date (e.g. raspi-firmware or firmware-nonfree).

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Dylan Aïssi authored 1 year ago and Walter Lozano committed 1 year ago

For some packages, Apertis is upstream that means the main and
only git repository is https://gitlab.apertis.org/pkg/*

, so
there is no need to run uscan to scan another repository and for
them a missing debian/watch is expected.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Otherwise, it can generate too much noise in the gitlab job log.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Otherwise uscan fails with a message saying required files
are badly formatted.

This issue is not observed in Apertis because all our repositories
are public, but it affects downstream since their repositories are
not publicly available.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

In Apertis is very common to disable the build of some binaries to avoid
license issues, reduce the number of dependencies or disable unsupported
architectures. To do it, usually d/rules is tweak to skip some binary
packages. However, this approach makes dashboard to complain about missing
binaries.

A possible approach could be to remove the binary packages from d/control
but this approach adds more delta.

To avoid getting a wrong idea of status of the packages, ignore this issue
for the packages we know we tweaked.

The approach of using wildcards to reduce the number of entries was discussed
however, to improve readability and to avoid extra complexity while checking
the idea was discarded.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

The Apertis development channel is still based on Bookworm, but
some specific packages are backported from Trixie. Not tracking
Trixie will leave us with outdated packages.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Walter Lozano authored 1 year ago and Andrej Shadura committed 1 year ago

Since v2024dev3 has already been mirrored by downstreams drop it from
channels to save time and space.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

After switching to JSON in commit 254b71ed, the output of the dashboard is
not longer sorted, which is a bit annoying. To fix that, force the sort
of the output before rendering it.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

In commit 254b71ed dashboard switched to JSON to improve efficiency. While
doing that several functions and strings were renamed to point to the new
format. Unfortunately there are still references to YAML in several places
which are misleading.

Fix remaining references to YAML to avoid confusions.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

During the switch from YAML to JSON (commit 254b71ed),
the saved object was changed by mistake. Instead of saving the
results of updates analysis, the input data were saved in
the JSON file. Because of wrong data written, the dashboard
no longer complains about missing updates from Debian.
This commit reverts restore the right object to be saved in
the JSON file.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

In commit 254b71ed the format of the intermediate files was changed to JSON
to improve performance and save resources. With this change the logic to
dump version information was adapted, but unfortunately not all the cases
were covered, causing version to be dumped as null. As a consequence, the
checks for updates also fail.

Fix the dumping of version information by using string.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

In commit 09e15175 the default value for license report was changed
to True to match the most common case. However, this change was not
propagated to whole code.

Fix the issue by using the correct default value.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>