Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

To make it less austere

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This makes it easier to check the Debian's package status.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Move the call to `gitlab-rulez` to a separate job to shorten a bit the
already extra-long `packaging-data-fetch-downstream` job, and shift it
to the `check` stage as it seems more appropriate.

This also causes the pipeline to properly honor the `$GITLAB_RULES_URL`
variable, as it stops using the hardcoded
gitlab.apertis.org/infrastructure/apertis-infrastructure URL in
`bin/packaging-data-fetch-downstream`.

Reporting is also improved by showing details about the incorrect
settings by using the recently introduced `--output json` option in
`gitlab-rulez`, currently only available starting from Debian Trixie.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

For packages not imported from Debian, run uscan to check if new upstream
releases are available. If a new version is found report it on the dashboard.
Also report missing and broken debian/watch files to ensure we continue to track
the correct upstream.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Source packages contain a "Package-list" field listing all binary packages
generated. All these generated binary packages should be published and
listed in the package index file to avoid possible inconsistencies
between the published Source/Binary packages in the APT repository.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Currently the dashboard shows only counters for total
errors. In addition to the total errors, show other
errors for different domains.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>

Branch name is already displayed in the error
and the reduntant one can be removed from
the error message.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>

Report error if lintian job is not enabled in development branch
for packages not from debian.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>

Some packages are fetched from untracked Debian releases, we need a way to
list them in order to check if they require an update or to be sync against
a tracked Debian release.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Some Apertis packages come from a different Debian release. In order keep
them in sync, active descendant branches are analyzed to determine in which
apertis versions they landed. Then, this information is used to report on the
dashboard which apertis releases need to be updated.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Git repositories settings are set by gitlab-rulez.
If a setting is wrongly set a an error is displayed on the
dashboard.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

A button is added on the dashboard next to the "Update all" button,
to run the trigger-gitlab-rulez job using the "apply" mode.
This allows developers to easily fix "pkg/*" repositories settings.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

`apertis/*`, `debian/*` and `pristine-lfs` should be protected.
If these branches are not protected an error is displayed on the
dashboard.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

The gitlab default branch should match the latest Apertis release
providing the package. This check verifies if the default branch
corresponds to the latest Apertis release based on the published
package. If there is a mismatch, an error is displayed on the
dashboard.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

Severity of security updates are changed to error.
Added a ftbfs domain for package build failures.
This allows filtering the dashboard report for OBS
package build error and security updates easily.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>

Due to badly formatted tag the detection of updates in the dashboard was
broken. The version in debian/bullseye was higher than the one actually
in bullseye. This prevented the dashboard from reporting about pending
updates. This commit reports error when the version in debian/buster or
debian/bullseye is higher than the one in upstream buster/bullseye.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>

Emanuele Aina authored 2 years ago and Vignesh Raman committed 2 years ago

In some cases the dashboard generated links to trigger new pipelines
with an empty ref because it was using a non-existing `name` property
on the branch in the report, which is just a string.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

LICENSING_REPORT_MISSING: no `apertis/debian/copyright` found -> warn
LICENSING_OVERRIDE: `override-license:` is found in `apertis/debian/copyright.yml` -> warn
LICENSING_GLOBAL_WHITELIST: `*` is found in `apertis/debian/copyright.whitelist` -> notice
LICENSING_GLOBAL_DEFAULT: `.*`: is found in `apertis/debian/copyright.yml` -> notice (unless unfriendly - dual or not)
LICENSING_GLOBAL_DEFAULT_UNFRIENDLY: global default found, not dual but unfriendly -> warn
LICENSING_GLOBAL_DEFAULT_DUAL_UNFRIENDLY: global default found, set to dual and at least one of them is unfriendly -> warn

Signed-off-by: Andre Moreira Magalhaes <andre.magalhaes@collabora.com>

For this initial version domains are derived from the `Report`
enum name but this change should add more flexibility for future
improvements/categorization.

Signed-off-by: Andre Moreira Magalhaes <andre.magalhaes@collabora.com>

Signed-off-by: Andre Moreira Magalhaes <andre.magalhaes@collabora.com>

GitLab makes retrying failed jobs easy, but sometimes it's preferable to
retrigger the whole pipeline from scratch for consistency, so let's make
it more convenient by adding a dedicated button to each failure report.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Arnaud Ferraris authored 3 years ago and Detlev Casanova committed 3 years ago

In order to be able to easily identify licensing issues, those should be
reported in a different way than other issues. This is achieved by
changing the severity of those issues to "warning" and using "license"
as the text for the severity badge. This also allows us to easily filter
those issues based on the selected severity.

This commit also changes the filtering checkboxes labels to better
reflect which issues are being taken into account.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

Arnaud Ferraris authored 3 years ago and Detlev Casanova committed 3 years ago

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

Emanuele Aina authored 3 years ago and Vignesh Raman committed 3 years ago

Applying filters one-at-a-time can lead do broken behavior:

* deselect both the `info` and `error` severity checkboxes
* all reports are correctly hidden
* deselect the `apertis/v2021` channels checkbox
* all the other reports incorrectly show up again, regardless of
  their severity

This is because when applying the updated channels filter the code does
not look at the other active filters.

Tweak the code to always check *all* filters regardless of the one that
has been updated.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Point to the GitLab compare tool when a branch is lagging behind the
others, to let people find easily the missing commits and potentially
create a merge request for them using the handy button in the comparison
page itself.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Highlight the `Filter` button when any filter is active and reports may
be hidden.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Rather than using plaintext error messages, use readable codes and
structured metadata for errors and updates to make them easier
to process.

This will be particularly useful for filtering: for instance we
preserve the branch information rather than muddling it in the
error message.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This adds a new tool, storage_stats, to gather statistics on the used
storage, as well as a new dashboard page to display the statistics. Due
to exceedingly poor performance from attempting to use pure Python to
parse all the packaging files in over 2000 snapshots (a total of several
thousand files to parse & process), Rust was used for the storage_stats
implementation; on my system, it determines the full extent of storage
usage in under 20 minutes.

On the dashboard's side, the common HTML and styling was moved into a
single shared template that both the current index/packages page *and*
the new storage page can use, and navigational links were added to the
pages.

https://phabricator.apertis.org/T8197



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>