Ryan Gonzalez authored 2 years ago and Emanuele Aina committed 2 years ago

If a snapshot has any missing files, it would make the entire dashboard
collection fail, which isn't really the ideal behavior to have.

https://phabricator.apertis.org/T8593



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Ryan Gonzalez authored 2 years ago and Emanuele Aina committed 2 years ago

All non-2xx statuses were actually being ignored, because status codes
aren't propagated as errors by default.

https://phabricator.apertis.org/T8593



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Odd that it managed to pass multiple runs of CI but is now suddenly
timing out, but it appears to still be working just fine based on the
log output, so just try to make it a tad faster & make sure it can take
longer.

Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Now, "unique" is used for unique storage on both repositories *and*
snapshots, and "direct" is used for the storage directly used, rather
than "total".

Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

This adds a new tool, storage_stats, to gather statistics on the used
storage, as well as a new dashboard page to display the statistics. Due
to exceedingly poor performance from attempting to use pure Python to
parse all the packaging files in over 2000 snapshots (a total of several
thousand files to parse & process), Rust was used for the storage_stats
implementation; on my system, it determines the full extent of storage
usage in under 20 minutes.

On the dashboard's side, the common HTML and styling was moved into a
single shared template that both the current index/packages page *and*
the new storage page can use, and navigational links were added to the
pages.

https://phabricator.apertis.org/T8197



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>