Skip to content
Snippets Groups Projects
Commit 38e0fc03 authored by Dylan Aïssi's avatar Dylan Aïssi
Browse files

Filter out packages not in Apertis during the packaging-data-fetch-sources-upstream job


Fetching upstream sources data for all Debian packages (~ 38100 pkgs)
and not only those in Apertis (~ 5600 pkgs) generates big YAML files
containing useless data. Processing these files lead to frequent
out-of-memory issues. To reduce the memory consumption, we can filter out
all packages not available in Apertis based on the cache file.

This job produces a 42 MB YAML file, merging this file with others YAML
files leads to out-of-memory issues in downstream jobs. Filtering the
generated YAML file at this step reduces the size to only 8 MB.

Signed-off-by: default avatarDylan Aïssi <dylan.aissi@collabora.com>
parent e5e94430
No related branches found
No related tags found
Loading
Checking pipeline status
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment