-
Dylan Aïssi authored
Fetching upstream sources data for all Debian packages (~ 38100 pkgs) and not only those in Apertis (~ 5600 pkgs) generates big YAML files containing useless data. Processing these files lead to frequent out-of-memory issues. To reduce the memory consumption, we can filter out all packages not available in Apertis based on the cache file. This job produces a 42 MB YAML file, merging this file with others YAML files leads to out-of-memory issues in downstream jobs. Filtering the generated YAML file at this step reduces the size to only 8 MB. Signed-off-by:
Dylan Aïssi <dylan.aissi@collabora.com>
Dylan Aïssi authoredFetching upstream sources data for all Debian packages (~ 38100 pkgs) and not only those in Apertis (~ 5600 pkgs) generates big YAML files containing useless data. Processing these files lead to frequent out-of-memory issues. To reduce the memory consumption, we can filter out all packages not available in Apertis based on the cache file. This job produces a 42 MB YAML file, merging this file with others YAML files leads to out-of-memory issues in downstream jobs. Filtering the generated YAML file at this step reduces the size to only 8 MB. Signed-off-by:
Dylan Aïssi <dylan.aissi@collabora.com>