diff --git a/content/concepts/automated-license-compliance.md b/content/concepts/automated-license-compliance.md index b284589f9ee44c6ee6c8dd8c739d6c1050b98c2d..ec31b6a238d87575bd5904ed587fba0bbbbd1a02 100644 --- a/content/concepts/automated-license-compliance.md +++ b/content/concepts/automated-license-compliance.md @@ -121,3 +121,46 @@ This same scripts also issues a warning in case a problematic license is found. Apertis utilizes [Debos](https://github.com/go-debos/debos) in its image generation pipeline, which provides a very versatile way of customizing them. During the final stage of the image creation, the script `generate_bom.py` is used to build the BOM file with the license information of the image and export it as an additional artifact. Finally as both `minimal` and `target`images should not shipped extra data, the contents of `/usr/share/doc/` are dropped from the image. + +# Step-by-step process + +This is a description of the steps in the process as currently implemented: + +1. when a package is imported from Debian to Apertis the + [`scan-license` job in the packaging pipeline](https://gitlab.apertis.org/infrastructure/ci-package-builder/-/blob/c2c59e28/ci-package-builder.yml#L313) + will call + [`ci-license-scan`](https://gitlab.apertis.org/infrastructure/apertis-docker-images/-/blob/6bc2a375/package-source-builder/overlay/usr/bin/ci-license-scan) + to submit the sources to the scanner, be it `scan-copyright`, FOSSology or any other tool +1. metadata in + [`debian/apertis/copyright.yml`](https://gitlab.apertis.org/pkg/gnutls28/-/blob/dae6f34d/debian/apertis/copyright.yml) + can be used to override things where the scanner gives the wrong results, + which should no longer be needed once the switch to FOSSology is completed and + the correct licensing information is stored in its database +1. the output is committed in the + [`debian/apertis/copyright` YAML files in the sources](https://gitlab.apertis.org/pkg/gnutls28/-/blob/dae6f34d/debian/apertis/copyright) +1. if some files have problematic licenses but they do not really affect us for + any reason, the reason is documented in + [`debian/apertis/copyright.whitelist`](https://gitlab.apertis.org/pkg/gnutls28/-/blob/dae6f34d/debian/apertis/copyright.whitelist) +1. for packages meant to be installed on production devices, the packaging + pipeline will fail if problematic licenses are detected and the affected + files are not whitelisted +1. when the sources are submitted to OBS, the + [`dh_dwarf2sources` subcommand for Debhelper](https://gitlab.apertis.org/pkg/debhelper/-/blob/8abfd8a5/dh_dwarf2sources) + calls the + [`dwarf2sources` tool](https://gitlab.apertis.org/pkg/dwarf2sources/) + to generate a mapping from binaries to the source files used to build them +1. the output is included in the same `.deb` file as the processed + library/executable, under `/usr/share/doc/$packagename/copyright_report.gz` +1. for each installed `.deb` package the + `/usr/share/doc/$packagename/copyright_report.gz` files get unpacked during + image generation +1. the [`generate_bom.py` script](https://gitlab.apertis.org/infrastructure/apertis-image-recipes/-/blob/283bcd3f/scripts/generate_bom.py) + is [invoked at the end of each image recipe](https://gitlab.apertis.org/infrastructure/apertis-image-recipes/-/blob/283bcd3f/image-uboot.yaml#L150), + loading all the `/usr/share/doc/*/copyright_report.gz` files producing + [a JSON report](https://images.apertis.org/release/v2022dev2/v2022dev2.0/arm64/minimal/apertis_v2022dev2-minimal-arm64-rpi64_v2022dev2.0.img.licenses.gz) + alongside each produced image, using the source→license and binary→source + mappings above to match each installed library and executable to the + licenses of the sources used to build them +1. human-readable reports in any format can be generated by the JSON data + describing the licenses that apply to the libraries and executables shipped + in the image itself