Draft: Export the APT metadata

On Apertis side the information about the used APT snapshots is exported to the metadata file created during image pipeline execution. This approach works fine on Apertis side and a simple post processing can be implemented to extract the information about the used APT snapshots. The same approach fits the downstream distribution Apertis pro. But the product teams using Apertis pro do not follow or do only follow partially this approach. Additionally, these product teams have a more complex APT repository structure and therefore the mentioned approach is not suitable. To ease the overall approach to collect the used APT snapshots, especially for complex APT repository structures, the proposed solution is to export the APT source list(s). Beside this, it make sense to export the APT pinning information (APT preferences). These files can be used for simple post processing as well.

Signed-off-by: Tino Lippold tino.lippold@de.bosch.com

marked this merge request as draft

@wlozano : I have created this MR to start the discussion about the proposed change. Therefore, this MR is marked as Draft and only one file is adapted. Could you please check and let me know your view.

It is an interesting scenario the one you describe, thanks for the detailed explanation. Besides that this MR requires additional changes (other recipes, adapt artifacts to take into account these files, etc), let me ask you a few questions in different threads.

In which way you will use this information later? Should we add some parsing for it?. I understand that you prefer to handle that in separately, as each product team might have different scenarios.

The first step is to store the information beside the OS packs and images - as a preparation for the future.

The second step is to use this information to extract the APT snapshot information to e.g. do an APT snapshot cleanup. This would be an offline tool/script. A tool/script to parse the APT snapshot information from the APT source list is not needed on Apertis side because there is already a script available that does the parsing based on the metadata file (written by em).

Of course, we need a parser tool/script on Apertis pro side especially for the OS packs and images generated by the product teams. In principle, a tool/script developed on Apertis pro side should work on Apertis side as well. But this parser shall be handled separately. Atm, no need to add parsing functionality on Apertis side (based on APT source list).

Should auth information be handled in any way?

I will try to answer the question because I am not sure what is the intention of this question.

Starting with v2024 release the authentication information is separated into /etc/apt/auth.conf file and no longer part of the /etc/apt/sources.list file. The disclosure of credentials e.g. by storing on the image server should be avoided. Hence the auth information shall not be exported.

To parse the exported APT source lists, the auth information is not needed because access to APT repos is not required during this step.

Sorry for not being clear about the intention. As you mentioned in v2024 there were changes to avoid the use of auth info in /etc/apt/sources.list. I understand that this is working fine, but since this is not needed in Apertis I cannot confirm that.

I remember this support is in place for main images and docker images, but what about https://gitlab.apertis.org/infrastructure/apertis-lava-recipes/? For https://gitlab.apertis.org/infrastructure/tiny-image-recipes/ I don't think it is necessary as probably that information is dropped, but we need to check it. Of course, this only makes sense in the case that this support is needed for those images.

This is a very good hint. I propose to introduce the changes for the apt_source.sh script for all GitLab projects using this script to have a consistent solution across all these GitLab projects.

If continuing with the changes proposed by this MR, then the same changes are applicable for apertis-lava-recipes and tiny-image-recipes as well.

For v2025dev2: MRs prepared for https://gitlab.apertis.org/infrastructure/apertis-lava-recipes/ ( merge_requests/53 ) and https://gitlab.apertis.org/infrastructure/tiny-image-recipes/ ( merge_requests/136 ).

For v2024: MRs prepared for https://gitlab.apertis.org/infrastructure/apertis-lava-recipes/ ( merge_requests/54 ) and https://gitlab.apertis.org/infrastructure/tiny-image-recipes/ ( merge_requests/137 ).

Why the redirection "2>/dev/null">

A good question but no good answer. Redirection needs to be removed if we continue.

Just to have @em in the loop as he might have valid feedback.

requested review from @wlozano