Use the UML fakemachine/Debos backend in the image builder

The image builder so far required /dev/kvm to be available in the container.

Unfortunately this is not possible on workers hosted on cloud providers, like our autoscaling workers on AWS, nor on the Azure ones.

The chroot backend for Debos cannot run inside a container as it needs to do actually privileged operatins like mounting filesystems.

The new User Mode Linux backend for fakemachine/Debos allows us to avoid those issues and run unprivileged builds in places where KVM is not available, like cloud CI workers or like the Apertis SDK under VirtualBox.

This MR changes the way containers should be invoked. Exposing /dev/kvm is no longer needed, but UML needs the ptrace capability and a executable tmpfs mounted on /scratch (--cap-add=SYS_PTRACE --tmpfs /scratch:exec on the docker run command line).

The last build of the KVM-based version is saved as registry.gitlab.apertis.org/infrastructure/apertis-docker-images/v2021dev2-image-builder:latest-kvm.

added 1 commit

• a4d8464e - Don't set HOME to a non-existing directory

Compare with previous version

mentioned in merge request apertis-image-recipes!206 (merged)

added 5 commits

• fb3d41d7 - Convert calculate-release to before_script
• a8a4d944 - Tag build-base-rootfs job with kvm
• 2e23261d - Add a test debos run to test the image builder docker
• bd1cf4e9 - image-builder: Install qemu-user-static
• 03f0fc3c - Correct fakemachine setup

Compare with previous version

unmarked as a Work In Progress

mentioned in merge request !108 (closed)

added 12 commits

• b9de87f5 - Explicitly use ARG for variables used in FROM
• 203a5858 - image-builder: Add the UML bits for Debos
• 0d8dd16c - debos-wrapper: Use slirp-seq not slirp-helper
• 641758f9 - image-builder: Copy by name not number
• 2fc4395d - image-builder: Explain why /scratch is used
• da32d8c6 - image-builder: Correct package installation
• 13edbdea - image-builder: Don't set HOME to a non-existing directory
• 27466691 - gitlab-ci: Convert calculate-release to before_script
• 5cb9551f - gitlab-ci: Tag build-base-rootfs job with kvm
• e98e402c - image-builder: Add a test job buidling a simple Debos recipe
• 79dd6682 - image-builder: Install qemu-user-static
• 07eefc8c - image-builder: Fix the fakemachine setup

Compare with previous version

This should be landed at the same time as apertis-image-recipes!206 (merged) since the new images do not have a working fakemachine KVM backend and the Jenkins workers are not set up to run the UML backend.

What needs to happen is then:

1. people both this MR and apertis-image-recipes!206 (merged)
2. a latest-kvm tag is added to the then latest image-builder Docker image in the registry, just in case
3. this gets landed
4. apertis-image-recipes!206 (merged) is changed to use the latest tag for the image-builder image
5. apertis-image-recipes!206 (merged) is landed
6. the v2021dev2 debos-image-builder Jenkins job gets disabled

mentioned in commit apertis-image-recipes@2a905a79

mentioned in commit apertis-image-recipes@7c572ef1

mentioned in commit apertis-image-recipes@f583b700

mentioned in commit apertis-image-recipes@36cc6dd9

mentioned in commit apertis-image-recipes@693192c9

mentioned in commit apertis-image-recipes@0b5a1256

mentioned in commit apertis-image-recipes@7e688d3b

mentioned in commit apertis-image-recipes@822ce3fc

@martyn do you think it is useful to keep all the history on this MR, or should I squash some stuff?

@em I was happy with these patches as they didn't build upon each other (IIRC).

changed the description

changed the description

Now merging. Thanks @sjoerd and @martyn!

resolved all threads

merged

changed the description