diff --git a/content/designs/jenkins-docker.md b/content/designs/jenkins-docker.md deleted file mode 100644 index e438c58d16bb776d77311c5ef05b7e87a7ea54e3..0000000000000000000000000000000000000000 --- a/content/designs/jenkins-docker.md +++ /dev/null @@ -1,208 +0,0 @@ -+++ -title = "Jenkins and Docker" -short-description = "Standardizing on Docker as the environment for Jenkins jobs" -weight = 100 -aliases = [ - "/old-designs/latest/jenkins-docker.html", - "/old-designs/v2019/jenkins-docker.html", - "/old-designs/v2020/jenkins-docker.html", - "/old-designs/v2021dev3/jenkins-docker.html", -] -outputs = [ "html", "pdf-in",] -date = "2019-09-27" -+++ - -# Jenkins and Docker - -This document provides a high-level overview of the reasons to adopt Docker for -the Jenkins jobs used by the Apertis infrastructure and covers the steps needed -to transition existing non-Docker jobs. - -## What is Jenkins - -Jenkins is the automation server that ties all the components of the Apertis -infrastructure together. - -It is responsible for: - -* building source packages from git repositories and submitting them to OBS -* building ospacks and images -* submitting test jobs to LAVA -* rendering documentation from Markdown to HTML and PDF and publishing it -* building sample app-bundles -* bundling test helpers - -## What is Docker - -Docker is the leading system to build, manage and run server applications in -a containerized environment. - -It simplifies reproducibility by: -* providing an easy way to build container images -* providing a registry for already built container images -* isolating the applications using the container images from the host system - -## Why Docker with Jenkins - -Running Jenkins jobs directly on a worker machine has several drawbacks: - -* all the jobs share the same work environment which can cause unwanted interactions -* the work environment has to be provisioned manually by installing packages on - the machine and hand-tweaking the configuration -* the work environment has to be kept up-to-date manually -* reproducing the same work environment on different workers is very error - prone as it relies on manual action -* customizing the work environment needs privileged operations -* the work environment can't be reproduced on developers' machines -* conflicting requirements (for instance, building against different releases) - cannot be fulfilled as the work environment is shared -* scaling is complex - -Jenkins jobs can be instead configured to use Docker containers as their -environment, which gives the advantages below: - -* each job runs in a separate container, giving more control about resource usage -* Docker containers are instantiated automatically by Jenkins -* rebuilding Docker containers from scratch to get the latest updates can be - done with a single click -* the containers provide a reproducible environment across workers -* Docker container images are built from `Dockerfiles` controlled by developers - using the normal review workflow with no special privileges -* the same container images used on the Jenkins workers can be used to - reproduce the work environment on developers' machines -* containers are ephemeral, a job changing the work environment does not affect - other jobs nor subsequent runs of the same job -* containers are isolated from each other and allow to address conflicting - requirements using different images -* several service providers offer Docker support which can be used for scaling - -## Apertis jobs using Docker - -Apertis already uses Docker containers for a few key jobs: in particular the -transition to Debos has been done by targeting Docker from the start, -greatly simplifying setup and maintenance compared to the previous mechanism. - -### Image recipes - -The [jobs building ospacks and images](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/tree/master/image-recipes) -use the [image-builder](https://gitlab.apertis.org/infrastructure/apertis-docker-images/tree/master/apertis-image-builder) -Docker container, based on Debian `stretch`. - -A special requirement for those jobs is that `/dev/kvm` must be made accessible -inside the container: particular care must then be taken for the worker -machines that will run these jobs, ruling out incompatible virtualization -mechanisms (for instance VirtualBox) and service providers that cannot provide -access to the KVM device. - -Developers can retrieve and launch the same environment used by Jenkins with a -single command: - -``` -$ docker run \ - -t docker-registry.apertis.org/apertis/apertis-18.09-image-builder \ - /bin/bash -``` - -### Documentation - -The [jobs building the designs and development websites](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/tree/master/hotdoc) -use the [documentation-builder](https://gitlab.apertis.org/infrastructure/apertis-docker-images/tree/master/apertis-documentation-builder) -Docker container, based on Debian `stretch`. - -Unlike the containers used to build images, the documentation builder does not -have any special requirement. - -### Docker images - -The Docker images used are generated and kept up-to-date -[through a dedicated Jenkins job](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/tree/master/docker-images) -that checks out the [docker-images](https://gitlab.apertis.org/infrastructure/apertis-docker-images/) -repository, uses Docker to build all the needed images and pushes them to -[our Docker registry](http://docker-registry.apertis.org/) to make them -available to Jenkins and to developers. - -## Converting the remaining jobs to Docker - -All the older jobs still run directly on a specifically configured worker -machine. By converting them to use Docker we would get the benefits listed -above and we would also be able to repurpose the special worker machine to -become another Docker host, doubling the number of jobs that can be run -in parallel. - -The affected jobs are: - -* [`packaging/*`](https://jenkins.apertis.org/job/apertis-18.09/job/packaging/) -* [`packages/*`](https://jenkins.apertis.org/job/apertis-18.09/job/packages/) -* [`samples/*`](https://jenkins.apertis.org/job/apertis-18.09/job/samples/) -* [`apertis-check-commit`](https://jenkins.apertis.org/job/phabricator/job/apertis-check-commit/) -* [`apertis-master-build-snapshot`](https://jenkins.apertis.org/job/phabricator/job/apertis-master-build-snapshot/) -* [`apertis-build-package-all-masters`](https://jenkins.apertis.org/job/apertis-build-package-all-masters/) - -### Creating a new Docker image - -The first step is to create a new Docker image to reproduce the work -environment needed by the jobs. - -A new [package-builder](https://gitlab.apertis.org/infrastructure/apertis-docker-images/tree/master/apertis-package-builder) -recipe is introduced. - -Unlike other images so far, this one is based on Apertis itself rather than -Debian. This means that a minimal Apertis ospack is produced during the build -and is then used to seed the `Dockerfile`, which installs all the needed -packages on top of it. - -### Converting the packaging jobs - -All the `packages/*` and `packaging/*` jobs are similar as they involve -checking out a git tree for a package, launching `build-snapshot` to build them -against the work environment and submit the resulting source package to OBS. - -Once all the dependencies have been made available in the work environment again, -[converting the job templates](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/commit/59af079a3b5) -only requires minor changes. - -### Converting the sample app-bundle jobs - -The jobs building the sample applications need `ade` and the dependencies of the app-bundles themselves. - -The changes required to -[switch the job template to use Docker](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/commit/c9562929b04) -are pretty similar to the ones required by the packaging jobs. - -### Converting the build-package-all-masters job - -This job's purpose is to check that no API breakage is introduced in -the application framework and HMI packages by building them from sources -in sequence. - -The changes required to -[switch the job template to use Docker](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/commit/c9582cd56b8) -are pretty similar to the ones required by the packaging jobs. - -### Converting the Phabricator jobs - -While the plan is to officially switch to GitLab for all the code reviews, the -jobs used to validate the patches submitted to Phabricator need to be ported to -avoid regressions. - -The changes [to port them to Docker](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/commit/60eb971fc0c) -are similar to the ones for the other jobs, but -additional fixes are needed to ensure they work smoothly in ephemeral Docker containers, -[relaxing ssh host keys checking](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/commit/3213294757f) and -[avoiding the interactive behavior of git-phab](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/commit/d9e932d1fd1). - - -## Steps to be taken by downstreams - -Downstreams are already likely to have a Docker-capable worker machine for their -Jenkins instance in order to run the Debos-based jobs. - -By merging the latest changes in the -[apertis-docker-images](https://gitlab.apertis.org/infrastructure/apertis-docker-images/) -repository a new `package-builder` image should be available in their -Docker registry. - -The updates to the templates in the -[apertis-jenkins-jobs](https://gitlab.apertis.org/infrastructure/apertis-jenkins-jobs/) -repository can then be merged and deployed to Jenkins to make use of the new -Docker image.