Skip to content
Snippets Groups Projects

Update branching process

Merged Dylan Aïssi requested to merge wip/daissi/branching-update into master
All threads resolved!
Files
15
@@ -70,17 +70,19 @@ Some terms in regard to *release branching*:
* `RELEASE`: The current release codename
* `NEXT_RELEASE`: The new release codename that should be branched to
* `PREV_STABLE_RELEASE`: The previous stable release codename. Only used when branching a new stable release (i.e. not a `dev` or `pre` release)
* `PURPOSE`: Use string `branching` to instruct CI that it is a release procedure job
* `NOACT`: Set to `0` to trigger the release
By default, the release procedure job will run in `dry-run` mode, i.e. with `NOACT:1`. It is a good practice to first run the release process in `dry-run` mode to uncover any possible issues. When ready, re-run the CI pipeline jobs with `NOACT:0` to run in `real` mode.
By default, the release procedure jobs will run in dry-run mode, i. e. with `NOACT` set to `1`. After running the release process in dry-run mode, go through the job logs to uncover potential issues. When ready, re-run the CI pipeline jobs with `NOACT` set to `0` to actually apply the changes.
![500px|thumb|Release Branching Trigger Summary](/images/Initial_Trigger.png)
![Release Branching Trigger Summary](/images/Initial_Trigger.png)
This is the inital step to trigger release branching. Here:
* We specify the current `RELEASE` to be branched, i.e. `v2023dev0`
* `NEXT_RELEASE` is the name of the new release that is to be branched, i.e. `v2023dev1`
* Specify the current `RELEASE` to be branched, i.e. `v2025dev2`
* `NEXT_RELEASE` is the name of the new release that is to be branched, i.e. `v2025dev3`
* If the new release to be branched is a stable release (i.e. not a `dev` or `pre` release), then `PREV_STABLE_RELEASE` needs to be defined, i.e. when branching `v2025`, this variable will be defined on `v2024`
* `PURPOSE` is set to `branching` to tell the CI Pipeline that this is a release branching job.
Note: `NOACT` has been deliberately left out in this example screenshot. So, in this case, the entire procedure will run in `dry-run` mode. To run in effective mode, pass `NOACT` with value `0`
@@ -92,16 +94,17 @@ The above example complete the *semi-automated* aspect of the **Release Branchin
After the completion of the **Release Branching** CI Jobs, a certain set of manual steps need to be performed, which are outlined below.
* Add a directory for the release to the LAVA NFS path at: `/srv/images/public/lava/nfs/` on host images.apertis.org. The LAVA test jobs pick the kernel and initrd image from the release's directory path. Refer to the README for details
* Instantiate a *new timer to rotate images* for the new release (eg. On host `aurora`, run command `systemctl enable --now rotate-images@v2019.timer`)
* Update the [ci-package-builder](https://gitlab.apertis.org/infrastructure/ci-package-builder/) to target the next release
* Update the [gitlab-rulez](https://gitlab.apertis.org/infrastructure/gitlab-rulez/) recipe to point to the new development release
* Set the default development branch for all git projects
* Set the protection settings onto all `apertis/*` branches, which were unprotected in the initial release steps
* Add a directory for the release to the LAVA NFS path at: `/srv/images/public/lava/nfs/` on host images.apertis.org. The LAVA test jobs pick the kernel and initrd image from the release's directory path. Refer to the README for details.
* Instantiate a *new timer to rotate images* for the new release (eg. On host `aurora`, run command `systemctl enable --now rotate-images@v2019.timer`).
* Update the [ci-package-builder](https://gitlab.apertis.org/infrastructure/ci-package-builder/) to target the next release.
* Update the [gitlab-rulez](https://gitlab.apertis.org/infrastructure/gitlab-rulez/) recipe to point to the new development release.
* Set the default development branch for all git projects.
* Set the protection settings onto all `apertis/*` branches, which were unprotected in the initial release steps.
* Update the [dashboard](https://gitlab.apertis.org/infrastructure/dashboard)'s channels to track the new release.
* Manually branch the [base-files](https://gitlab.apertis.org/pkg/base-files/) package, after the new docker images have been built.
* This step should automatically update the release specific attributes in the new package and auto propose the changes through a new merge request
* This step should automatically update the release specific attributes in the new package and auto propose the changes through a new merge request.
* Validate the newly created pipeline ensuring that it generates the right set of changes with respect to the new release.
* Validate the new package is built proper and pushed to OBS and the APT repositories
* Validate the new package is built proper and pushed to OBS and the APT repositories.
# Stable Point Release
@@ -316,27 +319,27 @@ in `~/.ssh/config`
The previous sections showed the steps to launch the branching through Gitlab CI. This section provides additional
information to developers on the use of this particular CI.
![500px|thumb|Stages Overview](/images/Stages.png)
![Release branching jobs queued in the pipeline](/images/Stages.png)
This is an overview of all the release branching jobs queued in the pipeline
![500px|thumb|Stages Dependency Overview](/images/Job_Dependencies.png)
![Release branching jobs in the pipeline with dependencies](/images/Job_Dependencies.png)
This is an example of the same jobs, with their dependencies chalked out. There are multiple jobs that depend on one another. This is to ensure that jobs, that depend on certain outputs or tasks to be completed, are run only after.
![500px|thumb|Dependent Run Overview](/images/Dependent_Run.png)
![Release branching jobs in the pipeline with dependencies, one job is still waiting on its dependency](/images/Dependent_Run.png)
This example highlights the dependency in action. A job is only run *after* its parent dependency job is run successfully. Jobs that do not have a connecting line are independent jobs that are run in parallel. In this exmaple, most jobs in the `branch` stage are *independent* and run in *parallel*
![500px|thumb|Manual Job Overview](/images/Manual_Job_Summary.png)
![Pipeline job blocked on manual dependency](/images/Manual_Job_Summary.png)
This example highlights *Manual Jobs* and their dependencies. *Manual Jobs* are differentiated with the *Play* Button. A *Manual Job* is to be run manually by a user. In the context of this document, a manual job constitutes of certain commands that the user is expected to run by hand on respective servers. Most manual jobs, in this case, are to be run on the **OBS Backend** server. For each manual job, the exact commands will be displayed in the job's console view.
![500px|thumb|Manual Job Run Detail](/images/Manual_Job_Run_Detail.png)
![Log of manual job showing commands user has to run](/images/Manual_Job_Run_Detail.png)
This example highlights the exact set of commands that should be manually run. Most commands to be manually run, are to be run on specific servers, which will be mentioned in the console output for the particular job.
![500px|thumb|Manual Job Run Dependency Chain](/images/Manual_Job_Run_Dependency_Chain.png)
![Dependency chain with manual job](/images/Manual_Job_Run_Dependency_Chain.png)
Most manual jobs are made dependent on other job. This is to ensure that steps are performed in a particular order. In the above example, the dependency is defined as: `obs_prjconf => obs_reprepro => obs_clone_binary`, where `obs_prjconf` runs a certain set of tasks, which are needed by `obs_reprepro`. Similarly `obs_reprepro` performs a certain set of tasks which are a pre-requisite for `obs_clone_binary`. The job chain is strictly defined and users should ensure to successfully run the manual jobs on respective servers, before progressing to the next job.
@@ -344,11 +347,11 @@ Most manual jobs are made dependent on other job. This is to ensure that steps a
There can be scenarios during the *Release Branching* wherein the user would be required to re-run the entire pipeline. Under such situation, it can be desired to have certain specific jobs to be skipped from the consecutive pipeline run. For example, in first run, `branch-create-mass-branches` was run and it created all the branches in respect to the `NEXT_RELEASE`. For such cases, on consecutive run, it is desired to skip the `branch-create-mass-branches` job. Below is an example on how to invoke the pipeline while defining a set of jobs to be skipped.
![500px|thumb|FORCE_SKIP_JOB_INVOKE](/images/FORCE_SKIP_JOB_INVOKE.png)
![Run pipeline page with FORCE_SKIP_JOB variable set](/images/FORCE_SKIP_JOB_INVOKE.png)
In the above example screenshot, we specify that the `branch-create-mass-branches` and `branch-update-tests-projects` jobs be skipped. The important bit is to pass the `FORCE_JOB_SKIP` variable with the required values.
![500px|thumb|FORCE_SKIP_JOB](/images/FORCE_SKIP_JOB.png)
![Pipeline with jobs skipped](/images/FORCE_SKIP_JOB.png)
As was specified in the previous job invocaton, this example pipeline job run has the `branch-create-mass-branches` and `branch-update-tests-projects` skipped.
Loading