Walter Lozano authored 7 months ago and Dylan Aïssi committed 6 months ago

Since as_list will be deprecated soon, switch to the equivalent
iterator=True.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Adapted from create-branches script

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

When we create the new branch we do not want to trigger a pipeline, as
that would overwhelm GitLab with thousands of useless jobs.

Even worse, with the latest ci-package-builder this would trigger a
horribly time consuming rebuild on OBS.

Unfortunately the GitLab API does not allow us to skip CI when creating
a new branch.

We worked around that by disabling CI completely on the repo, creating
the new branch, and then re-enabling CI. However, it does not work
reliably: when GitLab is under load, it ends up processing the branch
creation asynchronously, once CI has been already re-enabled.

When pushing via git, the `ci.skip` option does exactly what we want,
but we do not want to check out thousands of git repotories either.

But we do not have to check them out if we learn to speak the minimal
subset of the git HTTP protocol needed to push a single new ref!

Let's then manually craft the right HTTP request to emulate a git push
of the new ref over HTTP, passing the `ci.skip` option to create the
new branch without triggering any pipeline.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Ritesh Raj Sarraf authored 3 years ago and Emanuele Aina committed 3 years ago

Currently, when a repository has improper settings, create-branches will
fail with exception. While we can ascertain the cause, in current
exception, it isn't obvious which ones are the faulty project
repositories.

With this change, it should be able to print the project name.

```
Got 5756 projects
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/gitlab/exceptions.py", line 279, in wrapped_f
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/gitlab/mixins.py", line 296, in update
    return http_method(path, post_data=new_data, files=files, **kwargs)
  File "/usr/lib/python3/dist-packages/gitlab/__init__.py", line 713, in http_put
    result = self.http_request(
  File "/usr/lib/python3/dist-packages/gitlab/__init__.py", line 565, in http_request
    raise GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 403: 403 Forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 98, in _create_branch
    project.save()
  File "/usr/lib/python3/dist-packages/gitlab/mixins.py", line 385, in save
    server_data = self.manager.update(obj_id, updated_data, **kwargs)
  File "/usr/lib/python3/dist-packages/gitlab/exceptions.py", line 281, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabUpdateError: 403: 403 Forbidden
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/gitlab/exceptions.py", line 279, in wrapped_f
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/gitlab/mixins.py", line 296, in update
    return http_method(path, post_data=new_data, files=files, **kwargs)
  File "/usr/lib/python3/dist-packages/gitlab/__init__.py", line 713, in http_put
    result = self.http_request(
  File "/usr/lib/python3/dist-packages/gitlab/__init__.py", line 565, in http_request
    raise GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 403: 403 Forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 251, in <module>
    b.branch(args.branch_prefix, args.base_release, args.new_release, args.really)
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 180, in branch
    thread_pool(num_worker_threads, _branch, self.projects, num_retries=2)
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 59, in thread_pool
    results = [f.result() for f in futures]
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 59, in <listcomp>
    results = [f.result() for f in futures]
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 433, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 40, in func_with_retries
    func(item)
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 176, in _branch
    branch = _create_branch(project, base_branch, new_branch, really)
  File "/builds/infrastructure/apertis-infrastructure/release-scripts/./create-branches", line 122, in _create_branch
    project.save()
  File "/usr/lib/python3/dist-packages/gitlab/mixins.py", line 385, in save
    server_data = self.manager.update(obj_id, updated_data, **kwargs)
  File "/usr/lib/python3/dist-packages/gitlab/exceptions.py", line 281, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabUpdateError: 403: 403 Forbidden
Cleaning up project directory and file based variables
```

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Try to give more time to GitLab to realize we created a new branch so
hopefully it will stop creating pipelines for it.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

When naively creating new branches, a CI/CD pipeline is created for
each of them. This means thousands of pipelines created in a couple of
minutes, which end up clogging the runners system with jobs that in most
cases will either fail since dependencies are not there yet (for
instance, image builds will lack the docker images) or won't be useful
anyway (for instance, package uploads would be a no-op since OBS is
branched separatedly).

To avoid this, temporarily disable the CI/CD on the project before
creating the new branch, and re-enable it immediately afterwards.

This Python reimplementation of the script uses a thread pool to process
all the projects a lot faster, using the same approach as the one from
infrastructure/gitlab-rulez> and infrastructure/dashboard>.

The script should also be idempotent, interrupting and re-running it
should make it proceed from where it left.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

gitlab-rulez rules have been split into branch.yaml and rulez.yaml
wherein the latter is a general purpose yaml configuration. branc.yaml
is solely dedicated to release branching, hence use that.

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

It helps have passwordless access using SSH keys

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

These are some commands that should be used under the NOACT variables
state

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

The branching script requires GITLAB_TOKEN=… in ~/.gitlab-apertis-org-token.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrew Shadura <andrew.shadura@collabora.co.uk>
Reviewed-by: Emanuele Aina <emanuele.aina@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D7181