Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

When deploying in a proxy-per-worker configuration, it’s useful to
consolidate all proxy instances on a single hostname, since DNS names
can be scarce.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

On lightweight runners, tests would sometimes run out of disk space
or memory or both.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Sjoerd Simons authored 2 years ago and Andrej Shadura committed 2 years ago

Create a helm chart for deploying the server side of the obs proxy. The
main thing that needs to be configured by users here is the ingress as
well as the obs backend server. The backend server when running in
kubernetes as well should be available via <cluster ip name>.<namespace>
as the dns name as per kubernetes conventions.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>

Allow the authentication token to come in via the environment makes it
easier to stash it into a k8s secret without requiring the whole
configuration file to be a secret.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>

Sjoerd Simons authored 2 years ago and Andrej Shadura committed 2 years ago

Create a trivial dummy endpoint which just returns a 200 reply. This is
useful e.g. for Kubernetes readiness tests to check if a service is up
and running.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>

Kaniko unfortunately cannot build multi-arch containers as it has no way
to create a manifest of more then one image. Switch to buildah as the
builder instead which can do this to build both an amd64 and arm64
container

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Keeping files open at all times prevents them from being cleaned up
properly, and when cleanups happen, the existing code couldn’t handle
it properly and would either error out, or continue logging into an
invisible file (referenced to only by its open file descriptor).

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Jinja 3.1 removed a bunch of deprecated symbols, leading to:

    from jinja2 import escape, Markup
E   ImportError: cannot import name 'escape' from 'jinja2'

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

The logging subsystem is more versatile and provides more features
than manual log prints. Timestamps are very often useful for tracking
down timing issues, enable them for the root logger.

This current logging configuration doesn’t include the module/logger
name in the message format as it clutters the logs. This may be
revisited later when the logging system is overhauled to support
structured logging and better tracing.

Quart/Hypercorn’s logger has its own log printing handler, so as we
currently only print logs on the stdout, we need to disable the
propagation to prevent access logs from being duplicated.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Default arguments are not cloned upon a function call, so once a default
value is mutated, it will be passed on to the function the next time it
is called, similarly to static variables in C. Since this is clearly a
footgun, don’t do that.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

We don’t test with 3.8 anymore, production systems run 3.9 too,
so this doesn’t reflect the reality.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

If an exception happens in the stream iterator, there is no guarantee
the response is cleaned up, potentially leading to resource leaks. To
avoid this, call .aclose() from the finally section.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

We actually want the routes to be imported when we use the blueprint.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Maximum connection limits run out quickly when talking to the backend,
possibly because they don’t expire for some not completely understood
reasons. Since the backend doesn’t talk HTTP/2 or TLS, connection
pooling doesn’t get utilised as much (or at all), so it doesn’t make
much sense to have one client rather than create a client per request or
a related group of requests thus keeping their lifetimes limited and
avoiding the issue altogether.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Since we’re currently pooling connections in a single client,
100 connections is a too low a limit given the number of connections
we need to handle.

In future, we should probably reduce the reliance on a single client
and only reuse a global client in some cases but not in the others.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

5xx errors tend to be from transient issues, so it may be worthwhile
retrying in such cases.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

The proxy support in websockets (proxied_ws) is closely tied to the
version 8 of websockets, which doesn’t support Python 3.10:

    obs_proxy.client: Caught an exception:
    obs_proxy.client: As of 3.10, the *loop* parameter was removed from Lock()
    since it is no longer necessary

Since we build Docker images based on Debian Bullseye, this doesn’t yet matter
as Python 3.9 is the default in Bullseye.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Python 3.10 removed this parameter.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>