qa-report-app-admin.md

+++
title = "QA Report App Administration"
toc = true
date = "2022-10-07"

aliases = [
    "/guides/qa-report-app-admin",
]
+++

[QA Report App](https://gitlab.apertis.org/infrastructure/qa-report-app) is the
Apertis tool for reporting and viewing the results of both manual and automated
(via LAVA) QA testing.

# Primary Configuration File

QA Report App's configuration is handled via the file `secrets/config.yaml`.

## General Options

### Resource Links and virtual location

QA Report App needs to be able to generate links to the test cases and images:

```yaml
app-url: https://qa.apertis.org
image-root: https://images.apertis.org
```

`app-url` should be the URL where QA Report App is running at, and `image-root`
should point to the web location where images downloads are available at.

Additionally, in cases where the app needs to be mounted under a specific URL
prefix (its virtual location), the `url-prefix` option can be set:

```yaml
url-prefix: /URL-PREFIX
```

Where the prefix must start with a leading slash (`/`). Setting this option will
cause the prefix to be automatically added to all URLs generated by the application.

Behind the scenes this will cause the WSGI [SCRIPT_NAME](https://wsgi.readthedocs.io/en/latest/definitions.html#envvar-SCRIPT_NAME)
to be set to this prefix and the [PATH_INFO](https://wsgi.readthedocs.io/en/latest/definitions.html#envvar-PATH_INFO) of any request
which is prefixed with this value to be stripped of the prefix.

### Debug Logging

Debug logging from the service can be enabled via:

```yaml
debug: true
```

### Open ID Connect user login

To activate authentication, the following configuration options must be set:

```yaml
openid:
  client-id: CLIENT-ID
  client-secret: CLIENT-SECRET
  well-known-url: WELL-KNOWN-URL
  always-require-login: BOOL  # (optional)
```

Both `client-id` and `client-secret` are created by the OpenID Connect
identity provider.
It should also provide the `well-known-url`.

E.g.: For using a Gitlab instance as OpenID Connect identity provider,
an application must be added on gitlab, with the openid scope and the
`https://<qa-report-app-server>/openid_callback` callback URL.
For Gitlab, the `well-known-url` value is
`https://<gitlab-instance>/.well-known/openid-configuration`.

`always-require-login` is an optional boolean value.
When set to `true`, the website will require users to be logged in to access
any page. This is set to `false` by default.

### Authentication groups

In addition to the login options described above, a set of authentication
groups can be defined for a more fine grained control on which users/groups
are allowed to login and their respective permissions:

```yaml
auth-groups:
  - name: GROUP-1
  - name: GROUP-2
    extra-perms:
    - submit-reports
    - tag-images
```

Each group can optionally configure a set of extra permissions (defaults to empty).
The supported values for `extra-perms` are:
- submit-reports: users in this group can submit manual test reports
- tag-images: users in this group can tag images

If `auth-groups` is omitted or empty (default), any user with the right credentials
will be able to login, submit reports and tag images.

### LAVA Authentication

QA Report App uses shared secrets to authenticate legitimate LAVA callback
submissions, which can be set in the config file:

```yaml
lava-callback-tokens:
  - TOKEN-1
  - TOKEN-2
  - ...
```

Additionally, they can be set in the
[environment](#environment-for-docker-compose).

In order to obtain a new token, you need to create a new token in the LAVA UI
(`API` → `Authentication Tokens`) for the user submitting the `notify` callback.
The token's name should match the name set in the `notify`→`callback`→`token`
property of the LAVA job definition YAML.

### Filter displayed releases

The `display` configuration section can be used to alter the minimal version to
be displayed, as well as the number of results displayed for each release on
the main page.

This makes it easier to go through the results.

The section is optional and can be configured as follows:

```yaml
display:
  min-version: MIN-VERSION
  series-results:
    dailies: RESULT-COUNT
    releases: RESULT-COUNT
    weeklies: RESULT-COUNT
```

`MIN-VERSION` is a string value that specifies the minimal version to be shown.
E.g.: `'v2021'`.
All results for versions inferior to `MIN-VERSION` will not be shown on the
main page.

Each `RESULT-COUNT` is an integer value that specifies the number of results
to show for each release.
E.g.: At the start of year 2023, with

```yaml
display:
  min-version: 'v2022'
  series-results:
    dailies: 7
    releases: 2
    weeklies: 4
```

the displayed results will be:

- for releases: show results for v2024dev0.0, v2023.0 & v2023pre.0 (last 2 release results in the v2023 series) and v2022.3 & v2022.2
- for dailies: show last 7 results for each of v2024, v2023 and v2022 daily builds
- for weeklies: show last 4 results for each of v2024, v2023 and v2022 weekly builds

## Defining Supported Platforms and Deployments

QA Report App is designed to take in test case results from a given *platform*
that the test cases were run on, such as `fixedfunction-armhf-uboot-public` or
`minimal-armhf-uboot-internal`, along with a *deployment* describing how the OS
was installed onto the platform (e.g. `apt`, `ostree`, `lxc`, `nfs`).

The support platforms and deployments can be configured via the `images` section
of the configuration file:

```yaml
images:
  architectures:
    ...
  variants:
    ...
  deployments:
    ...
  platform-groups:
    ...
```

### Defining Variants and Architectures

Image platforms as described contain both *variants* (different versions of the
same distribution) and *architectures*. For example, given the following
platforms:

- fixedfunction-armhf-uboot-public
- tiny-lxc-amd64-amd64-uefi-internal

Each of these starts with a variant (`fixedfunction`, `tiny-lxc`) that is
immediately followed by the architecture (`armhf`, `amd64`). The supported
variants and architectures should be defined at the top of the `images` section
of the config file:

```yaml
images:
  architectures:
    - amd64
    - arm64
  variants:
    - fixedfunction
    - tiny-lxc
```

### Defining Deployments

```yaml
images:
  deployments:
    user-selectable:
      NAME-1: DESCRIPTION 1
    lava-only:
      - NAME-2
      - NAME-3
```

Deployments that can be selected by the user when manually submitting test
results go in the `user-selectable` section, with each deployment name mapped
to a human-readable description. For instance, we may have:

```yaml
user-selectable:
  ostree: OSTree
```

In this case, `ostree`, is the deployment name, and `OSTree` is the
human-readable description that will be shown in the UI.

If, instead, a deployment should *not* appear for manual test result submission,
but instead only be available for automatically reported LAVA results, the
deployment names should be placed in the `lava-only` section, e.g.:

```yaml
lava-only:
  - lxc
  - nfs
```

No human-readable name is required, since these are never shown explicitly in
the UI.

### Defining Platforms

```yaml
images:
  platform-groups:
    - platforms:
        PLATFORM-1: DESCRIPTION 1
        PLATFORM-2: DESCRIPTION 2
      deployments: [DEPLOYMENT-1, DEPLOYMENT-2]
      supported-by:
        - first:
            release: FIRST-RELEASE
            version: FIRST-VERSION
          last:
            release: LAST-RELEASE
            version: LAST-VERSION
    - ...
```

Image platforms, such as `fixedfunction-armhf-uboot-public`, are defined in
groups called "platform groups". Each group contains:

- A list of platforms defined in this group.
- A list of deployments supported by the given platforms.
- (Optional) A list of release/version ranges that support the given platforms.

The purpose of placing platforms into these groups is to be able to define
multiple platforms that support the same deployments same conditions in one
place, reducing repition.

Example of some basic platform groups:

```yaml
platform-groups:
  - platforms:
      tiny-lxc-armhf-uboot-public: Tiny LXC ARM
      tiny-lxc-arm64-uboot-public: Tiny LXC ARM64
      tiny-lxc-amd64-uefi-public: Tiny LXC AMD64
    deployments: [lxc]

  - platforms:
      nfsroot-armhf-uboot-public: Nfsroot ARM
      nfsroot-arm64-uboot-public: Nfsroot ARM64
      nfsroot-amd64-uefi-public: Nfsroot AMD64
    deployments: [nfs]
```

#### Platform Support Ranges

```yaml
supported-by:
  - first:
      release: FIRST-RELEASE
      version: FIRST-VERSION
    last:
      release: LAST-RELEASE
      version: LAST-VERSION
  - first:
      release: FIRST-RELEASE
      version: FIRST-VERSION
    last:
      release: LAST-RELEASE
      version: LAST-VERSION
```

The `supported-by` key defines a list of *ranges* of releases that support the
platforms in the current group.

The individual ranges inside have two keys, at least one of which must be given:

- `first`: the first / oldest version that supports this platform. If not
  given, it is assumed that the platform has been supported since the very first
  release.
- `last` defines the last / newest version that supports this platform. If not
  given, it is assumed that the platform is still supported by the very latest
  release.

Each of these endpoints of the range in turn contains `release` and `version`,
to set the release and version of the given endpoint. The version is optional
and, if not given, all versions in the release are counted. For example, this
platform group is valid for all releases from v2021 to v2022 version 20220624:

```yaml
supported-by:
  - first:
      release: v2021
    last:
      release: v2022
      version: '20220624'
```

The end ranges can be defined to be exclusive instead of inclusive, via:

```yaml
supported-by:
  - first:
      release: v2021
      inclusive: false
```

This would normally mean "every release since *and including* v2021 is
supported", but setting `inclusive: false` changes it to mean "every release
since *and not including* v2021".

Multiple support ranges can be given:

```yaml
supported-by:
  # Supported from v2019 to v2021 and from v2022dev0 to v2022dev2.
  - first:
      release: v2019
    last:
      release: v2021
  - first:
      release: v2022dev0
    last:
      release: v2022dev2
```

Some more examples:

```yaml
# Supported from every release since, and including, v2022dev3, 20210910:
supported-by:
  - first:
      release: v2022dev3
      version: '20210910'

# Supported from every version *after* v2021, 20210910 up until, and including,
# v2022.
supported-by:
  - first:
      release: v2021
      version: '20210910'
      inclusive: false
    last:
      release: v2022
```

## Changing the Test Cases Source

Test cases are, by default, pulled from the [Apertis test cases Git
repository](https://gitlab.apertis.org/tests/apertis-test-cases). The individual
YAML files describing test cases are then taken from every branch starting with
`apertis/`, with the text afterwards being treated as the release (e.g. given
`apertis/v2020`, the release would be `v2020`). If you want to change the URL
and the branch prefix (the `apertis/`), set `test-cases-repository`:

```yaml
test-cases-repository:
  url: THE-URL
  branch-prefix: THE-BRANCH-PREFIX
```

where:

- `URL` is a URL pointing to a Git repository containing the test cases.
- `BRANCH-PREFIX` is the prefix

Note that the portion of the branch name after the branch prefix is treated as
the release/suite.

## Bug Tracker Integration

### Phabricator

```yaml
bug-tracking: phabricator
phabricator-url: https://PHRABRICATOR-URL/
phabricator-token: PHABRICATOR-TOKEN
phabricator-tasks:
  space: S2
  filter-tag: PRIMARY-TAG
  tags:
    - ADDITIONAL-TAG-1
    - ADDITIONAL-TAG-2
    - ...
```

This sets up integration with Phabricator, for automated issue creation.
`PHABRICATOR-TOKEN` is a Phabricator Conduit API token, which can be obtained
via the following steps:

- Click your profile picture in Phabricator on the top-right -> Settings.
- Select *Conduit API Tokens* in the left sidebar.
- Click the *Generate Token* button.

The `phabricator-tasks` section controls the behavior of tasks that are
automatically created by QA Report App from LAVA failures.

`space` is the Phabricator space that should have visibility into the created
tickets.

`filter-tag` is a Phabricator tag should identify all tasks automatically filed;
new tasks filed will have this tag assigned, and existing ones will be searched
using this tag. `tags` defines any additional tags that should be added when
opening a new ticket (but these are not used for any other purpose).

### GitLab

```yaml
bug-tracking: gitlab
gitlab-url: https://GITLAB-URL/
gitlab-token: GITLAB-TOKEN
gitlab-project: GITLAB/PROJECT
gitlab-issues:
  filter-label: PRIMARY-LABEL
  labels:
    - ADDITIONAL-LABEL-1
    - ADDITIONAL-LABEL-2
    - ...
```

This sets up integration with GitLab issues, for automated issue creation.
`GITLAB-TOKEN` should be a [GitLab access token]
https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token)
(use the *api* scope), and `GITLAB/PROJECT` should be the GitLab project where
issues are stored (e.g. `infrastructure/apertis-issues`).

The `gitlab-issues` section controls the behavior of issues that are
automatically created by QA Report App from LAVA failures. `filter-label` is a
GitLab label that should identify all issues automatically filed; new issues
filed will have this label assigned, and existing ones will be searched using
this label. `labels` defines any additional labels that should be added when
opening a new issue (but these are not used for any other purpose).

Note that `gitlab-url` is additionally used for configuring OAuth logins and
thus may be set even if GitLab issues integration is not set up.

# Environment for docker-compose

In addition to the standalone configuration file, a separate file `.env`,
providing environment variables to docker-compose, can be created in the
qa-report-app directory. These variables are used to configure hosting aspects,
rather than the behavior of the application itself.

The following variables are available to be set, as well as their default
values:

```bash
# The port the application will be exposed on.
QA_REPORT_APP_PORT=28080
# The user ID and group ID the application and database will run as. In
# particular, this results in the database files also being owned by this user
# and group.
RUN_USER=1000:1000
# If lava-callback-tokens is not set in the config file, it should be set here.
LAVA_CALLBACK_TOKENS=["TOKEN-1", "TOKEN-2"]
```