Helper tools for tests
This repository contains tools used by the test infrastructure.
Workflow
This section describes the workflow for handling tests that need binaries which are not part of the immutable images.
The main point is that adding stuff on the images purely for testing is undesirable, as it makes them diverge from what an actual product image would look like. LAVA can fetch contents from git easily, so the overall idea is to put binaries in dedicated git repositories.
Tests that do not need extra binaries
These are easy, data and scripts can be put in a git repository and we're done.
For instance, see how webkit2gtk-drag-and-drop relies on HTML files from tests/webkit2gtk.
Tests that need extra binaries from one or more existing packages
These tests need to use the tests/helper-tools pipeline.
For instance the glib-gio-fs
testcases relies on
tests/glib-gio-fs to ship
binary test programs from the libglib2.0-tests
package.
To do so it lists the required files in
external-binaries.cfg
and the .gitlab-ci.yml
includes gitlab-ci/update-test-binaries.yaml
in this repository.
A daily schedule triggers the GitLab pipeline and:
- fetches the required packages (
libglib2.0-tests
in this case); - extracts the needed files;
- commits them to the repository (if they changed).
See this run as an example.
Tests that need to build new binaries
To handle this case, a new proper standard package needs to be created to build the required binaries on OBS.
See for instance the apparmor-pulseaudio-tester
package
used by the tests/apparmor-pulseaudio
repository
for the
apparmor-pulseaudio
testcase.
Contents of the repository
update_test_binaries.sh
:
update_test_binaries is a script used to update external binaries required by
tests and storing them in individual test case git repositories. It will read
the external binaries definition from external-binaries.cfg
located in the test
case git repository and update the binaries in the test repo accordingly.
The binaries will be copied in an architecture specific tree in the test repository.
gitlab-ci/update-test-binaries.yaml
:
The snippet to be included by the projects needing the external binaries.