There is no release folder for 2023dev2 so that we may need to
specify a date for the image, just go back to 2023dev1

Add small helper script ltp.sh to generate the test jobs on a
imx6 platform.

Signed-off-by: Julien Massot <julien.massot@collabora.com>

LTP project offers test for each Linux Kernel syscall.

Signed-off-by: Julien Massot <julien.massot@collabora.com>

Detlev Casanova authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Emanuele Aina committed 2 years ago

The grilo test installs 2 packages as dependencies. On OSTree, apt is
not available, so the test cannot be run on these images.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Add usage of boot-delay service to be able to know and have time to reset
the board before boot completes.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Emanuele Aina authored 3 years ago and Frederic Danis committed 2 years ago

Merge the amd64 steps in the main testcase to avoid duplication.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Apertis CI <devel@lists.apertis.org>

Akshay M authored 3 years ago and Frederic Danis committed 3 years ago

Signed-off-by: Akshay M <M.Akshay@in.bosch.com>

LAVA version 2021.11 introduces a change in the way paths in compressed
overlays are handled breaking Apertis tests.

Fix the issue by specifying the path to be use to extract the compressed
overlay.

Additional information:

https://gitlab.collabora.com/lava/lava/-/commit/07048f042c65202eb75ac874cc93eabd2d96cd54



Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

aum-offline-upgrade-branch is not generated with following error:
template variable not defined in aum-offline-upgrade-branch: 'osname' is undefined

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

This reverts commit c000fc66.

This simplification breaks the qa-report-app and would make to much
confusion in that code. Let's keep nfsroot

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Frederic Danis committed 3 years ago

This test was created to verify some DBUS robustness patches that ended
up not being accepted upstream.

The downstream patches have been dropped since v2019 when we rebased to
Debian, so this test doesn't have value anymore.

Avoid random failures by removing it.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

RPi fails on AUM tests on Lava due to U-Boot.
This introduce a workaround to prevent those tests to be run.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

This is to be closer to the type that is used in apertis-lava-recipes.
This way, we avoid using more variables to cover image type (nfs) and
test type (nfsroot). Now, they are both called nfs.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This is the type used by the nfs images generated on the
apertis-lava-recipes repository.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

arm64-rpi64 bundles don't exist.
RPi64 boards use arm64-uboot bundles for the upgrade.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

This program is used to send the generated lava jobs to a lava server
using lqa.

It also sends the given metadata to the lavaphab bridge.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This program does not use LQA to generate the job descriptions.

It is capable of finding all the tests for a given image and group them
into the adequate template.

Each job is written into a yaml file: job-${image_name}-${group}.yaml

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

These tests were not referenced by any template and therefore don't have
a group.

Because no templates were referencing them, they were not executed as
part of the automated test suite. Their removal doesn't affect the test
results.

This test is run as part of the sanity checks.

Setting the type to sanity allows the generate-jobs script to know that
this test is not supposed to be in any group and can be ignored when
generating job files.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Each group template must be named
  group-[group_name]-tpl.yaml

A new lava/group-default-tpl.yaml template is added and will be used by
all groups that do not need specific variables set in the Job
Description.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This also add devices.yaml that specifies the list of device types and
its configuration for a given arch/board configuration.

A default configuration file is added as well.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

These groups are set according to the template name in which the test is
referenced.

The group of a test will be used to generate the different test jobs
that can be run with the same template on the same boot, without having
to keep each test name in the templates.

Note that only the automated tests have a group set.

Sanity checks are still hardcoded in templates header.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

https://phabricator.apertis.org/T8363



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Ryan Gonzalez authored 3 years ago and Andre Moreira Magalhaes committed 3 years ago

https://phabricator.apertis.org/T8363



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

--no-lava is used because the test step was adding ':pass' at the end of
each line.
This was then parsed by the `parse: pattern:` line.

The test is only printing values (nothing is actually checked) so there
is no need to add the `:pass`, just let `run-test-in-systemd` report the
status of the command to LAVA.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

LAVA provides several ways to report result from test cases, parsing the
output is one of them, however, it is discourage since it is error prone.
For that reason Apertis provides several scripts which make use of lava-*
scripts, such as run-test-in-systemd. Unfortunately, many test cases that
use those scripts still declare parse options which are misleading.

Drop the parsing options which make use of run-test-in-systemd to avoid
confusions.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

Test cases that uses run-test-in-systemd provides several options to set
test names, but by default they use the script name. Since in some cases
the name does not reflect the test being performed try to provide a more
rationale name.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>