Signed-off-by: Apertis CI <devel@lists.apertis.org>

Akshay M authored 3 years ago and Frederic Danis committed 3 years ago

Signed-off-by: Akshay M <M.Akshay@in.bosch.com>

LAVA version 2021.11 introduces a change in the way paths in compressed
overlays are handled breaking Apertis tests.

Fix the issue by specifying the path to be use to extract the compressed
overlay.

Additional information:

https://gitlab.collabora.com/lava/lava/-/commit/07048f042c65202eb75ac874cc93eabd2d96cd54



Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

aum-offline-upgrade-branch is not generated with following error:
template variable not defined in aum-offline-upgrade-branch: 'osname' is undefined

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

This reverts commit c000fc66.

This simplification breaks the qa-report-app and would make to much
confusion in that code. Let's keep nfsroot

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Frederic Danis committed 3 years ago

This test was created to verify some DBUS robustness patches that ended
up not being accepted upstream.

The downstream patches have been dropped since v2019 when we rebased to
Debian, so this test doesn't have value anymore.

Avoid random failures by removing it.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

RPi fails on AUM tests on Lava due to U-Boot.
This introduce a workaround to prevent those tests to be run.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

This is to be closer to the type that is used in apertis-lava-recipes.
This way, we avoid using more variables to cover image type (nfs) and
test type (nfsroot). Now, they are both called nfs.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This is the type used by the nfs images generated on the
apertis-lava-recipes repository.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

arm64-rpi64 bundles don't exist.
RPi64 boards use arm64-uboot bundles for the upgrade.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

This program is used to send the generated lava jobs to a lava server
using lqa.

It also sends the given metadata to the lavaphab bridge.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This program does not use LQA to generate the job descriptions.

It is capable of finding all the tests for a given image and group them
into the adequate template.

Each job is written into a yaml file: job-${image_name}-${group}.yaml

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

These tests were not referenced by any template and therefore don't have
a group.

Because no templates were referencing them, they were not executed as
part of the automated test suite. Their removal doesn't affect the test
results.

This test is run as part of the sanity checks.

Setting the type to sanity allows the generate-jobs script to know that
this test is not supposed to be in any group and can be ignored when
generating job files.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Each group template must be named
  group-[group_name]-tpl.yaml

A new lava/group-default-tpl.yaml template is added and will be used by
all groups that do not need specific variables set in the Job
Description.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This also add devices.yaml that specifies the list of device types and
its configuration for a given arch/board configuration.

A default configuration file is added as well.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

These groups are set according to the template name in which the test is
referenced.

The group of a test will be used to generate the different test jobs
that can be run with the same template on the same boot, without having
to keep each test name in the templates.

Note that only the automated tests have a group set.

Sanity checks are still hardcoded in templates header.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

https://phabricator.apertis.org/T8363



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Ryan Gonzalez authored 3 years ago and Andre Moreira Magalhaes committed 3 years ago

https://phabricator.apertis.org/T8363



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

--no-lava is used because the test step was adding ':pass' at the end of
each line.
This was then parsed by the `parse: pattern:` line.

The test is only printing values (nothing is actually checked) so there
is no need to add the `:pass`, just let `run-test-in-systemd` report the
status of the command to LAVA.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

LAVA provides several ways to report result from test cases, parsing the
output is one of them, however, it is discourage since it is error prone.
For that reason Apertis provides several scripts which make use of lava-*
scripts, such as run-test-in-systemd. Unfortunately, many test cases that
use those scripts still declare parse options which are misleading.

Drop the parsing options which make use of run-test-in-systemd to avoid
confusions.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

Test cases that uses run-test-in-systemd provides several options to set
test names, but by default they use the script name. Since in some cases
the name does not reflect the test being performed try to provide a more
rationale name.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Since apparmor-ofono runs a test suite with several tests fix the
parsing to allow LAVA to catch all the results.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Walter Lozano authored 3 years ago and Frederic Danis committed 3 years ago

Since apparmor-geoclue runs a test suite with several tests fix the
parsing to allow LAVA to catch all the results.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

The manual ADE test was meant to test the build and remote debugging of an
application with ADE, while other automated tests perform building checks.
After the switching to Flatpak, the use of ADE is only meant to handling
sysroots and not to debug applications, making this test useless.

Drop the manual test since it has not real value.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

On commit 2d75d3c0 several tests have been moved to Fixed Function in
order to follow the guideline to run all the AUM tests on this type of
images. However, apertis-update-manager-amd64-rollback test is still
run on HMI images, which has no sense. Move this test to Fixed
Function in order to follow the guideline.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The apparmor ofono test can now be run on OSTree images, but not on
fixedfunction images where ofono is not installed.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

As defined in test-cases/apparmor-session-lockdown-no-deny.yaml this test
doesn't apply for fixedfunction images.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Replace macro_ostree_preconditions by install entry to run on both APT and
OSTree images.
Command should not start by `$`

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Sudarshan CP authored 3 years ago and Ritesh Raj Sarraf committed 3 years ago

Signed-off-by: Monisha Muthukumaran <Muthukumaran.Monisha@in.bosch.com>

Ryan Gonzalez authored 3 years ago and Walter Lozano committed 3 years ago

It relies on legacy appfw packages that are no longer present.

https://phabricator.apertis.org/T7945



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Command should not start by `$`

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Latest version of the test is able to run on boards or SDKs, APT or OSTree.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>