The LAVA lab is currently facing badnwidth issues. Switching to a
different ISP is planned, but due to the COVID impact there's no
progress on that front.

The current ade tests got badly impacted, with the download phase
consuming all the allocated time.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `source:wip:` metadata can be used to indicate that a submission
came from a work-in-progress branch and that it should not be accounted
when checking the overall health of the projects.

For instance, the Phabricator bridge can use this information to avoid
opening bug tasks for failures coming from a WIP branch.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Introduce the variables below:

* source_ref
* source_commit
* source_pipeline
* source_job

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In preparation for adding more metadata, make the `source` entry an
object rather than a single string.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Default to `null` rather than hardconding a possibly misleading source.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Emanuele Aina committed 4 years ago

If a test case requires multiple repositories it can now use a list of
objects as the OSTree preconditions to check them all out.

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Submit every modification to LAVA as part of the pipeline to set up a
direct feedback loop on non-main branches.

The image recipes pipeline can then re-use the job template here.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Port the below MR from apertis-tests, which got landed while we were
moving the templates around:
https://gitlab.apertis.org/pkg/development/apertis-tests/-/merge_requests/190



This reverts the introduction of the OTA update tests on the internal
minimal armhf since their behavior isn't stable and there is no value to
test AUM on this platform.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Keeping the LAVA job templates in `apertis-tests` has many problems.

Since `apertis-tests` is also a package, updates for stable releases
should go through `-updates` branches, but that plays badly with the
image building recipes as they only pull from the the main branch.

There's also an annoying disconnect about what the testcases here say
about where they should be run, and the actual groups used by the
templates.

By moving the LAVA templates here we can later improve them by
generating the groups based on the actual data in the testcases.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Hattihal Basavana Gowda Chethan authored 4 years ago and Emanuele Aina committed 4 years ago

When some test scripts got moved from /usr/share/chaiwala-tests/ to
/usr/lib/apertis-tests/ their path in the test definitions didn't get
updated, causing the tests to fail:

    + common/run-test-in-systemd --basename --user=user -- /usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh
    /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: 19: /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: Job for generated-test-case-test-valgrind.service failed because the control process exited with error code./usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh: not found

To fix the issue, rather that using the version shipped in
/usr/lib/apertis-tests/ use the one in the git repository since we
already have it checked out.

Fixes: https://phabricator.apertis.org/T6349
Fixes: https://phabricator.apertis.org/T6350
Fixes: https://phabricator.apertis.org/T6351
Fixes: https://phabricator.apertis.org/T6352



Signed-off-by: hattihalbasavanagowda.chethan <hattihalbasavanagowda.chethan@in.bosch.com>

This reverts commit 8ea0915d.
Disable internal armhf image from supported images types
for OTA update tests since their behavior aren't stable and
there is no value to test AUM on this platform.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 4 years ago

This commit fixes the name of the virtual machines for the psdk-test,
and make small fixes for typos and grammar issues.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>
Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 4 years ago

This commit updates the SDK persistent disk test cases to use the new
psdk-test script that semi-automate the execution of these tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

The templates hardcoded the `master` branch for the repositories
shipping the actual test scripts and binaries, but we use release-coded
branch names like `apertis/v2021dev2` there so the links in the
instructions pointed to the wrong place.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use names for the precondition variables that reflect the properties in
the testcase definitions so it's easier to spot what's going on.

This will alos make it easier to later expand them to use multiple
values.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Frederic Danis committed 4 years ago

1. sdk-flatpak-build-helloworld-app.yaml: building the app by using mildenhall runtime.
2. sdk-flatpak-helloworld-app.yaml: running the helloworld-app present in the server.

Signed-off-by: Tarun Baghmar <Tarun.Baghmar@in.bosch.com>

Add internal armhf image into supported images types
for OTA update tests.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Add the test for checking non-successful update during OTA update.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ensure that only updates signed with valid key can be applied.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Test OTA updates behavior with lack of disk space.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add automatic test of unattended network updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add automatic test for network updates API.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This reverts commit ab727249 because the
automation script still needs to be adapted to run under the
restrictions faced by RBEI. The test case will be updated once the
automation is updated.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>

This reverts commit b7e68a3f because the
automation script still needs to be adapted to run under the
restrictions faced by RBEI. The test case will be updated once the
automation is updated.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>

Baghmar Tarun authored 4 years ago and Martyn Welch committed 4 years ago

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Allow to use the board without fused Apertis SRK for limited
testing -- without validation of signed U-Boot.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This test ensures:
- U-Boot is compiled with HAB and FIT support
- SRK hash is fused and U-Boot is signed ('hab_status' call)
- U-Boot is able to verify signed OS image in FIT format
- U-Boot hangs in case if we try to boot with unsigned binary

Unfortunately the DUT in "open" mode assume any signature as valid,
hence it is not possible to check the boot hang for FIT image signed
with incorrect key -- the HW return 'success' while checking signature.

If the system is in 'closed' state then it is able to use incorrect
signature for the image, for example taken from document
"High Assurance Boot (HAB) for dummies" by Boundary Devises:
https://boundarydevices.com/high-assurance-boot-hab-dummies/



To check if the system is in "closed" state:
  => fuse read 0 6
     Reading bank 0:

     Word 0x00000006: 00000012

the last digit must be "2".

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

The `connmand` profile is now in enforce mode.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Baghmar Tarun authored 4 years ago and Sjoerd Simons committed 4 years ago

1. ofono-sms-recieve
2. ofono-sms-send

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Currently `apparmor-bluez-avrcp-volume` test request to run
`bluez-avrcp-volume` test, allowing to install all dependencies for both
tests. Running solely one test ends up in errors due to missing
dependencies.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Benani Sagar Kishore authored 4 years ago and Luis Araujo committed 4 years ago

Signed-off-by: Sagar Kishore Benani <SagarKishore.Benani@in.bosch.com>

Allow for Mildenhall components known to generate unrelated AppArmor
complaints.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

These ports are valid services that should be running in the images.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This commit automates the iptables-nmap test case to run from LAVA.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

This makes the test case name more explicit and keeps consistency with
the rest of the psdk tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>