Baghmar Tarun authored 4 years ago and Emanuele Aina committed 4 years ago

If a test case requires multiple repositories it can now use a list of
objects as the OSTree preconditions to check them all out.

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Submit every modification to LAVA as part of the pipeline to set up a
direct feedback loop on non-main branches.

The image recipes pipeline can then re-use the job template here.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Port the below MR from apertis-tests, which got landed while we were
moving the templates around:
https://gitlab.apertis.org/pkg/development/apertis-tests/-/merge_requests/190



This reverts the introduction of the OTA update tests on the internal
minimal armhf since their behavior isn't stable and there is no value to
test AUM on this platform.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Keeping the LAVA job templates in `apertis-tests` has many problems.

Since `apertis-tests` is also a package, updates for stable releases
should go through `-updates` branches, but that plays badly with the
image building recipes as they only pull from the the main branch.

There's also an annoying disconnect about what the testcases here say
about where they should be run, and the actual groups used by the
templates.

By moving the LAVA templates here we can later improve them by
generating the groups based on the actual data in the testcases.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Hattihal Basavana Gowda Chethan authored 4 years ago and Emanuele Aina committed 4 years ago

When some test scripts got moved from /usr/share/chaiwala-tests/ to
/usr/lib/apertis-tests/ their path in the test definitions didn't get
updated, causing the tests to fail:

    + common/run-test-in-systemd --basename --user=user -- /usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh
    /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: 19: /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: Job for generated-test-case-test-valgrind.service failed because the control process exited with error code./usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh: not found

To fix the issue, rather that using the version shipped in
/usr/lib/apertis-tests/ use the one in the git repository since we
already have it checked out.

Fixes: https://phabricator.apertis.org/T6349
Fixes: https://phabricator.apertis.org/T6350
Fixes: https://phabricator.apertis.org/T6351
Fixes: https://phabricator.apertis.org/T6352



Signed-off-by: hattihalbasavanagowda.chethan <hattihalbasavanagowda.chethan@in.bosch.com>

This reverts commit 8ea0915d.
Disable internal armhf image from supported images types
for OTA update tests since their behavior aren't stable and
there is no value to test AUM on this platform.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 4 years ago

This commit fixes the name of the virtual machines for the psdk-test,
and make small fixes for typos and grammar issues.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>
Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 4 years ago

This commit updates the SDK persistent disk test cases to use the new
psdk-test script that semi-automate the execution of these tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

The templates hardcoded the `master` branch for the repositories
shipping the actual test scripts and binaries, but we use release-coded
branch names like `apertis/v2021dev2` there so the links in the
instructions pointed to the wrong place.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use names for the precondition variables that reflect the properties in
the testcase definitions so it's easier to spot what's going on.

This will alos make it easier to later expand them to use multiple
values.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Frederic Danis committed 4 years ago

1. sdk-flatpak-build-helloworld-app.yaml: building the app by using mildenhall runtime.
2. sdk-flatpak-helloworld-app.yaml: running the helloworld-app present in the server.

Signed-off-by: Tarun Baghmar <Tarun.Baghmar@in.bosch.com>

Add internal armhf image into supported images types
for OTA update tests.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Add the test for checking non-successful update during OTA update.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ensure that only updates signed with valid key can be applied.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Test OTA updates behavior with lack of disk space.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add automatic test of unattended network updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add automatic test for network updates API.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This reverts commit ab727249 because the
automation script still needs to be adapted to run under the
restrictions faced by RBEI. The test case will be updated once the
automation is updated.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>

This reverts commit b7e68a3f because the
automation script still needs to be adapted to run under the
restrictions faced by RBEI. The test case will be updated once the
automation is updated.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>

Baghmar Tarun authored 4 years ago and Martyn Welch committed 4 years ago

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Allow to use the board without fused Apertis SRK for limited
testing -- without validation of signed U-Boot.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This test ensures:
- U-Boot is compiled with HAB and FIT support
- SRK hash is fused and U-Boot is signed ('hab_status' call)
- U-Boot is able to verify signed OS image in FIT format
- U-Boot hangs in case if we try to boot with unsigned binary

Unfortunately the DUT in "open" mode assume any signature as valid,
hence it is not possible to check the boot hang for FIT image signed
with incorrect key -- the HW return 'success' while checking signature.

If the system is in 'closed' state then it is able to use incorrect
signature for the image, for example taken from document
"High Assurance Boot (HAB) for dummies" by Boundary Devises:
https://boundarydevices.com/high-assurance-boot-hab-dummies/



To check if the system is in "closed" state:
  => fuse read 0 6
     Reading bank 0:

     Word 0x00000006: 00000012

the last digit must be "2".

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

The `connmand` profile is now in enforce mode.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Baghmar Tarun authored 4 years ago and Sjoerd Simons committed 4 years ago

1. ofono-sms-recieve
2. ofono-sms-send

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Currently `apparmor-bluez-avrcp-volume` test request to run
`bluez-avrcp-volume` test, allowing to install all dependencies for both
tests. Running solely one test ends up in errors due to missing
dependencies.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Benani Sagar Kishore authored 5 years ago and Luis Araujo committed 5 years ago

Signed-off-by: Sagar Kishore Benani <SagarKishore.Benani@in.bosch.com>

Allow for Mildenhall components known to generate unrelated AppArmor
complaints.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

These ports are valid services that should be running in the images.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This commit automates the iptables-nmap test case to run from LAVA.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

This makes the test case name more explicit and keeps consistency with
the rest of the psdk tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 5 years ago

This commit updates the SDK persistent disk test cases to use the new
psdk-test script that semi-automate the execution of these tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Test case allows to install and check org.apertis.demo.curl application.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Luis Araujo authored 5 years ago and Emanuele Aina committed 5 years ago

Update test case to use the automated script.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

The minimal images does not embedded pulseaudio which is part of this test.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>