Use job names that are identifiers rather than textual blurbs to pack
more information in a easier to consume way.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Update all references to v2021* releases to point to the v2021.0 one.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Martyn Welch authored 3 years ago and Martyn Welch committed 3 years ago

The legacy software framework is being dropped from v2022dev3, as a result
the tests for these services can also be dropped.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Emanuele Aina authored 3 years ago and Martyn Welch committed 3 years ago

With the update to LAVA 2021.05 jobs on the Minnowboard failed:

    Setting prompt string to ['Booting in blind mode', 'error: missing (.*) symbol.']
    ...
    Apertis v2020 apertis ttyS0
    auto-login-action timed out after 853 seconds

Apparently `kernel_start_message` is not working correctly in the
`context` section, so move it to the relevant `boot` sections.

It may also be related that the correct spelling is
`kernel-start-message` so I am not sure about anything anymore.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 3 years ago and Martyn Welch committed 3 years ago

With the update to LAVA 2021.05 a new deployment type has been added to
ensure that things work despite:

1. `/bin/sh` not pointing to Bash
2. a read-only `/` that prevents unpacking the overlay there

In particular the latter causes all the OSTree tests to fail when
downloading and unpacking the overlay with:

    ./lava-4147029/
    tar: ./lava-4147029: Cannot mkdir: Operation not permitted

See https://gitlab.collabora.com/lava/lava/-/commit/d621f93191591714449e67945ebd95cb6396727a


for the commit introducting the `apertis` type.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Sagar <SagarKihore.Benani@in.bosch.com>

Signed-off-by: Monisha <Muthukumaran.Monisha@in.bosch.com>

Since now maynard ships its own menu file, installing gnome-menus is not
longer needed, so drop it to avoid installing python and all its
dependencies.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Arnaud Ferraris authored 3 years ago and Frederic Danis committed 3 years ago

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

Walter Lozano authored 3 years ago and Martyn Welch committed 3 years ago

Since agl-compositor can be run in SDK images to facilitate development,
update the test case accordingly.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Walter Lozano authored 3 years ago and Martyn Welch committed 3 years ago

After the rebase on top of Bullseye and dropping the old framework
agl-compositor is the default compositor for Apertis target images,
so adjust the test case accordingly.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: sagar <SagarKishore.Benani@in.bosch.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

- Remove hmi-audio-detail-view test as it refers to deprecated Audio app,
- Update hmi-audio-play-pause test to use Rhythmbox Flatpak app instead
  of the deprecated Audio app.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This test is based on apparmor-pulseaudio.yaml.
pipewire.malicious.expected test will be implemented later.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

The latest PipeWire/WirePlumber use same naming as PulseAudio.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

PipeWire doesn't use the exact same name than PulseAudio.
Most of PulseAudio utilities can be used with PipeWire, except `pacmd`
which should be replaced by `pactl`.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Monisha Muthukumaran authored 3 years ago and Martyn Welch committed 3 years ago

Signed-off-by: Monisha <Muthukumaran.Monisha@in.bosch.com>

Benani Sagar Kishore authored 3 years ago and Martyn Welch committed 3 years ago

Signed-off-by: Sagar Kishore Benani <sagarkishore.benani@in.bosch.com>

Benani Sagar Kishore authored 3 years ago and Emanuele Aina committed 3 years ago

Corrected the file paths in the execution steps for picking the bluez-avrcp-volume.sh and aa_get_complaints.sh script

Denis Pynkin authored 3 years ago and Emanuele Aina committed 3 years ago

Added the `dry-run` mode feature testing.
Added the network polling time change feature testing.
Updated output for manual OTA rollback upgrade test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Benani Sagar Kishore authored 4 years ago and Emanuele Aina committed 3 years ago

Signed-off-by: Sagar Kishore Benani <SagarKishore.Benani@in.bosch.com>

We're not building target images with the internal bits since a while,
so stop referencing them from the testcases.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The common tests group currently blows up on LAVA when submitted to
QEMU-based DUTs as we do for the SDK and base SDK images because the
dispatcher runs out of disk space while unpacking the overlay before
uploading it to the DUT:

    Job error: tar_in: tar subcommand failed on directory: /:
    tar: 0/tests/1_disk-rootfs-fsck/apertis-tests/.git/objects/pack/pack-c1d7de14a133c29fdcacfeef5eba98291ada6c91.pack:
    Wrote only 2048 of 10240 bytes tar: 0/tests/1_disk-rootfs-fsck/apertis-tests/.git/packed-refs:
    Cannot write: No space left on device

In particular this is because the git repositoriee installed by the
apparmor-gstreamer1.0 and gstreamer1.0-decode testcases used to ship a
large copy of the Big Buck Bunny movie. Even if the test file has been
dropped from the HEAD of both repositories in favor of the media sample
file for the same movie pre-installed on the images, LAVA fails because
it does not do a shallow checkout but it ends up putting the whole git
hisotry in the overlay.

To side-step the issue, move all the apparmor tests still in the
`common-standalone` group to the `apparmor-standalone` group, but keep
skipping them on minimal images.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Added templates list for ostree-based profiles without AUM tests.
At the moment we do not support AUM tests for Amd64 and for internal
Arm32 images, hence use a base list for them.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This reverts commit 0dc36df3.

Revert this commit since it is breaking tests submission to LAVA with
`lqa` tool started in Apertis environment:

 Submitting job aum-ota-rollback-blacklist.yaml: <Fault 400: 'Problem
 with submitted job data: Loading job submission failed: while parsing
 a block mapping\n  in "<unicode string>", line 2, column 1\ndid not
 find expected key\n  in "<unicode string>", line 154, column 1.'>

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Emanuele Aina authored 4 years ago and Denis Pynkin committed 4 years ago

Commit 39e6ec4f made `tar` output the extracted filenames while
deployng the overlays and later commit 39e6ec4f made it
customizable since the AUM tests do many reboot cycles each with their
own (redundant) overlay deployment, causing the logs to grow massively
and going over the limits imposed by the LAVA web UI, which refuses to
display them and only offers to download them in JSON format.

Unfortunately the fix on 39e6ec4f worked only partially and the
overlays in the loop were still upacked in verbose mode, resulting in
unreadable logs.

For some reason that currently escapes me related to the way Jinja2
imports work, re-importing the template with the macros from the
looping template fixes the issue.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Submitting tests to LAVA from this repository has been broken for a long
time and has proven to be not useful. Yet, it generates plenty of noise
on the LAVA side, so disable the actual submission.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Add missing screenshot which shows the popup functionality of agl-compositor.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The new `psdk-test` requests `paramiko` to run.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

For AUM tests unpack of overlay with tests in verbose mode
produce too lot of almost useless information. This is the reason
why the job log exceeding 5MB limit and should be downloaded as
plain text which is hard to analyze.

Added common macros `tar_unpack_cmd()` which create dynamic unpack
string depending on variable `tar_additional_options` ('-v' by default).

Additional verbose option for `tar` was switched off for all AUM tests
to simplify log analysis in WEb interface.

It is possible to override default options with `lqa` command, for
example: `lqa -t tar_additional_options:""`.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>