Signed-off-by: monisha <Muthukumaran.Monisha@in.bosch.com>

Arnaud Ferraris authored 3 years ago and Frederic Danis committed 3 years ago

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

Walter Lozano authored 3 years ago and Martyn Welch committed 3 years ago

Since agl-compositor can be run in SDK images to facilitate development,
update the test case accordingly.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Walter Lozano authored 3 years ago and Martyn Welch committed 3 years ago

After the rebase on top of Bullseye and dropping the old framework
agl-compositor is the default compositor for Apertis target images,
so adjust the test case accordingly.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: sagar <SagarKishore.Benani@in.bosch.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

- Remove hmi-audio-detail-view test as it refers to deprecated Audio app,
- Update hmi-audio-play-pause test to use Rhythmbox Flatpak app instead
  of the deprecated Audio app.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This test is based on apparmor-pulseaudio.yaml.
pipewire.malicious.expected test will be implemented later.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

The latest PipeWire/WirePlumber use same naming as PulseAudio.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

PipeWire doesn't use the exact same name than PulseAudio.
Most of PulseAudio utilities can be used with PipeWire, except `pacmd`
which should be replaced by `pactl`.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Monisha Muthukumaran authored 3 years ago and Martyn Welch committed 3 years ago

Signed-off-by: Monisha <Muthukumaran.Monisha@in.bosch.com>

Benani Sagar Kishore authored 3 years ago and Martyn Welch committed 3 years ago

Signed-off-by: Sagar Kishore Benani <sagarkishore.benani@in.bosch.com>

Benani Sagar Kishore authored 3 years ago and Emanuele Aina committed 3 years ago

Corrected the file paths in the execution steps for picking the bluez-avrcp-volume.sh and aa_get_complaints.sh script

Denis Pynkin authored 3 years ago and Emanuele Aina committed 3 years ago

Added the `dry-run` mode feature testing.
Added the network polling time change feature testing.
Updated output for manual OTA rollback upgrade test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Benani Sagar Kishore authored 4 years ago and Emanuele Aina committed 3 years ago

Signed-off-by: Sagar Kishore Benani <SagarKishore.Benani@in.bosch.com>

We're not building target images with the internal bits since a while,
so stop referencing them from the testcases.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The common tests group currently blows up on LAVA when submitted to
QEMU-based DUTs as we do for the SDK and base SDK images because the
dispatcher runs out of disk space while unpacking the overlay before
uploading it to the DUT:

    Job error: tar_in: tar subcommand failed on directory: /:
    tar: 0/tests/1_disk-rootfs-fsck/apertis-tests/.git/objects/pack/pack-c1d7de14a133c29fdcacfeef5eba98291ada6c91.pack:
    Wrote only 2048 of 10240 bytes tar: 0/tests/1_disk-rootfs-fsck/apertis-tests/.git/packed-refs:
    Cannot write: No space left on device

In particular this is because the git repositoriee installed by the
apparmor-gstreamer1.0 and gstreamer1.0-decode testcases used to ship a
large copy of the Big Buck Bunny movie. Even if the test file has been
dropped from the HEAD of both repositories in favor of the media sample
file for the same movie pre-installed on the images, LAVA fails because
it does not do a shallow checkout but it ends up putting the whole git
hisotry in the overlay.

To side-step the issue, move all the apparmor tests still in the
`common-standalone` group to the `apparmor-standalone` group, but keep
skipping them on minimal images.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Added templates list for ostree-based profiles without AUM tests.
At the moment we do not support AUM tests for Amd64 and for internal
Arm32 images, hence use a base list for them.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This reverts commit 0dc36df3.

Revert this commit since it is breaking tests submission to LAVA with
`lqa` tool started in Apertis environment:

 Submitting job aum-ota-rollback-blacklist.yaml: <Fault 400: 'Problem
 with submitted job data: Loading job submission failed: while parsing
 a block mapping\n  in "<unicode string>", line 2, column 1\ndid not
 find expected key\n  in "<unicode string>", line 154, column 1.'>

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Emanuele Aina authored 4 years ago and Denis Pynkin committed 4 years ago

Commit 39e6ec4f made `tar` output the extracted filenames while
deployng the overlays and later commit 39e6ec4f made it
customizable since the AUM tests do many reboot cycles each with their
own (redundant) overlay deployment, causing the logs to grow massively
and going over the limits imposed by the LAVA web UI, which refuses to
display them and only offers to download them in JSON format.

Unfortunately the fix on 39e6ec4f worked only partially and the
overlays in the loop were still upacked in verbose mode, resulting in
unreadable logs.

For some reason that currently escapes me related to the way Jinja2
imports work, re-importing the template with the macros from the
looping template fixes the issue.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Submitting tests to LAVA from this repository has been broken for a long
time and has proven to be not useful. Yet, it generates plenty of noise
on the LAVA side, so disable the actual submission.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Add missing screenshot which shows the popup functionality of agl-compositor.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The new `psdk-test` requests `paramiko` to run.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

For AUM tests unpack of overlay with tests in verbose mode
produce too lot of almost useless information. This is the reason
why the job log exceeding 5MB limit and should be downloaded as
plain text which is hard to analyze.

Added common macros `tar_unpack_cmd()` which create dynamic unpack
string depending on variable `tar_additional_options` ('-v' by default).

Additional verbose option for `tar` was switched off for all AUM tests
to simplify log analysis in WEb interface.

It is possible to override default options with `lqa` command, for
example: `lqa -t tar_additional_options:""`.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Without quoting they could be assumed as number which will lead to
incorrect usage. For instance build `20210119.0130` was cut to
`20210119.013` leading to incorrect path to delta URL construction.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The new `psdk-test` perform the 3 previous tests (etc, home-user and
sysroot) without user interaction.
It can use a pre-downloaded virtual disk image, or directly the URL.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

python-dialog should be installed by apertis-tests, dialog itself is a
dependency of python-dialog so will be installed as well.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Starting from kernel 5.5 FDT names were split and
renamed to align with config options. This leads to
boot failure of old or new Apertis images depending of
naming schema used in `fdtfile` environment variable.

U-Boot used in Apertis and LAVA have additional feature
allowing to try the legacy FDT naming schema in cases
of boot failure with default naming schema.

On v2021 for the switching branches test we have to boot
images with old naming for DTB file in v2020 (5.4 kernel) and
new naming schema in v2021 (5.9 kernel).
So it is not possible to hardcode proper variable `fdtfile`
for booting both versions in LAVA.

Instead we remove hardcoded DTB file and use additional boot
target `legacy_mmc0` which automatically set the "old" name
for the DTB file in case if boot with a new one was failed.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The most of the tests are covered by automatic tests or by
other manual tests. Hence remove unneeded manual tests.

Lower priority for manual OTA rollback test since we have an
automation counterpair.

We keep both rollback tests for offline and online upgrades to
have a check point of our automation -- due complexity of the
testing process, automatic test _may_ have errors or false-positive
behavior in case if we change the AUM, U-Boot or libostree.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Fixed the name for the test -- it was the same as already existing.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The correct testing of rollback relies to U-Boot behavior.
In Apertis we expect the U-Boot should to use rollback configuration
in case if boot counter is above the limit.

Unfortunately we break the normal boot of SabreLite and Renesas boards
in LAVA by running direct boot with `run bootcmd_mmc0` command.

This completely breaks the correct workflow for rollback since it relies
to command `altbootcmd` which is a part of Apertis distro boot
modification.
The U-Boot bootloader should to drop the warning if boot counter is
truggering the rollback:

  File System is consistent
  file found, deleting
  File System is consistent
  Warning: Bootlimit (3) exceeded. Using altbootcmd.

and proceed with the call of `altbootcmd` instead of `distro_bootcmd`.
But since we stop the boot and do a manual boot from mmc0, alternative
boot is never called.

To proceed with the correct behavior after breaking the boot process we
check the boot limit additionally and choose the proper boot variant.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The path to git repository should be:
 https://gitlab.apertis.org/pkg/apertis-tests

otherwise LAVA throw an error:
 Infrastructure error: Unable to fetch git repository
 'https://gitlab.apertis.org/infrastructure/apertis-tests

'

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>