Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

Test case of iptables-nmap fails on SDK with error

Unable to apt-get update in lxc container
Certificate verification failed: The certificate is NOT trusted. The
certificate issuer is unknown.  Could not handshake: Error in the
certificate verification. [IP: 199.232.142.132 443]

Since the problem is only present on QEMU without a clear cause, after
checking that clock time is OK workaround it by using http instead of
https.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The expected logo is pointing to an old wifi url, which is missing.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Frederic Danis authored 3 years ago and Arnaud Ferraris committed 3 years ago

`target` has been replaced by `hmi` images.
And some AUM tests are only run on `fixedfunction` images.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Flatpak apps are now installed by default on `hmi` images. Moreover, we 
now ship Totem instead of the "VIDEO & TV application", so make sure the 
test case reflects the new state.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

Frederic Danis authored 3 years ago and Arnaud Ferraris committed 3 years ago

`armhf` is supported by `hmi` images, update test files accordingly;

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

`target` images are being renamed to `hmi` in order to have a more
descriptive name, so the test cases must be updated accordingly.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

The fixedfunction image should serve a minimal static web page.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

The test case for traprain requires development and including gpgv to be
run, but installing them on target/hmi makes the test useless since it
does not resemble a standard target/hmi image. Drop it only run in on
sdk.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

As defined in the test description, this test should not be run on
fixedfunction images.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

We don't support `internal` images anymore.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Rename `minimal` to `fixedfunction` to reflects the "Fixed function device
scenario" from Apertis Platform Technical Vision.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Drop tests for didcot, frome and rhosydd since these packages are not
longer supported by Apertis.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

traprain test needs to install devscripts which depends on gnupg2 packages,
and ultimately on gpgv package. When running on target board which embed
the sequoia-gpgv replacement, the installation procedure fails due to the
conflict between sequoia-gpgv and gpgv.
gpgv package should be explicitly installed to replace sequoia-gpgv.

frome and rhosydd have the same dependency problem.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Drop invalid reference to ribchester.yaml as it is not longer supported
in v2022dev3.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Now that the aum tests no longer redeploy the overlay at every reboot
there's no need to disable verbose reporting in those cases.

Since that was the entire purpose of the `tar_unpack_cmd()` macro and
the relative supporting code, let's drop it.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Deploy the overlay only on the first boot, rather than re-deploying it
every time we reboot.

We deploy it under `/var` so it is preserved over reboots: re-deploying
only makes the tests more time consuming.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The first reboot is not needed, so skip it to avoid wasting precious
time on the devices under tests.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

SDK should be able to run Docker containers out-of-the-box.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Since canterbury is not longer supported drop related tests. During this
process keep the file group-modules-standalone-tpl to follow the structure
of other tests even if it is now, at least momentarily, not very useful.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The test for rhosydd install several packages on target images, one of them
is ssl-cert, however, it fails to install since it depends on bash.

Since bash is an essential package, there is no sense is adding it as
a dependency, so update the testcase to install it.

This issue is not present in SDK images since bash is preinstalled,
but this fix should not cause any consequences.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: monisha <Muthukumaran.Monisha@in.bosch.com>

Monisha Muthukumaran authored 3 years ago and Frederic Danis committed 3 years ago

Signed-off-by: monisha <Muthukumaran.Monisha@in.bosch.com>

Followup on commit 4e9a8b1d to apply the fix on the snippet
rebooting devices during the Apertis Update Manager tests, which
are currently failing since they can't detect the prompt after the
kernel has booted.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

`apparmor-pipewire` currently fails on QEmu with following logs:
```
+ cd apparmor-pipewire
+ common/run-test-in-systemd --name run-test-sh --timeout 120 ./run-test.sh
#=== running test script: /var/lib/lava-4322051/1/tests/7_apparmor-pipewire/apparmor-pipewire/common/run-test-in-systemd --user=1000 /var/lib/lava-4322051/1/tests/7_apparmor-pipewire/apparmor-pipewire/pipewire ===
>> 1
Currently in use: 2 blocks containing 8.0 KiB bytes total.
Allocated during whole lifetime: 2 blocks containing 8.0 KiB bytes total.
Sample cache size: 0 B
>> 2
Failed to get sink information: No such entity
Job for generated-test-case-_var_lib_lava-4322051_1_tests_7_apparmor-pipewire_apparmor-pipewire_pipewire.service failed because the control process exited with error code.
See \"systemctl --user status generated-test-case-_var_lib_lava-4322051_1_tests_7_apparmor-pipewire_apparmor-pipewire_pipewire.service\" and \"journalctl --user -xe\" for details.
Command exited with non-zero status 1
real	0m 0.23s
user	0m 0.00s
sys	0m 0.00s
Job for generated-test-case-run-test-sh.service failed because the control process exited with error code.
NOTE: Run command with the --debug or --full-debug option to enable journal logs
#--- end of test script, status: 1
# /var/lib/lava-4322051/1/tests/7_apparmor-pipewire/apparmor-pipewire/common/run-test-in-systemd --user=1000 /var/lib/lava-4322051/1/tests/7_apparmor-pipewire/apparmor-pipewire/pipewire exited 1
```
This is due to missing audio card.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

`PulseAudio` has been replaced by `PipeWire` in v2022dev3

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Sagar Kishore Benani <SagarKishore.Benani@in.bosch.com>

Bad test definition prevents to run the test.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Instead of installing flatpak in system context switch to user context to
avoid requiring authentication and misuse of system partition which has
size constraints.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

On QEMU devices LAVA will mount the overlay data from a local ext2
image so there's no need to explicitly download it as we do on other
kind of devices.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit "lava: Work around LAVA 2021.05 regression with kernel_start_message"
4e9a8b1d still spelled `kernel_start_message` in the QEMU boot
template while the correct spelling is now `kernel-start-message`, as
used by other templates.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use job names that are identifiers rather than textual blurbs to pack
more information in a easier to consume way.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Update all references to v2021* releases to point to the v2021.0 one.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Martyn Welch authored 3 years ago and Martyn Welch committed 3 years ago

The legacy software framework is being dropped from v2022dev3, as a result
the tests for these services can also be dropped.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Emanuele Aina authored 3 years ago and Martyn Welch committed 3 years ago

With the update to LAVA 2021.05 jobs on the Minnowboard failed:

    Setting prompt string to ['Booting in blind mode', 'error: missing (.*) symbol.']
    ...
    Apertis v2020 apertis ttyS0
    auto-login-action timed out after 853 seconds

Apparently `kernel_start_message` is not working correctly in the
`context` section, so move it to the relevant `boot` sections.

It may also be related that the correct spelling is
`kernel-start-message` so I am not sure about anything anymore.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 3 years ago and Martyn Welch committed 3 years ago

With the update to LAVA 2021.05 a new deployment type has been added to
ensure that things work despite:

1. `/bin/sh` not pointing to Bash
2. a read-only `/` that prevents unpacking the overlay there

In particular the latter causes all the OSTree tests to fail when
downloading and unpacking the overlay with:

    ./lava-4147029/
    tar: ./lava-4147029: Cannot mkdir: Operation not permitted

See https://gitlab.collabora.com/lava/lava/-/commit/d621f93191591714449e67945ebd95cb6396727a


for the commit introducting the `apertis` type.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Sagar <SagarKihore.Benani@in.bosch.com>