Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Ryan Gonzalez authored 3 years ago and Emanuele Aina committed 3 years ago

This reorganizes and tweaks the Flatpak test cases to better express
their intent:

- Renames the `flatdeb-` prefix, which wasn't all that clear.
- Renames the legacy tests depending on mildenhall to match, and ensures
  they're tied to v2021.
- Rewords the contents of the tests to be more consistent and updated
  for the current state of things.
- Reworks the HMI demo test to use the app icon rather than launching
  from the CLI

https://phabricator.apertis.org/T8170



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

Build on commit c49ebbd7 "Add deployment type in the metadata" to
send out the same metadata for the SDK/BaseSDK jobs using QEMU and for
the NFS jobs.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Due to a typo we ended up putting the `image_type` (that is,
`fixedfunction`, `hmi`, etc.) in `image.deployment` rather than the
actual deployment type (`apt`, `ostree`, `nfs`, etc.)

Fix the job template so `image.deployment` is set correctly.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Frederic Danis authored 3 years ago and Ryan Gonzalez committed 3 years ago

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Frederic Danis authored 3 years ago and Ryan Gonzalez committed 3 years ago

As PipeWire uses the PulseAudio tools for its configuration, it's better
to only refer to the audio manager in tests.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Frederic Danis authored 3 years ago and Ryan Gonzalez committed 3 years ago

Test that audio policy to mute song application when switching to video
application.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Detlev Casanova authored 3 years ago and Emanuele Aina committed 3 years ago

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Emanuele Aina committed 3 years ago

The added jobs are for:
 * fixedfunction: ostree and apt
 * hmi: ostree

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Emanuele Aina committed 3 years ago

These are the same profiles as for arm64 but with the
device-arm64-bcm2711-rpi-4-b device.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Emanuele Aina committed 3 years ago

This device entry will be used to run tests on the devices with the
bcm2711-rpi-4-b device_type.

It uses the release v2021.0 instead of v2019.1 to deploy via NFS because
raspberry pi 4 was not supported in that version.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

This test checks basic import-debian-package functionalities.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Emanuele Aina authored 3 years ago and Frederic Danis committed 3 years ago

Run the apparmor-session-lockdown-no-deny only on APT images since it
requires the installation of a few packages.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The QA report app now relies on heuristics to determine the deployment
type, and it some cases it gets it wrong: for instance it looks for the
`ostree` string in the URL, so it had misclassified SDK images like
https://images.apertis.org/test/wip-em-ostree-only-pull-metadata/daily/v2021dev2/20200412.1719/amd64/sdk/apertis_v2021dev2-sdk-amd64-sdk_20200412.1719.img.gz



If we set the right metadata the heuristics will be ignored.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This reverts commit daf18238 "Move aum-offline test to HMI images"
to some extent by moving back to the `fixedfunction` images the offline
upgrade tests with the Apertis Update Manager now that `udisks2` is
installed there.

The move to the HMI images prevented testing offline updates on the
R-Car boards, which does not supports HMI images due to the lack of any
open graphical stack.

By using the `fixedfunction` images offline upgrades can be tested again
on R-Car boards, like it is done on the `minimal` images from older
release streams.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `v2022dev3` image doesn't have enough space to perform the branch
update. This has been fixed in `v2022pre`.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Multiline commands in preconditions, execution steps, postconditions and
notes are currently getting squashed to a single line.

Use `<pre>` block elements to preserve newlines.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 3 years ago and Ryan Gonzalez committed 3 years ago

Pick the right Docker image automatically rather than hardcoding the
Apertis v2021 image.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Pick the right osname automatically when deriving the image to test
upgrading from, rather than hardcoding `apertis`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Ritesh Raj Sarraf authored 3 years ago and Emanuele Aina committed 3 years ago

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

After switching to bsdtar update download command according to bsdtar,
which requires "-f -" to read from stdin.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Our Flatpak repo now implements ED25519 signatures and provides a
`.flatpakrepo` file for making it easier to configure the remote,
including signature verification. Switch this test case to using this
method.

Other Flatpak-related test cases use the `flathub` repo and therefore
should be left untouched for now.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>

Since commit 9fad8001, the lava timeout has been increase to 20 min.
Do the same for the systemd service tests.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

LAVA only has two minnowboards which cannot handle the current load. In
order to reduce the load avoid running OSTree offline test that were
recently added to amd64.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Frederic Danis authored 3 years ago and Walter Lozano committed 3 years ago

`aum-offline-upgrade-signed` test fails during phase 0 on timeout when
trying to create the different static-delta used during next phases.

Increasing the timeout allows to pass the test.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

When using test cases from git repositories by default LAVA includes their
history, however, in general this is not useful, requires more space on
overlays and extra time to copy them. Remove it by setting history to False.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

When using revision LAVA is forced to do a full clone. However, in case
using branch, LAVA is smart enough to do a shallow clone. Switch
to branch then to take advantage of this feature.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

It seems there's a side effect of geoclue preventing to pass
functional-demo test.
Furthermore geoclue is also failing.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

The test script repositories use `apertis/v20…` default branch name, and
the source archive should be downloaded using it.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Since some scenarios are only covered by HMI images, such using offline
mechanisms, move them to HMI.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

After switching to Fixed Function and HMI images the upgrades from a
previous branch only make sense from a branch that already supports the
new images. Switch then to v2022dev3 for upgradable_release.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

As Fixed Function images don't support offline upgrades move the tests
to HMI images. For automatic test use armhf and amd64 as in other LAVA
OSTree tests, while for manual ones use armhf and arm64.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This test should also be run on BaseSDK

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Walter Lozano authored 3 years ago and Emanuele Aina committed 3 years ago

Test case of iptables-nmap fails on SDK with error

Unable to apt-get update in lxc container
Certificate verification failed: The certificate is NOT trusted. The
certificate issuer is unknown.  Could not handshake: Error in the
certificate verification. [IP: 199.232.142.132 443]

Since the problem is only present on QEMU without a clear cause, after
checking that clock time is OK workaround it by using http instead of
https.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The expected logo is pointing to an old wifi url, which is missing.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Frederic Danis authored 3 years ago and Arnaud Ferraris committed 3 years ago

`target` has been replaced by `hmi` images.
And some AUM tests are only run on `fixedfunction` images.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Flatpak apps are now installed by default on `hmi` images. Moreover, we 
now ship Totem instead of the "VIDEO & TV application", so make sure the 
test case reflects the new state.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>