Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Many comments in the test cases mention the Minnowboard as the target
reference board. This is now retired and replaced by the UP Squared 6000
board, so replace any occurrence.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The lava test templates have some comments on the timeouts referring to
the Minnowboard, now retired. Let's remove this old comments, but keep
the used timeouts, as that's the current setup working on LAVA.

A follow up task should revisit these timeouts and set them more tightly
if needed.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Ariel D'Alessandro authored 2 years ago and Detlev Casanova committed 2 years ago

The goal of this test is to ensure we can execute the getting-started
instructions in the apertis-image-recipes README from inside the Apertis
SDK running on Virtualbox.

We don't actually have Virtualbox in LAVA, so it must be simulated.
As Virtualbox doesn't support nested KVM virtualization, let's remove
the KVM module. This should trigger the chroot-based Debos backend
instead of the fakemachine one"

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The Apertis v2022 image used to flash the testing images is reporting an
issue related to the wifi driver:

See https://lava.collabora.dev/scheduler/job/7001988#L1209
```
[   27.837322] iwlwifi 0000:02:00.0: minimum version required: iwlwifi-ty-a0-gf-a0-39
[   27.837324] iwlwifi 0000:02:00.0: maximum version supported: iwlwifi-ty-a0-gf-a0-66
[   27.837325] iwlwifi 0000:02:00.0: check git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git


[   27.847366] stack segment: 0000 [#1] SMP NOPTI
[   27.952703] CPU: 3 PID: 85 Comm: kworker/3:1 Tainted: G            E     5.15.0-trunk-amd64 #1  Debian 5.15.22-0~apertis1
[   27.964951] Hardware name: AAEON UPN-EHL01/UPN-EHL01, BIOS UNEHAM19 04/07/2022
[   27.973021] Workqueue: events request_firmware_work_func
[   27.978961] RIP: 0010:kfree+0x64/0x250
[   27.983150] Code: 80 48 01 dd 0f 82 f5 01 00 00 48 c7 c0 00 00 00 80 48 2b 05 fe 68 f0 00 48 01 c5 48 c1 ed 0c 48 c1 e5 06 48 03 2d dc 68 f0 00 <48> 8b 45 08 48 8d 50 ff a8 01 48 0f 45 ea 48 8b 55 08 48 8d 42 ff
[   28.004139] RSP: 0018:ffffb8e48023fcc0 EFLAGS: 00010207
[   28.009974] RAX: 00006f5940000000 RBX: 36415f5f0004000f RCX: 00000000000006bc
[   28.017946] RDX: 0000000000000000 RSI: ffffffffc0e16e86 RDI: 36415f5f0004000f
[   28.025916] RBP: 00d8e24923001000 R08: ffffffff9fa41940 R09: ffffb8e48023fbd8
[   28.033888] R10: ffff90a742874000 R11: 0000000000000000 R12: ffff90a7425572b8
[   28.041861] R13: ffff90a8a41b5900 R14: 0000000000000000 R15: ffff90a742557000
[   28.049834] FS:  0000000000000000(0000) GS:ffff90a8a4180000(0000) knlGS:0000000000000000
[   28.058874] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   28.065294] CR2: 0000565365b792b0 CR3: 00000001075a6000 CR4: 0000000000350ee0
[   28.073266] Call Trace:
[   28.075997]  <TASK>
[   28.078341]  iwl_dealloc_ucode+0x36/0x110 [iwlwifi]
[   28.083809]  iwl_req_fw_callback+0x2cf/0x23d0 [iwlwifi]
[   28.089650]  ? free_fw_priv+0x9f/0x110
[   28.093835]  ? _request_firmware+0x42f/0x640
[   28.098604]  ? kfree+0x1f8/0x250
[   28.102206]  ? _request_firmware+0x42f/0x640
[   28.106974]  request_firmware_work_func+0x4a/0x90
[   28.112229]  process_one_work+0x1ee/0x390
[   28.116707]  worker_thread+0x53/0x3e0
[   28.120793]  ? process_one_work+0x390/0x390
[   28.125454]  kthread+0x124/0x150
[   28.129059]  ? set_kthread_struct+0x40/0x40
[   28.133731]  ret_from_fork+0x1f/0x30
[   28.137727]  </TASK>
[   28.140163] Modules linked in: sha512_generic(E) snd_intel_sdw_acpi(E) snd_hda_codec(E) ctr(E) snd_hda_core(E) drbg(E+) aesni_intel(E+) snd_hwdep(E) asus_nb_wmi(E) iwlwifi(E) asus_wmi(E) crypto_simd(E) snd_pcm(E) ansi_cprng(E) cryptd(E) iTCO_wdt(E) platform_profile(E) mmc_block(E) intel_cstate(E) mei_hdcp(E) evdev(E) snd_timer(E) intel_pmc_bxt(E) battery(E) cfg80211(E) snd(E) ecdh_generic(E+) sparse_keymap(E) iTCO_vendor_support(E) intel_wmi_thunderbolt(E) wmi_bmof(E) soundcore(E) efi_pstore(E) pcspkr(E) rfkill(E) watchdog(E) ecc(E) ti_adc128s052(E) crc16(E) industrialio(E) i915(E) ttm(E) drm_kms_helper(E) sdhci_pci(E) cec(E) cqhci(E) ahci(E) rc_core(E) libahci(E) sdhci(E) drm(E) libata(E) mei_me(E) mmc_core(E) scsi_mod(E) mei(E) scsi_common(E) intel_pmc_core(E) acpi_tad(E) acpi_pad(E) button(E) fuse(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) nfsv3(E) nfs_acl(E) nfs(E) lockd(E) grace(E) sunrpc(E) fscache(E) netfs(E) spi_pxa2xx_platform(E) dw_dmac(E) dw_dmac_core(E)
[   28.140232]  crc32_pclmul(E) crc32c_intel(E) igb(E) i2c_i801(E) i2c_smbus(E) i2c_algo_bit(E) dca(E) igc(E) xhci_pci(E) ptp(E) xhci_hcd(E) pps_core(E) usbcore(E) intel_lpss_pci(E) intel_lpss(E) usb_common(E) idma64(E) fan(E) wmi(E) video(E) pinctrl_elkhartlake(E)
[   28.263736] ---[ end trace 2553bae38d1333f9 ]---
```

To overcome this, use the following image as it includes a newer kernel
v5.15.34, where the iwlwifi issue is fixed: weekly/v2022/20220803.0154

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Use a more recent image, as there is no support for the UP Squared 6000
board in v2019.1.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The recently added UP Squared 6000 device uses the eMMC for deployment,
which is different from the default setting (sd-card). Let's add a
parameter so each LAVA device can set the deploy device in each case.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The Apertis amd64 reference platform -MinnowBoard Turbot board- went out
of production a while ago and are no longer available. The UP Squared
6000 board has been chosen as the new reference platform, so let's
replace it.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Now that QA Report App renders the test cases in its interface the deploy
using apertis-test-cases-web is not longer needed.

Drop the deploy of this repo and also the deploy in forks to avoid confusions.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Starting org.apertis.Frome.MockWebStoreServer fails with the error:

  ModuleNotFoundError: No module named 'gi'

Making sure the package is installed.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Detlev Casanova authored 3 years ago and Walter Lozano committed 2 years ago

This test was created to verify some DBUS robustness patches that ended
up not being accepted upstream.

The downstream patches have been dropped since v2019 when we rebased to
Debian, so this test doesn't have value anymore.

Avoid random failures by removing it.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

The frome test currently fails with the message:
  UserWarning: Parsing of Deb822 data with python3-apt's apt_pkg was
    requested but this package is not importable. Is python3-apt installed?

python3-debian only recommends to to install python3-apt but does not
depend on it.

Forcing installation.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

LAVA version 2021.11 introduces a change in the way paths in compressed
overlays are handled breaking Apertis tests.

Fix the issue by specifying the path to be use to extract the compressed
overlay.

Additional information:

https://gitlab.collabora.com/lava/lava/-/commit/07048f042c65202eb75ac874cc93eabd2d96cd54



Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Latest version of the test is able to run on boards or SDK, APT or OSTree.

Replace macro_ostree_preconditions by install entry to run on both APT and
OSTree images.
Command should not start by `$`

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Walter Lozano authored 3 years ago and Frederic Danis committed 3 years ago

Since apparmor-geoclue runs a test suite with several tests fix the
parsing to allow LAVA to catch all the results.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Command should not start by `$`

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Sudarshan CP authored 3 years ago and Walter Lozano committed 3 years ago

Signed-off-by: Monisha Muthukumaran <Muthukumaran.Monisha@in.bosch.com>

When using test cases from git repositories by default LAVA includes their
history, however, in general this is not useful, requires more space on
overlays and extra time to copy them. Remove it by setting history to False.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

When using revision LAVA is forced to do a full clone. However, in case
using branch, LAVA is smart enough to do a shallow clone. Switch
to branch then to take advantage of this featur

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

It seems there's a side effect of geoclue preventing to pass
functional-demo test.
Furthermore geoclue is failing since 08/24/2020, see
https://phabricator.apertis.org/T7333

.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Test case of iptables-nmap fails on SDK with error

Unable to apt-get update in lxc container
Certificate verification failed: The certificate is NOT trusted. The
certificate issuer is unknown.  Could not handshake: Error in the
certificate verification. [IP: 199.232.142.132 443]

Since the problem is only present on QEMU without a clear cause, after
checking that clock time is OK workaround it by using http instead of
https.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

As defined in the test description, this test should not be run on
minimal images.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Followup on commit 4e9a8b1d to apply the fix on the snippet
rebooting devices during the Apertis Update Manager tests, which
are currently failing since they can't detect the prompt after the
kernel has booted.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit "lava: Work around LAVA 2021.05 regression with kernel_start_message"
1bb8ba90 still spelled `kernel_start_message` in the QEMU boot
template while the correct spelling is now `kernel-start-message`, as
used by other templates.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

With the update to LAVA 2021.05 jobs on the Minnowboard failed:

    Setting prompt string to ['Booting in blind mode', 'error: missing (.*) symbol.']
    ...
    Apertis v2020 apertis ttyS0
    auto-login-action timed out after 853 seconds

Apparently `kernel_start_message` is not working correctly in the
`context` section, so move it to the relevant `boot` sections.

It may also be related that the correct spelling is
`kernel-start-message` so I am not sure about anything anymore.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

With the update to LAVA 2021.05 a new deployment type has been added to
ensure that things work despite:

1. `/bin/sh` not pointing to Bash
2. a read-only `/` that prevents unpacking the overlay there

In particular the latter causes all the OSTree tests to fail when
downloading and unpacking the overlay with:

    ./lava-4147029/
    tar: ./lava-4147029: Cannot mkdir: Operation not permitted

See https://gitlab.collabora.com/lava/lava/-/commit/d621f93191591714449e67945ebd95cb6396727a


for the commit introducting the `apertis` type.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Added the `dry-run` mode feature testing.
Added the network polling time change feature testing.
Updated output for manual OTA rollback upgrade test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The common tests group currently blows up on LAVA when submitted to
QEMU-based DUTs as we do for the SDK and base SDK images because the
dispatcher runs out of disk space while unpacking the overlay before
uploading it to the DUT:

    Job error: tar_in: tar subcommand failed on directory: /:
    tar: 0/tests/1_disk-rootfs-fsck/apertis-tests/.git/objects/pack/pack-c1d7de14a133c29fdcacfeef5eba98291ada6c91.pack:
    Wrote only 2048 of 10240 bytes tar: 0/tests/1_disk-rootfs-fsck/apertis-tests/.git/packed-refs:
    Cannot write: No space left on device

In particular this is because the git repositoriee installed by the
apparmor-gstreamer1.0 and gstreamer1.0-decode testcases used to ship a
large copy of the Big Buck Bunny movie. Even if the test file has been
dropped from the HEAD of both repositories in favor of the media sample
file for the same movie pre-installed on the images, LAVA fails because
it does not do a shallow checkout but it ends up putting the whole git
hisotry in the overlay.

To side-step the issue, move all the apparmor tests still in the
`common-standalone` group to the `apparmor-standalone` group, but keep
skipping them on minimal images.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Denis Pynkin authored 4 years ago and Emanuele Aina committed 4 years ago

Added templates list for ostree-based profiles without AUM tests.
At the moment we do not support AUM tests for Amd64 and for internal
Arm32 images, hence use a base list for them.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Submitting tests to LAVA from this repository has been broken for a long
time and has proven to be not useful. Yet, it generates plenty of noise
on the LAVA side, so disable the actual submission.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

The new `psdk-test` requests `paramiko` to run.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

The new `psdk-test` perform the 3 previous tests (etc, home-user and
sysroot) without user interaction.
It can use a pre-downloaded virtual disk image, or directly the URL.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Walter Lozano authored 4 years ago and Emanuele Aina committed 4 years ago

Add a new test case to confirm that a licensing summary is generated for minimal
and target images.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Add missing screenshot which shows the popup functionality of agl-compositor.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This project was never actually uploaded as a package, as it would not
be particularly useful to do so.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>