Since package geoclue has been moved to developement it has no sense to
run it on fixed function or HMI images. The test itself passes, however,
the development repositories need to be enabled and many packages need
to be installed, which modifies the image completely.

Additionally, on HMI image, due to the number of packages that need to
be installed the LAVA job fails with

Test error: ['Unable to start test stage 1', 'Unable to start test stage 1']

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Siva Krishna Prasad Chinthalapudi authored 1 year ago and Siva Krishna Prasad Chinthalapudi committed 1 year ago

remove deprecated pkg `Xephyr` and use agl-compositor

Signed-off-by: Chinthalapudi Siva Krishna Prasad <sivakrishnaprasad.chinthalapudi@in.bosch.com>


(cherry picked from commit d27919af)

Nithin Mysore Nagesh authored 1 year ago and Nithin Mysore Nagesh committed 1 year ago

Modify the steps to run the test on SDK and enable nested virtualization
as part of pre-condition.

Signed-off-by: MN Nithin <MN.Nithin@in.bosch.com>


(cherry picked from commit 057e3b51)

Nithin Mysore Nagesh authored 1 year ago and Nithin Mysore Nagesh committed 1 year ago

- Add group entry: Group entry was missing due to which jobs were not
  generated for rhosydd.

- Modify image-types: Since this test executes unit test cases and
  uses devscripts which is available in development repository, this
should be run only on sdk and basesdk. Separate test to be implemented
for target.

- Add dependencies

Signed-off-by: MN Nithin <MN.Nithin@in.bosch.com>

After relaxing hmi-switch-graphical-applications to use any application
some instructions were not properly updated. Update the test case to
be coherent with the new approach.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The test hmi-switch-graphical-applications requests to open Fonts application
which is installed by default in HMI images via Flatpak, but not installed
on SDK. Since the test can be run with any graphical application, relax
it and mention Fonts only as a suggestion.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Switch to the new R-Car r8a779m1-ulcb device type as it is the new
reference board for this release.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This test is expected to be run on a system with its rootfs on an eMMC storage
and not on a USB device. Using a rootfs from a USB device can lead to
failures like the systemd service 'usbmount@dev-sda1.service' failing to
start since /dev/sda1 is already mounted at /boot/efi.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This reverts commit eb8ce592.

The MR was merge by accident while being reviewed. The picture itself is
not incorrect, however, it does not follow the format of others pictures
related to agl-compositor.

Reverting this, while preparing a better MR.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The sanity manual test is the first test run to confirm the health of the
image. For this reason, the test itself states that no other manual test
should run if there is a failure.

This policy aims to avoid running tests on an image which is not healthy
enough since the results will not be valid. This policy is too strict,
since due to failure in one are, like WiFi, will block the rest of the
tests.

To provide a better trade off, update the policy to avoid running other
manual tests until the issue is triaged.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Improve the wording of the pre conditions section in the secure boot test
to instruct the flashing to the correct u-boot image based on the release
under test.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This test relies on a Debian LXC image to be deployed in the DUT which
presents a problem in some scenarios, devices requires access to public
Internet and QEMU requires LXC support.

Disable this test until it is reimplemented to overcome the current
limitations.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: SivaKrishnaPrasad <sivakrishnaprasad.chinthalapudi@in.bosch.com>

Signed-off-by: Andre Moreira Magalhaes <andre.magalhaes@collabora.com>

In commit 5bf3c4a9 tests for amd64 board were re enabled after adding
support for Up Squared 6000 board, and having enough boards in LAVA.

After this, it was seen that tests that relies in bootcount fail and
require a rework to align u-boot and uefi.

In the mean time, to avoid creating noise and masking issues disable
tests that rely in bootcount.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The use of the git repos pointing at v2023dev1 was introduced in 6ae11598
which was merged later on v2023dev2, causing the branching script not
being able to update the branch name.

Manually fix the issue.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Apertis CI <devel@lists.apertis.org>

The previous code would only ever check if the architecture was directly
listed in the test case, which meant it excluded the visibility
suffixes.

https://phabricator.apertis.org/T8949

This was getting rendered as a messy plain-text paragraph.

https://phabricator.apertis.org/T8949

Because the test logs are private, the visibility is '-internal', so
update the test cases to match.

https://phabricator.apertis.org/T8949



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

The group template was created after
9dc1fafa was merged, which removed all
instances of the 'name:' key. However, this turned out to break test
submission, which is why it was then reverted in
79a2b0df.

Unfortunately, I did not realize that the new IoT group was missing
this field, which means that 'name:' was never added back there and IoT
image test submission is broken.

https://phabricator.apertis.org/T8949



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

The tests need credentials to be passed to them, so a new LAVA group is
created that passes them down to the scripts. In order to avoid any
leakage, visibility controls are added to the job generation, so that
the IoT jobs can set the visibility to be internal-only.

https://phabricator.apertis.org/T8949



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

qa.apertis.org is being served by the qa-report-app.
So modify the LAVA callback to point to qa.apertis.org.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This reverts commit 0f2e8a1a.

On commit 0f2e8a1a a clean up was introduced to remove validation warnings.
Unfortunately the tool prompts is key is used by lava to detect when the
deploy action has finished when the tool is different from dd as described
in:

https://github.com/Linaro/lava/blob/c38d449719d7e4dca3d07e6b5e9bb63d5a5579a9/lava_dispatcher/actions/deploy/removable.py#L151

Revert the previous changes to allow LAVA to detect that the deploy action
has finished.

This reverts commit 9dc1fafa.

Revert this changes since QA Report App tries to use this metadata and
currently rises the following exception:

    File "/app/testobjects.py", line 143, in lava_jobdata_extract_results
      t['group'] = testgroup['name']
    KeyError: 'name'

This causes tests results for v2023pre not being processed.

It is not clear if this metadata is used, however, currently it is not
possible to easily deploy a new version of QA Report app, so the best
approach is to revert this change.

Edmund Smith authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

The way test submission works changed significantly when we switched
to using the lava_runner. In that change set, I preserved the logic
around when the submit tasks ran, such that they only ran when a valid
LAVA configuration existed in the job variables, and when the pipeline
is not for a merge request.

In repository preceding those changes, there was a valid LAVA
configuration, but the test submission was given `--dry-run` so that
we would run our templating, but stop short of submitting the jobs,
even though we required a valid LAVA configuration to exist.

In the repository after these changes, there is not a valid LAVA
configuration (since the required variables changed) and now the test
generation step is no longer occurring on pipelines. Moreover, if we
made the LAVA configuration valid for the state we have now, it would
both generate and run the tests, because the generation and run steps
have identical rules governing them in CI.

Therefore, remove the checks for a valid LAVA configuration from the
generation step. This means we do not need a valid LAVA configuration
in order to get the same behaviour we had before: generate the tests,
but do not submit them.

Validating the job definitions when resubmitting them yields:

    Valid definition with warnings:
      extra keys not allowed
      @ data['actions[4]']['test']['definition']['name']

Get rid of those extra keys to make LAVA happier.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Validating the job definitions when resubmitting them yields:

    Valid definition with warnings:
      extra keys not allowed
      @ data['actions[2]']['deploy']['usb']['tool']

Get rid of the extra setting to make LAVA happier.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit cde077ec ("lava: amd64-upsquared6000: User a newer v2022 first
stage image") set the UP Squared 6000 1st stage image to a weekly build.
As the weekly image may get deleted sooner than later, let's use a
release point.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

When running the aum-rollback-blacklist test, u-boot must rollback to
the older version when the bootlimit of 3 has been reached.

Using just `boot` will bypass the boot count check, so we need to add
the check here in the device template, like it is done for the other
device types.

Fixes: infrastructure/apertis-issues#129
Fixes: infrastructure/apertis-issues#130

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>

Most of test cases sort supported architectures by relevance instead of
alphabetically, so apply the same principle in general to keep consistency.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

AUM tests for amd64 were never fully enabled due to the lack of hardware
in LAVA. After adding support to the new UP Squared 6000 boards this has
changed, so re enable test on amd64 to make testing consistent across all
the supported boards.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Apertis CI <devel@lists.apertis.org>

Commit 160e8737 ("lava: amd64-upsquared6000: Use v2022 1st stage image
with firmware pkgs") set the UP Squared 6000 1st stage image to a daily
build as it included the required firmware pkgs. As the daily image will
get deleted soon, let's use a weekly one until the next v2022 release it
out.

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>