We are no longer using cgroups in any special way in Apertis. In
addition it has been discovered that these tests have a flaw that allows
them to silently fail. Remove the tests as they are both broken and not
required.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

ABHINISH KUMAR authored 1 year ago and Walter Lozano committed 11 months ago

(cherry picked from commit 825ba5a4)

There are several tests that installation additional packages to run tests, even
if this is a bad practice, and additionally requires extra space in the root
partition.

A proper fix is to rewrite the test in its own repo to only hold the important
information and binaries to run the test, but this approach requires much time.
In the meantime, at least remove unneeded dependencies to save space which is
quite limited. Also, the lack of this space in some cases can make tests to
fail, as it is not possible to install the dependencies.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Since the functionality provided by ade has little value after the switch
to Flatpak, change the test priority to medium.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Since the current test case uses several customizations to target device,
add additional clarifications and checks to ensure the system is properly
setup.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Since these tests also are applicable to AMD 64 let's run them.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This set of tests ensures that Apertis has the support for different type
of WiFi configuration. Since our OSTree image are not shipped with hostapd
enabled a set of configurations are required to properly setup the AP.

While this is possible to in OSTree images, it is easier to do it on APT
images. Moreover, the test seems to be designed to run on APT images as
it tries to create files on a read-only file system.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

After systemd has started to support pstore the test needs to be adapted to use this new functionality.
Upon startup, systemd-pstore will monitor and extract the pstore contents and save it inside /var/lib/systemd/pstore/.

This also fixes the failure in the current test as /sys/fs/pstore: is not longer available.

Signed-off-by: Nithin M N <MN.Nithin@in.bosch.com>

Since package geoclue has been moved to developement it has no sense to
run it on fixed function or HMI images. The test itself passes, however,
the development repositories need to be enabled and many packages need
to be installed, which modifies the image completely.

Additionally, on HMI image, due to the number of packages that need to
be installed the LAVA job fails with

Test error: ['Unable to start test stage 1', 'Unable to start test stage 1']

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Siva Krishna Prasad Chinthalapudi authored 1 year ago and Siva Krishna Prasad Chinthalapudi committed 1 year ago

remove deprecated pkg `Xephyr` and use agl-compositor

Signed-off-by: Chinthalapudi Siva Krishna Prasad <sivakrishnaprasad.chinthalapudi@in.bosch.com>


(cherry picked from commit d27919af)

Nithin Mysore Nagesh authored 1 year ago and Nithin Mysore Nagesh committed 1 year ago

Modify the steps to run the test on SDK and enable nested virtualization
as part of pre-condition.

Signed-off-by: MN Nithin <MN.Nithin@in.bosch.com>


(cherry picked from commit 057e3b51)

Nithin Mysore Nagesh authored 1 year ago and Nithin Mysore Nagesh committed 1 year ago

- Add group entry: Group entry was missing due to which jobs were not
  generated for rhosydd.

- Modify image-types: Since this test executes unit test cases and
  uses devscripts which is available in development repository, this
should be run only on sdk and basesdk. Separate test to be implemented
for target.

- Add dependencies

Signed-off-by: MN Nithin <MN.Nithin@in.bosch.com>

After relaxing hmi-switch-graphical-applications to use any application
some instructions were not properly updated. Update the test case to
be coherent with the new approach.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The test hmi-switch-graphical-applications requests to open Fonts application
which is installed by default in HMI images via Flatpak, but not installed
on SDK. Since the test can be run with any graphical application, relax
it and mention Fonts only as a suggestion.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Switch to the new R-Car r8a779m1-ulcb device type as it is the new
reference board for this release.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This test is expected to be run on a system with its rootfs on an eMMC storage
and not on a USB device. Using a rootfs from a USB device can lead to
failures like the systemd service 'usbmount@dev-sda1.service' failing to
start since /dev/sda1 is already mounted at /boot/efi.

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>

This reverts commit eb8ce592.

The MR was merge by accident while being reviewed. The picture itself is
not incorrect, however, it does not follow the format of others pictures
related to agl-compositor.

Reverting this, while preparing a better MR.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The sanity manual test is the first test run to confirm the health of the
image. For this reason, the test itself states that no other manual test
should run if there is a failure.

This policy aims to avoid running tests on an image which is not healthy
enough since the results will not be valid. This policy is too strict,
since due to failure in one are, like WiFi, will block the rest of the
tests.

To provide a better trade off, update the policy to avoid running other
manual tests until the issue is triaged.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Improve the wording of the pre conditions section in the secure boot test
to instruct the flashing to the correct u-boot image based on the release
under test.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This test relies on a Debian LXC image to be deployed in the DUT which
presents a problem in some scenarios, devices requires access to public
Internet and QEMU requires LXC support.

Disable this test until it is reimplemented to overcome the current
limitations.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: SivaKrishnaPrasad <sivakrishnaprasad.chinthalapudi@in.bosch.com>

Signed-off-by: Andre Moreira Magalhaes <andre.magalhaes@collabora.com>

In commit 5bf3c4a9 tests for amd64 board were re enabled after adding
support for Up Squared 6000 board, and having enough boards in LAVA.

After this, it was seen that tests that relies in bootcount fail and
require a rework to align u-boot and uefi.

In the mean time, to avoid creating noise and masking issues disable
tests that rely in bootcount.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

The use of the git repos pointing at v2023dev1 was introduced in 6ae11598
which was merged later on v2023dev2, causing the branching script not
being able to update the branch name.

Manually fix the issue.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Signed-off-by: Apertis CI <devel@lists.apertis.org>

The previous code would only ever check if the architecture was directly
listed in the test case, which meant it excluded the visibility
suffixes.

https://phabricator.apertis.org/T8949

This was getting rendered as a messy plain-text paragraph.

https://phabricator.apertis.org/T8949

Because the test logs are private, the visibility is '-internal', so
update the test cases to match.

https://phabricator.apertis.org/T8949



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

The group template was created after
9dc1fafa was merged, which removed all
instances of the 'name:' key. However, this turned out to break test
submission, which is why it was then reverted in
79a2b0df.

Unfortunately, I did not realize that the new IoT group was missing
this field, which means that 'name:' was never added back there and IoT
image test submission is broken.

https://phabricator.apertis.org/T8949



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

The tests need credentials to be passed to them, so a new LAVA group is
created that passes them down to the scripts. In order to avoid any
leakage, visibility controls are added to the job generation, so that
the IoT jobs can set the visibility to be internal-only.

https://phabricator.apertis.org/T8949



Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>

qa.apertis.org is being served by the qa-report-app.
So modify the LAVA callback to point to qa.apertis.org.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

This reverts commit 0f2e8a1a.

On commit 0f2e8a1a a clean up was introduced to remove validation warnings.
Unfortunately the tool prompts is key is used by lava to detect when the
deploy action has finished when the tool is different from dd as described
in:

https://github.com/Linaro/lava/blob/c38d449719d7e4dca3d07e6b5e9bb63d5a5579a9/lava_dispatcher/actions/deploy/removable.py#L151

Revert the previous changes to allow LAVA to detect that the deploy action
has finished.

This reverts commit 9dc1fafa.

Revert this changes since QA Report App tries to use this metadata and
currently rises the following exception:

    File "/app/testobjects.py", line 143, in lava_jobdata_extract_results
      t['group'] = testgroup['name']
    KeyError: 'name'

This causes tests results for v2023pre not being processed.

It is not clear if this metadata is used, however, currently it is not
possible to easily deploy a new version of QA Report app, so the best
approach is to revert this change.

Edmund Smith authored 2 years ago and Andre Moreira Magalhaes committed 2 years ago

The way test submission works changed significantly when we switched
to using the lava_runner. In that change set, I preserved the logic
around when the submit tasks ran, such that they only ran when a valid
LAVA configuration existed in the job variables, and when the pipeline
is not for a merge request.

In repository preceding those changes, there was a valid LAVA
configuration, but the test submission was given `--dry-run` so that
we would run our templating, but stop short of submitting the jobs,
even though we required a valid LAVA configuration to exist.

In the repository after these changes, there is not a valid LAVA
configuration (since the required variables changed) and now the test
generation step is no longer occurring on pipelines. Moreover, if we
made the LAVA configuration valid for the state we have now, it would
both generate and run the tests, because the generation and run steps
have identical rules governing them in CI.

Therefore, remove the checks for a valid LAVA configuration from the
generation step. This means we do not need a valid LAVA configuration
in order to get the same behaviour we had before: generate the tests,
but do not submit them.