After updating of Renesas boards in LAVA Lab with U-Boot
version 2020.04+dfsg-2co5 we are able to test all ostree-based
capabilities on Arm64 platform.

Enable all manual AUM/OSTree tests for Renesas boards.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

After updating of Renesas boards in LAVA Lab with U-Boot
version 2020.04+dfsg-2co5 we are able to test all ostree-based
capabilities on Arm64 platform.
Moved all ostree tests into `templates-minimal-ostree` group since
only amd64 have no full ostree support.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

AUM has now more specific error messages when trying to use an incorrect
static update bundle.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Walter Lozano authored 4 years ago and Emanuele Aina committed 4 years ago

As agl-compositor in declared as a dependency in maynard remove it
from the installation command.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Walter Lozano authored 4 years ago and Emanuele Aina committed 4 years ago

As now all the dependecies were moved to target repository there is
no need to enable the development respository, so remove the this
step from the test case.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Samir Kumar Sarangi authored 4 years ago and Emanuele Aina committed 4 years ago

Some testcases not longer work due to bitrot. They were initially set up
to test specific enhancements applied to the WebKit-Clutter codebase but
now that we ship a standard WebKitGTK+ they do not att much value.

Since the webkit2gtk-gstreamer1.0, webkit2gtk-mt-touch-events and
webkit2gtk-ac-3d-rendering already cover most of the web stack already
and also exercise the important integration points we decided to drop
the bitrotted testcases:

* webkit2gtk-ac-animations
* webkit2gtk-ac-scrolling
* webkit2gtk-drag-and-drop
* webkit2gtk-event-handling-redesign

Apertis: T7104

Signed-off-by: Samir <external.SamirKumar.Sarangi@in.bosch.com>

Samir Kumar Sarangi authored 4 years ago and Emanuele Aina committed 4 years ago

The manual testcases below have an automated counterpart, so let's drop
them as they are redundant:

* apertis-update-manager-api
* apertis-update-manager-diskfull
* apertis-update-manager-ota-auto
* apertis-update-manager-ota-diskfull
* apertis-update-manager

Apertis: T7104

Signed-off-by: Samir <external.SamirKumar.Sarangi@in.bosch.com>

Our guidelines say that to contribute people should fork repositories,
but at the moment this gives them a broken pipeline.

Tweak things in a way that the main project triggers
tests/apertis-test-cases-web to collect and indirectly publish all the
release branches while forked repositories directly publish their own
branches via Pages for easier testing.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Minnowboard currently get stuck for a few minutes during boot with
this message:

    Clear memory in MRC per MOR request Start, Please wait for some minutes...

This indicates that the firmware is busy overwriting (slowly) the RAM
contents. This is triggered by the `CONFIG_RESET_ATTACK_MITIGATION`
kernel option, enabled by default in Debian kernels, setting the
`MemoryOverwriteRequestControl`/`MemoryOverwriteRequestControlLock`
UEFI variables.

A request to drop the kernel option has been submitted upstream:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970861

Further details about the UEFI variables and their behaviour in the
EDK2 TianoCore firmware here:

https://edk2.groups.io/g/discuss/topic/77180459#381



The memory cleaning process seems to be largely outside of the control
of the free firmware, so the suggested option is to avoid triggering
it from the OS kernel. The best way to do so is still unclear and
should be discussed in the above Debian bug. In the meantime, increase
the timeouts.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Include a basic test for agl-compositor which downloads the required
packages and launches the compositor.

Signed-off-by: Walter Lozano <walter.lozano@collabora.com>

Emanuele Aina authored 4 years ago and Martyn Welch committed 4 years ago

The test uses `/usr/lib/chaiwala-tests/bluez/simple-agent` shipped in
the `chaiwala-tests` package, let's explicitly depend on it.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 4 years ago and Martyn Welch committed 4 years ago

After commit 45dae71b fixed the boot on Renesas boards with
linux >= 5.7 due to the change of the dtb name, the tests exercising
the NFS boot of the current release as the first stage broke since
they do not use `boot_commands` and thus didn't get the override.

To fix that, rather than constructing the path to the DTB from the
LAVA device name, which no longer match the board type with
linux >= 5.7, explicitly pass the DTB path for the first stage and the
release under test separately.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use the passed URL to Tiny RFS rather than re-construct the URL
with script hardcoded function. This allows to test the RFS prepared
in users GIT branches.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Benani Sagar Kishore (RBEI/ECO2) <SagarKishore.Benani@in.bosch.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Lightweight CI runners have been introduced to run jobs which have low
resource consumption and do not need privileged containers to reduce costs
and latency, so let's use them.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Add some additional YAML anchors so we further shrink the
*-internal-mx6qsabrelite profile definitions.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Martyn Welch authored 4 years ago and Emanuele Aina committed 4 years ago

Two of the test cases point to videos to show what a successful run looks
like. These were randomly stored on the old wiki machine. Rather than
having unlinked static content on the website, add these to the
`images` folder and update links.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Commit 822fb8b8 dropped a few testcases, but the profiles still
point to them, resulting in errors like on
https://lava.collabora.co.uk/scheduler/job/2572900

:

    Job error: Unable to open test definition 'test-cases/tracker-indexing-local-storage.yaml':
    [Errno 2] No such file or directory:
    '/var/lib/lava/dispatcher/tmp/2572900/lava-overlay-knf8ynqa/var/lib/lava-2572900/2/tests/21_tracker-indexing-local-storage/test-cases/tracker-indexing-local-storage.yaml'

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

After the discussions following test cases will be dropped.
apparmor-tracker
folks-ofono-backend
telepathy-ring
tracker-indexing-local-storage
tracker-indexing-mass-storage
traffic-control-basic

Apertis: T5734
Signed-off-by: Tarun Baghmar <tarun.baghmar@in.bosch.com>

Denis Pynkin authored 4 years ago and Emanuele Aina committed 4 years ago

Ask to wait a few seconds before to check if undeploy is really
finished. Sometimes the human is too fast to enter commands
so undeploy have no chance to finish before the check.
That is leading to false result of manual test while it works well.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Frederic Danis authored 4 years ago and Denis Pynkin committed 4 years ago

wget needs the complete URL to be able to download key0.key file.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Earlier, we were using an old kerne and rootfs from 18.09.xx.

We've now switched to using a much newer one from v2020, which is where
it requires an update of our assumpt login prompts

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

In commit 83772e1 and later commits DTB names in Linux kernel was
renamed to align names with config options.
Hence the old DTB name `r8a7796-m3ulcb.dtb` is `8a77960-ulcb.dtb` for
Renesas M3 boards used in LAVA.

Old U-Boot versions are using the old naming schema including version
used in v2021dev3 atm.

This commit set proper name only for v2021dev3 branch allowing to boot
images for Renesas boards in LAVA.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Martyn Welch authored 4 years ago and Emanuele Aina committed 4 years ago

Rather than looping over branches (in included template from `pages`),
generate the test cases here, provide these as an artifact and deploy
in a separate job. This avoids the regeneration of pages that have
not changed.

Use the apertis-test-cases-web as part of a multi-project pipeline to
deploy. Remove the deployment tools from this branch.

Whilst we are here, utilise an environment variable for the release to
make it easier to migrate this between releases.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

The bundle generation with an unknown key doesn't display correctly.
Move bundle generation to `pre-conditions`.
Add `cryptsetup` package installation

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

The GitLab CI logic has been moved to the pages branch, remove the
logic from this branch and use the version on the pages branch.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

We want to have the pages updated whenever there's a change on any
`apertis/` branch. Drop the test case and allow the pages pipeline to be
run on any `apertis/` branch.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

We are going to start generating test cases for multiple generations of
Apertis to enable testing of stable releases. Add a menu to enable
browsing of the various releases.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Having the argument parsing in the atc_renderer code makes it much
harder to use it as a library. Pull the arg parsing into the
application.

As it turns out, atc didn't end up being used in this manner, but fix
it now to avoid this potentially needing to be addressed again in the
future.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

We want to redirect linking that pages without a version specified to
the latest to ensure existing linking continues to work. For everything
else, lets redirect to the main page.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

We want to move to providing test cases for multiple generations of
Apertis so as to enable us to develop the bits we need to do this,
begin by putting the existing output in `latest/`.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

The bundle generation with an unknown key doesn't display correctly.
Add a command to retrieve the `key0.key` file from gitlab.apertis.org.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

By default SabreLite board plays sound through HDMI connector.
In case tester want to output sound through headphone connector he should
configure PulseAudio for it.

After the 2 first commands (set-default-sink and set-sink-port) the sound
is heard correctly, but not after a reboot. This is due to mute state not
saved at this time, this needs the third command (after reboot) to fix it.
Further reboot will play sound correctly.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This commit add commands to generate a static update bundle with a key
not known by the DUT, from the original encrypted update.
It allows to test that a static update encrypted with an unknown key is not
applied.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

New version of apertis-update-manager expects offline updates to be
encrypted.
The apertis-update-manager-automount test ensure that non encrypted
updates are not applied, while encrypted updates applies correctly.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>