We are going to start generating test cases for multiple generations of
Apertis to enable testing of stable releases. Add a menu to enable
browsing of the various releases.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Having the argument parsing in the atc_renderer code makes it much
harder to use it as a library. Pull the arg parsing into the
application.

As it turns out, atc didn't end up being used in this manner, but fix
it now to avoid this potentially needing to be addressed again in the
future.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

We want to redirect linking that pages without a version specified to
the latest to ensure existing linking continues to work. For everything
else, lets redirect to the main page.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

We want to move to providing test cases for multiple generations of
Apertis so as to enable us to develop the bits we need to do this,
begin by putting the existing output in `latest/`.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

The bundle generation with an unknown key doesn't display correctly.
Add a command to retrieve the `key0.key` file from gitlab.apertis.org.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

By default SabreLite board plays sound through HDMI connector.
In case tester want to output sound through headphone connector he should
configure PulseAudio for it.

After the 2 first commands (set-default-sink and set-sink-port) the sound
is heard correctly, but not after a reboot. This is due to mute state not
saved at this time, this needs the third command (after reboot) to fix it.
Further reboot will play sound correctly.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This commit add commands to generate a static update bundle with a key
not known by the DUT, from the original encrypted update.
It allows to test that a static update encrypted with an unknown key is not
applied.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

New version of apertis-update-manager expects offline updates to be
encrypted.
The apertis-update-manager-automount test ensure that non encrypted
updates are not applied, while encrypted updates applies correctly.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

The ssh-copy-id command syntax in sdk-ade-remote-debugging.yaml is
incorrect in two ways:

- The user is in capitals, whilst the actual username used should be in
  lower case
- The test case uses `$HOST` as a convention for the hostname/IP to be used
  for the target device, this command is missing the dollar sign.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Baghmar Tarun authored 4 years ago and Martyn Welch committed 4 years ago

Rework the testcase to not require packages to be installed and thus work on systems with an immutable rootfs like the ostree images.

Apertis: T5734

Signed-off-by: Tarun Baghmar <tarun.baghmar@in.bosch.com>

Commit  d0487486 "lava: Reduce redundancy in the profile
definitions" introduced a change that ended up causing issues with the
AUM tests, so let's go back to the previous state.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 4 years ago and Peter Senna Tschudin committed 4 years ago

Move from the old variable names which dates back to when the pipeline
was on Jenkins to something a bit more consistent.

This would be a breaking change for pipelines which include the
template, but since the main recipe repo has not switched over yet we
can play fast and loose safely.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 4 years ago and Peter Senna Tschudin committed 4 years ago

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use YAML anchors and extensions to avoid repeating the same contents over
and over.

This commit does not introduce any change in the generated job
definitions, except for the case below, which is harmless:

    apertis_ostree-minimal-armhf-uboot
    -      commands: ['reset']
    +      commands: ['run bootcmd_mmc0']

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The test currently fails due to timeout.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Use the multiple ostree preconditions support introduced in 6b2fd91e
instead of hardcoding the blurb for the second repository.

Also fix the branches and the resulting paths for the unpacked scripts.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

It turns out that LAVA really wants strings for the metadata entries and
didn't like nesting a dictionary there.

For this reason it was failing to submit jobs with this error:

    Submitting job group-apparmor-tpl.yaml: <Fault 400: "Problem with submitted job data: expected str for dictionary value @ data['metadata']['source']">

Unfortunately our custom `lava-submit.py` was silently eating the error
so the pipeline was marked as successful.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Emanuele Aina committed 4 years ago

Rework the testcase to not require packages to be installed and thus
work on systems with an immutable rootfs like the ostree images.

Apertis: T5734

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

The `${source_wip:=false}` snippet not only sets the `$source_wip`
variable if unset, but also executes the result of its expansion.

In our case it executed `false`, which made the pipeline very unhappy.

However, all tests submitted from this repository directly should be
considered "wip" and it will be up to the image repository including the
gitlab-ci job template to define the logic for which submissions should
be "wip" or not.

So drop the logic and use a plain `source_wip` variable defaulting to
`true`. The meaning of the variable was not documented either, so add a
blurb about it.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Address a regression introduced in commit 6b2fd91e "Turn the OSTree
preconditions into a list" where the iteration on the preconditions list
was done improperly.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The LAVA lab is currently facing badnwidth issues. Switching to a
different ISP is planned, but due to the COVID impact there's no
progress on that front.

The current ade tests got badly impacted, with the download phase
consuming all the allocated time.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `source:wip:` metadata can be used to indicate that a submission
came from a work-in-progress branch and that it should not be accounted
when checking the overall health of the projects.

For instance, the Phabricator bridge can use this information to avoid
opening bug tasks for failures coming from a WIP branch.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Introduce the variables below:

* source_ref
* source_commit
* source_pipeline
* source_job

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In preparation for adding more metadata, make the `source` entry an
object rather than a single string.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Default to `null` rather than hardconding a possibly misleading source.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Emanuele Aina committed 4 years ago

If a test case requires multiple repositories it can now use a list of
objects as the OSTree preconditions to check them all out.

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Submit every modification to LAVA as part of the pipeline to set up a
direct feedback loop on non-main branches.

The image recipes pipeline can then re-use the job template here.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Port the below MR from apertis-tests, which got landed while we were
moving the templates around:
https://gitlab.apertis.org/pkg/development/apertis-tests/-/merge_requests/190



This reverts the introduction of the OTA update tests on the internal
minimal armhf since their behavior isn't stable and there is no value to
test AUM on this platform.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Keeping the LAVA job templates in `apertis-tests` has many problems.

Since `apertis-tests` is also a package, updates for stable releases
should go through `-updates` branches, but that plays badly with the
image building recipes as they only pull from the the main branch.

There's also an annoying disconnect about what the testcases here say
about where they should be run, and the actual groups used by the
templates.

By moving the LAVA templates here we can later improve them by
generating the groups based on the actual data in the testcases.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Hattihal Basavana Gowda Chethan authored 4 years ago and Emanuele Aina committed 4 years ago

When some test scripts got moved from /usr/share/chaiwala-tests/ to
/usr/lib/apertis-tests/ their path in the test definitions didn't get
updated, causing the tests to fail:

    + common/run-test-in-systemd --basename --user=user -- /usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh
    /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: 19: /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: Job for generated-test-case-test-valgrind.service failed because the control process exited with error code./usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh: not found

To fix the issue, rather that using the version shipped in
/usr/lib/apertis-tests/ use the one in the git repository since we
already have it checked out.

Fixes: https://phabricator.apertis.org/T6349
Fixes: https://phabricator.apertis.org/T6350
Fixes: https://phabricator.apertis.org/T6351
Fixes: https://phabricator.apertis.org/T6352



Signed-off-by: hattihalbasavanagowda.chethan <hattihalbasavanagowda.chethan@in.bosch.com>

This reverts commit 8ea0915d.
Disable internal armhf image from supported images types
for OTA update tests since their behavior aren't stable and
there is no value to test AUM on this platform.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 4 years ago

This commit fixes the name of the virtual machines for the psdk-test,
and make small fixes for typos and grammar issues.

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>
Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Luis Araujo authored 5 years ago and Peter Senna Tschudin committed 4 years ago

This commit updates the SDK persistent disk test cases to use the new
psdk-test script that semi-automate the execution of these tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

The templates hardcoded the `master` branch for the repositories
shipping the actual test scripts and binaries, but we use release-coded
branch names like `apertis/v2021dev2` there so the links in the
instructions pointed to the wrong place.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use names for the precondition variables that reflect the properties in
the testcase definitions so it's easier to spot what's going on.

This will alos make it easier to later expand them to use multiple
values.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Frederic Danis committed 4 years ago

1. sdk-flatpak-build-helloworld-app.yaml: building the app by using mildenhall runtime.
2. sdk-flatpak-helloworld-app.yaml: running the helloworld-app present in the server.

Signed-off-by: Tarun Baghmar <Tarun.Baghmar@in.bosch.com>