Our guidelines say that to contribute people should fork repositories,
but at the moment this gives them a broken pipeline.

Tweak things in a way that the main project triggers
tests/apertis-test-cases-web to collect and indirectly publish all the
release branches while forked repositories directly publish their own
branches via Pages for easier testing.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Lightweight CI runners have been introduced to run jobs which have low
resource consumption and do not need privileged containers to reduce costs
and latency, so let's use them.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The test uses `/usr/lib/chaiwala-tests/bluez/simple-agent` shipped in
the `chaiwala-tests` package, let's explicitly depend on it.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Martyn Welch authored 4 years ago and Emanuele Aina committed 4 years ago

Two of the test cases point to videos to show what a successful run looks
like. These were randomly stored on the old wiki machine. Rather than
having unlinked static content on the website, add these to the
`images` folder and update links.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Martyn Welch authored 4 years ago and Emanuele Aina committed 4 years ago

We are going to start generating test cases for multiple generations of
Apertis to enable testing of stable releases. Add a menu to enable
browsing of the various releases.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Martyn Welch authored 4 years ago and Emanuele Aina committed 4 years ago

Having the argument parsing in the atc_renderer code makes it much
harder to use it as a library. Pull the arg parsing into the
application.

As it turns out, atc didn't end up being used in this manner, but fix
it now to avoid this potentially needing to be addressed again in the
future.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

By default SabreLite board plays sound through HDMI connector.
In case tester want to output sound through headphone connector he should
configure PulseAudio for it.

After the 2 first commands (set-default-sink and set-sink-port) the sound
is heard correctly, but not after a reboot. This is due to mute state not
saved at this time, this needs the third command (after reboot) to fix it.
Further reboot will play sound correctly.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Frederic Danis authored 4 years ago and Emanuele Aina committed 4 years ago

The test currently fails due to timeout.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Use the multiple ostree preconditions support introduced in 6b2fd91e
instead of hardcoding the blurb for the second repository.

Also fix the branches and the resulting paths for the unpacked scripts.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Address a regression introduced in commit 6b2fd91e "Turn the OSTree
preconditions into a list" where the iteration on the preconditions list
was done improperly.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The LAVA lab is currently facing badnwidth issues. Switching to a
different ISP is planned, but due to the COVID impact there's no
progress on that front.

The current ade tests got badly impacted, with the download phase
consuming all the allocated time.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 4 years ago and Emanuele Aina committed 4 years ago

If a test case requires multiple repositories it can now use a list of
objects as the OSTree preconditions to check them all out.

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

The templates hardcoded the `master` branch for the repositories
shipping the actual test scripts and binaries, but we use release-coded
branch names like `apertis/v2021dev2` there so the links in the
instructions pointed to the wrong place.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use names for the precondition variables that reflect the properties in
the testcase definitions so it's easier to spot what's going on.

This will alos make it easier to later expand them to use multiple
values.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Baghmar Tarun authored 5 years ago and Emanuele Aina committed 4 years ago

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Baghmar Tarun authored 5 years ago and Emanuele Aina committed 4 years ago

1. ofono-sms-recieve
2. ofono-sms-send

Signed-off-by: Baghmar Tarun <tarun.baghmar@in.bosch.com>

Frederic Danis authored 5 years ago and Emanuele Aina committed 4 years ago

Currently `apparmor-bluez-avrcp-volume` test request to run
`bluez-avrcp-volume` test, allowing to install all dependencies for both
tests. Running solely one test ends up in errors due to missing
dependencies.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Frederic Danis authored 5 years ago and Emanuele Aina committed 4 years ago

Allow for Mildenhall components known to generate unrelated AppArmor
complaints.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Benani Sagar Kishore authored 4 years ago and Emanuele Aina committed 4 years ago

Signed-off-by: Benani Sagar Kishore (RBEI/ECO2) <SagarKishore.Benani@in.bosch.com>

Baghmar Tarun authored 4 years ago and Emanuele Aina committed 4 years ago

After the discussions following test cases will be dropped.
apparmor-tracker
folks-ofono-backend
telepathy-ring
tracker-indexing-local-storage
tracker-indexing-mass-storage
traffic-control-basic

Apertis: T5734
Signed-off-by: Tarun Baghmar <tarun.baghmar@in.bosch.com>
Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Hattihal Basavana Gowda Chethan authored 4 years ago and Emanuele Aina committed 4 years ago

When some test scripts got moved from /usr/share/chaiwala-tests/ to
/usr/lib/apertis-tests/ their path in the test definitions didn't get
updated, causing the tests to fail:

    + common/run-test-in-systemd --basename --user=user -- /usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh
    /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: 19: /var/lib/lava-1826179/1/tests/6_sdk-debug-tools-valgrind/apertis-tests/common/tee-exec: Job for generated-test-case-test-valgrind.service failed because the control process exited with error code./usr/share/chaiwala-tests/sdk/automated/test-valgrind.sh: not found

To fix the issue, rather that using the version shipped in
/usr/lib/apertis-tests/ use the one in the git repository since we
already have it checked out.

Fixes: https://phabricator.apertis.org/T6349
Fixes: https://phabricator.apertis.org/T6350
Fixes: https://phabricator.apertis.org/T6351
Fixes: https://phabricator.apertis.org/T6352



Signed-off-by: hattihalbasavanagowda.chethan <hattihalbasavanagowda.chethan@in.bosch.com>

Rather than looping over branches (in included template from `pages`),
generate the test cases here, provide these as an artifact and deploy
in a separate job. This avoids the regeneration of pages that have
not changed.

Use the apertis-test-cases-web as part of a multi-project pipeline to
deploy.

Whilst we are here, utilise an environment variable for the release to
make it easier to migrate this between releases.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

This reverts commit c8252838.
Disable internal armhf image from supported images types
for OTA update tests since their behavior aren't stable and
there is no value to test AUM on this platform.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use separate commit for changing the target to avoid
potential merging issues in future.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add internal armhf image into supported images types
for OTA update tests.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add the test for checking non-successful update during OTA update.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ensure that only updates signed with valid key can be applied.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Test OTA updates behavior with lack of disk space.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add automatic test of unattended network updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add automatic test for network updates API.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Xorg is no more expected as process in complain mode.

Fixes: APERTIS-6233

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Fix aa_log_extract_tokens.sh path as it is intalled by
apertis-tests-apparmor-report package.

Fixes: APERTIS-6233

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Emanuele Aina authored 5 years ago and Luis Araujo committed 5 years ago

Target images using the etnaviv gfx stack are now built daily, and test jobs
are already being submitted, reflect this in the testcases.

See https://phabricator.apertis.org/T6395

 for more details.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

It has been observed that the execution time for the test varies.
Icrease the timeout to ensure that the build doesn't get killed halfway
through.

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

This reverts commit 96227328.

The AUM tests have been experimentally enabled on arm64 and amd64 but it seems
things are still too broken to officially support them.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Since the rebase to Buster we are no longer shipping `clutter/test-text`,
making the test execution impossible to complete.

Replace the test program with the MiniBrowser from WebKitGTK opening
a simple WebGL scene and testing that the output is not stretched.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Use `apertis/v2020pre` branch as default.
Do not test outdated branches.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add the description of how to add the proxy address for OSTree allowing
online upgrades both for `ostree` and AUM in restricted networks.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>