Emanuele Aina authored 5 years ago and Luis Araujo committed 5 years ago

The lack of links support in the `steps:` list caused the `sdk-persistent-disk`
to lack important instructions, making testers more confused than necessary.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Remove this package dependency from the dbus-installed-tests test case
since this package is already installed by coreutils.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Use a single step for reboot waiting. Added the example how to track the
upgrade status while waiting for automatic upgrade and reboot.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use apertis-update-manager-automount for the test case name so it
matches its filename.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This commit splits the sanity-check test case into a specific manual
version of it (sanity-check-manual).

This will solve the issue of overwriting automated tests results for
this test case on the qa-report-app.

The name sanity-check is preserved for the automated version so there is
no need to apply any changes to the LAVA jobs.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

ade-commands test is failing, and this seems due to long download
times. Increase the timeout of the test.
The sysroot weight 1GB
Refer to APERTIS-6124

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Test cases files should always match their metadata name.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Add a test to check if the persistent disk functionality is working
properly by:
 - Installing a sysroot on the old SDK
 - Configuring the old SDK to use the persistent disk
 - Configuring the new SDK to use the persistent disk
 - Check that the sysroot is available on the new SDK

Signed-off-by: Peter Senna Tschudin <peter.senna@collabora.com>

This test used a simple echo pass. The qa report was not able to
understand that the test passed successfully because it eliminated
tests results of the "lava" test group.
Use the specific lava tool for reporting instead

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Add a testcase to ensure that the sample image recipes can be built on the SDK
and that the commands in the documentation work correctly.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Frédéric Dalleau authored 5 years ago and Denis Pynkin committed 5 years ago

Currently paramiko do not support Keys in RFC4716 format that
have landed with the buster update.
So any keys need to be generated in legacy PEM format.

Fixes: APERTIS-5932
Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This reverts commit c86d18a7.
The problem of low speed bandwidth is kernel crash (APERTIS-6016) which
completely blocking the USB unplug test.

With changes provided in previous commit we provide the binary diff
big enough to catch the proper timing by human and without kernel
crashes.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

With this change we prepare the "outdated" commit without the
content of some directories. This allows to provide the significant
difference between "original" and "outdated" commits, so it is easy
to check it in booted deployment.
The side effect of huge size difference during upgrade is more stable
and convenient testing of "disk full" and "usb unplug" cases.

More detailed description is available here:
https://phabricator.apertis.org/T5439#189061



Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Test if update doesn't break anything if there is not enough free space
on system disk.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Test if rollback and blacklist mechanisms works for network updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Test if AUM is able to correctly read network-related settings
from configuration file and able to do automatic updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Manual test aimed to check the API between update manager and
`updatectl` tool allowing to start network updates with CLI.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Improve test case description.
Use the new NFS image-deployment value.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

After booting an nfsroot, an image is flashed and we only want to
ensure the image boots properly to a shell.
This test uses the nfsroot image type.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Frédéric Dalleau authored 5 years ago and Frédéric Dalleau committed 5 years ago

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Frédéric Dalleau authored 5 years ago and Frédéric Dalleau committed 5 years ago

That command is no longer needed because the u-boot environment is now
partially stored directly in the image. Everytime the image is reflashed, the
environment is wiped out.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The testcase uses `run-test-in-systemd` which in turn uses `lava-test-case` so
there's no need to set a custom pattern to parse test results.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The expected output doesn't really match what the tests will emit, and
there's little point in documenting the standard LAVA expectations here.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Need to reduce the read speed for the test since recent libostree do not
read the whole update bundle during the test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use CGroups blkio to limit the read speed to 1MBps allowing to
remove the USB flash in predictable state even on extremely fast
devices.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Need to generate the recent version of tests description for QA!

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Incorrect name has been used leading to failure of the container startup.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use branch apertis/v2019pre for all test cases.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Removed the mention of 'update_available' U-Boot variable -- we do not
use environment-based rollback driver anymore.
Updated the output for blacklisted part -- now we have additional
information about Collection ID.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Let `run-test-in-systemd` figure out the appropriate subtest name when
launching `ade-tests-custom.sh` and `ade-tests-defaults.sh`, instead of
confusingly setting the name as `run-test` for both.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use CGroups blkio to limit the read speed to 1MBps allowing to
remove the USB flash in predictable state even on extremely fast
devices.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The current test downloads from a legacy location the same contents already
shipped by the official multimedia demopack under `/home/shared`.

On the target and SDK images it is wasteful and silly, as those exact contents
come pre-installed.

On the minimal images this fails since there's not enough storage.

On the base SDK image the decision to not install the multimedia demopack was
taken explicitly as it is not relevant there.

Change the test to only run on the images where it is appropriate and rely on
the multimedia demopack being available out-of-the-box in `/home/shared`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The "as-root" run of dbus-installed-test failed as it exceeded the time
limit by 0.04s. Increase the time limit from 15 mins to 20 mins to avoid
this happening.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

The check-dbus-services aims to test all services configured as bus
activatable, however such a test is proving to be too naive and requires
significant investment to keep in a passing state.

There are an increasing number of services that have needed to be
blacklisted as they don't pass with reasonable rationale:

* Some require the ability to display a GUI and recieve a response that
  isn't currently viable with the current automated test platform nor is
  a GUI present on many of the image types.
* Some require specific devices to be present for the underlying service
  to start that isn't present in on the test systems.
* Some aren't completely installed and operational in the smaller cut-down
  images.

It has been decided that this test lacks sufficent benefit and significant
overhead and is thus being removed.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>