Some issues were reported in comments from T5439.
This patch fixes them.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Martyn Welch authored 6 years ago and Frédéric Dalleau committed 6 years ago

The regex for parsing the test result is incorrect, leading to the test
reporting a pass rather than a fail as it should be. Change the regex to
make it report the correct result.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

After completion of APERTIS-5594 the offline upgrade test has been changed.
Now it is not possible to use random update bundle hence we must to
prepare the system to be upgraded to the same commit.
Added the description how to do that into all AUM manual tests.

Fixes: APERTIS-5808

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Mark some manual tests as 'low' since there are automatic versions
but we need to keep the steps for manual testing.
Added links to automated versions of these tests.
But we still need 2 tests with USB HW for manual testing.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The test is currently being run on all the image types, update the testcase to
reflect that fact.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

There is no benefit to execute this unit test in LAVA and it is already
being executed in OBS during package build time.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Do not need to pass the test name to the 'run-test-in-systemd'.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Rename disk-fsck-boot-rootfs test case file to disk-rootfs-fsck.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

The apertis update manager tests should be executed only in public armhf
images at the moment.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Rework a parse pattern for these cases.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Check if update file with any combination of incorrect Collection ID,
binding refs and timestamp couldn't be used in running system.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Ensure the system is booted with properly configured Collection ID.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ritesh Raj Sarraf authored 6 years ago and Frédéric Dalleau committed 6 years ago

Install geoclue-2.0 binary packages provided in the distribution

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Frédéric Dalleau authored 6 years ago and Frédéric Dalleau committed 6 years ago

This commit add new tests for update manager of OStree images.
Upgrade test, API test, rollback test, power cut test and out of space test.
Refer to https://phabricator.apertis.org/T5684

 for informations

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This testcase is meant to test a custom libfolks feature we carried downstream
Aand that was proposed upstream and WONTFIX'ed in
https://bugzilla.gnome.org/show_bug.cgi?id=741643



We're thus considering that this testcase does not bring any value to the
project and is rather an hindrance since:
* the actual use-case for the feature is not really documented
* the implementation is problematic and has major architectural issues
* a proper solution would involve doing it in EDS rather than libfolks

No replacement is planned since contacts management via libfolks is not in the
current priorities.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This is in order to keep consistency between the test definition name
and the file name, at the same time of following the same standard for
the rest of the test cases.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

With the rebase to Buster, the tracker packaging ABI has been bumped up to 2.0.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

With the rebase to Buster, the grilo packaging ABI has been bumped up to 0.3.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In case if fsck has been started it savees 2 files in directory
'/run/initramfs':
- stamp file 'fsck-root' in case if fsck run was successful
- log file 'fsck.log' with filesystem check log

Since fsck is started on every boot it is enough to make sure we have
root file system check on initramfs stage.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Gustavo Noronha authored 6 years ago and Gustavo Noronha committed 6 years ago

With the move to upstream WebKit2GTK+, there is no longer a custom
launcher that uses Clutter, and MiniBrowser should be used instead.

We also take the opportunity to move away from the model of shipping
test data on packages, and add a new test data repository to gitlab
instead.

Gustavo Noronha authored 6 years ago and Gustavo Noronha committed 6 years ago

With the move away from a patched WebKit2GTK+ to an upstream version,
some features have been lost and there is no need (or way) to test them
anymore.

actor-tiles: no longer uses Clutter, the backing store does not use
Clutter actors anymore, that was a custom change

aligned-scroll: heuristic alignment lock of panning was a custom feature
which is not present upstream; upstream only supports setting the
direction lock before the pan starts

contextual-zoom: contextual zoom was a custom feature not available
upstream; it is not as important currently, though, since most pages are
now designed to be touchscreen friendly

lazy-click: also known as touch adjustment, it was also a custom feature
not available upstream; like contextual-zoom, it is not as relevant
nowadays, as sites are designed to be touch-friendly

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Test that the custom seccomp profile is applied and
works for Tiny containers.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Use new branch name for tiny containers tests.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Check if AppArmor profile for dbus-daemon in the container is loaded.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Add the test of AppArmor profile for Tiny containers.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Unprivileged container must to have a correct mappings to share the
device or folder.
Add 'user' user into 'disk' group for device sharing test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Split a single test case to several files per single test for
privileged and unprivileged containers.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

All needed packages are already the part of development image used as
host OS. As soon as we will be ready to switch to minimal image -- we
will need to switch to new test framework as well.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Add following tests:
- folder mapping
- device mapping

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The preconditions mentionned to download another test.
Actually this is because this test uses bluez-setup so we add the steps for
bluez-setup in generated form.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This field merges the image variant and arch fields in order to clearly
specify in which images a test case should be executed.

This commit adds this new field to each test case file with the specific
images where it should be executed and updates the tests and format
definition files.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Denis Pynkin authored 6 years ago and Frédéric Dalleau committed 6 years ago

Test is reworked to be more reliable and prevent the system reboot.
Added `core.min-free-space-percent = 0` option to avoid complex
calculation and re-creation of big file.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Additional hint for user stating when he need to remove the flash drive.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This commit adds support to have multiple image variants in the test
cases files.

It also adds the respective image variants list relevant to each test
case.

This commit also updates the tests files and format definition files.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This is a madantory field that specifies the deployment type of the
image where this test case can be executed.

The commit adds the image-deployment field for each test case, and it
currently can be a list with the following values: APT, OSTree, Any.

This commit also updates the tests files.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This commit renames the image-type field to image-variant
in the test cases files.

It also updates the parser, renderer code, test_case template
and unittests files in order to use the new name.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Add test to check proper behavior in the following circumstances:
- Power cut during an update
- USB mass storage unplug during an update
- The disk is full during an update

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Frédéric Dalleau authored 6 years ago and Denis Pynkin committed 6 years ago

Add a test to exercise the features of the Apertis update manager.
These tests only works on OSTree images and check following:
- API of update manager with `updatectl` utility
- automatic update from mass storage device with static delta
- manual upgrade with static delta file
- automatic rollback from "failed" update
- blacklist feature
- rollback feature of U-Boot bootloader

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>