Use CGroups blkio to limit the read speed to 1MBps allowing to
remove the USB flash in predictable state even on extremely fast
devices.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The current test downloads from a legacy location the same contents already
shipped by the official multimedia demopack under `/home/shared`.

On the target and SDK images it is wasteful and silly, as those exact contents
come pre-installed.

On the minimal images this fails since there's not enough storage.

On the base SDK image the decision to not install the multimedia demopack was
taken explicitly as it is not relevant there.

Change the test to only run on the images where it is appropriate and rely on
the multimedia demopack being available out-of-the-box in `/home/shared`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The "as-root" run of dbus-installed-test failed as it exceeded the time
limit by 0.04s. Increase the time limit from 15 mins to 20 mins to avoid
this happening.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

The check-dbus-services aims to test all services configured as bus
activatable, however such a test is proving to be too naive and requires
significant investment to keep in a passing state.

There are an increasing number of services that have needed to be
blacklisted as they don't pass with reasonable rationale:

* Some require the ability to display a GUI and recieve a response that
  isn't currently viable with the current automated test platform nor is
  a GUI present on many of the image types.
* Some require specific devices to be present for the underlying service
  to start that isn't present in on the test systems.
* Some aren't completely installed and operational in the smaller cut-down
  images.

It has been decided that this test lacks sufficent benefit and significant
overhead and is thus being removed.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Emanuele Aina authored 5 years ago and Luis Araujo committed 5 years ago

The `make-ssl-cert` script specifically targets bash, so let's install it
explicitly.

The `ssl-cert` package is brought in by `postfix`, which is needed by `quilt`,
suggested by `devscript`.

This fixes the error below:
    Setting up ssl-cert (1.0.39ba1) ...
    /var/lib/dpkg/info/ssl-cert.postinst: 48: /var/lib/dpkg/info/ssl-cert.postinst: make-ssl-cert: not found
    dpkg: error processing package ssl-cert (--configure):
     installed ssl-cert package post-installation script subprocess returned error exit status 127

The install error also caused the subsequent tests to be skipped due to the
inconsistent state.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Folks test cases are not so relevant anymore.

The apparmor-folks test case has not been executed for several releases
already (it was not executed for 18.12 and for 18.09 was only executed
for half the platforms with some failures too).

This commit removes the apparmor-folks test case.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This will allow the didcot test case page to be included in the test
case index page and test report page.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Some issues were reported in comments from T5439.
This patch fixes them.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Martyn Welch authored 5 years ago and Frédéric Dalleau committed 5 years ago

The regex for parsing the test result is incorrect, leading to the test
reporting a pass rather than a fail as it should be. Change the regex to
make it report the correct result.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

After completion of APERTIS-5594 the offline upgrade test has been changed.
Now it is not possible to use random update bundle hence we must to
prepare the system to be upgraded to the same commit.
Added the description how to do that into all AUM manual tests.

Fixes: APERTIS-5808

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Mark some manual tests as 'low' since there are automatic versions
but we need to keep the steps for manual testing.
Added links to automated versions of these tests.
But we still need 2 tests with USB HW for manual testing.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The test is currently being run on all the image types, update the testcase to
reflect that fact.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

There is no benefit to execute this unit test in LAVA and it is already
being executed in OBS during package build time.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Do not need to pass the test name to the 'run-test-in-systemd'.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Rename disk-fsck-boot-rootfs test case file to disk-rootfs-fsck.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

The apertis update manager tests should be executed only in public armhf
images at the moment.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Rework a parse pattern for these cases.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Check if update file with any combination of incorrect Collection ID,
binding refs and timestamp couldn't be used in running system.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Ensure the system is booted with properly configured Collection ID.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ritesh Raj Sarraf authored 5 years ago and Frédéric Dalleau committed 5 years ago

Install geoclue-2.0 binary packages provided in the distribution

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Frédéric Dalleau authored 6 years ago and Frédéric Dalleau committed 5 years ago

This commit add new tests for update manager of OStree images.
Upgrade test, API test, rollback test, power cut test and out of space test.
Refer to https://phabricator.apertis.org/T5684

 for informations

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This testcase is meant to test a custom libfolks feature we carried downstream
Aand that was proposed upstream and WONTFIX'ed in
https://bugzilla.gnome.org/show_bug.cgi?id=741643



We're thus considering that this testcase does not bring any value to the
project and is rather an hindrance since:
* the actual use-case for the feature is not really documented
* the implementation is problematic and has major architectural issues
* a proper solution would involve doing it in EDS rather than libfolks

No replacement is planned since contacts management via libfolks is not in the
current priorities.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This is in order to keep consistency between the test definition name
and the file name, at the same time of following the same standard for
the rest of the test cases.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

With the rebase to Buster, the tracker packaging ABI has been bumped up to 2.0.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

With the rebase to Buster, the grilo packaging ABI has been bumped up to 0.3.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The pre-condition steps generated by the `install`/`git-repos` directives
where only rendered for automated testcases. Arguably, manual testers are
more interested in those being rendered than LAVA, which usually considers
the YAML enough as-is.

This currently affected the WebKit manual testcases such as
webkit2gtk-ac-3d-rendering.yaml where no installation pre-condition was
rendered, omitting fundamental steps needed by manual testers.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

We have them, let's use them as much as possible.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit 108fb918 was a bit too eager in changing the branch names and also
changed the ones in the unit tests, breaking them since the reference files
weren't update accordingly.

Revert those changes so that unit tests pass again.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In case if fsck has been started it savees 2 files in directory
'/run/initramfs':
- stamp file 'fsck-root' in case if fsck run was successful
- log file 'fsck.log' with filesystem check log

Since fsck is started on every boot it is enough to make sure we have
root file system check on initramfs stage.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

We are moving to per-release branches in every repository and this one is
no exception. :)

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Gustavo Noronha authored 6 years ago and Gustavo Noronha committed 6 years ago

With the move to upstream WebKit2GTK+, there is no longer a custom
launcher that uses Clutter, and MiniBrowser should be used instead.

We also take the opportunity to move away from the model of shipping
test data on packages, and add a new test data repository to gitlab
instead.

Gustavo Noronha authored 6 years ago and Gustavo Noronha committed 6 years ago

With the move away from a patched WebKit2GTK+ to an upstream version,
some features have been lost and there is no need (or way) to test them
anymore.

actor-tiles: no longer uses Clutter, the backing store does not use
Clutter actors anymore, that was a custom change

aligned-scroll: heuristic alignment lock of panning was a custom feature
which is not present upstream; upstream only supports setting the
direction lock before the pan starts

contextual-zoom: contextual zoom was a custom feature not available
upstream; it is not as important currently, though, since most pages are
now designed to be touchscreen friendly

lazy-click: also known as touch adjustment, it was also a custom feature
not available upstream; like contextual-zoom, it is not as relevant
nowadays, as sites are designed to be touch-friendly

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Test that the custom seccomp profile is applied and
works for Tiny containers.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Use new branch name for tiny containers tests.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Check if AppArmor profile for dbus-daemon in the container is loaded.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Add the test of AppArmor profile for Tiny containers.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Unprivileged container must to have a correct mappings to share the
device or folder.
Add 'user' user into 'disk' group for device sharing test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Split a single test case to several files per single test for
privileged and unprivileged containers.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

All needed packages are already the part of development image used as
host OS. As soon as we will be ready to switch to minimal image -- we
will need to switch to new test framework as well.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>