The expected output doesn't really match what the tests will emit, and
there's little point in documenting the standard LAVA expectations here.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Need to reduce the read speed for the test since recent libostree do not
read the whole update bundle during the test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use CGroups blkio to limit the read speed to 1MBps allowing to
remove the USB flash in predictable state even on extremely fast
devices.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Need to generate the recent version of tests description for QA!

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Incorrect name has been used leading to failure of the container startup.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use branch apertis/v2019pre for all test cases.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Removed the mention of 'update_available' U-Boot variable -- we do not
use environment-based rollback driver anymore.
Updated the output for blacklisted part -- now we have additional
information about Collection ID.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Let `run-test-in-systemd` figure out the appropriate subtest name when
launching `ade-tests-custom.sh` and `ade-tests-defaults.sh`, instead of
confusingly setting the name as `run-test` for both.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Use CGroups blkio to limit the read speed to 1MBps allowing to
remove the USB flash in predictable state even on extremely fast
devices.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The current test downloads from a legacy location the same contents already
shipped by the official multimedia demopack under `/home/shared`.

On the target and SDK images it is wasteful and silly, as those exact contents
come pre-installed.

On the minimal images this fails since there's not enough storage.

On the base SDK image the decision to not install the multimedia demopack was
taken explicitly as it is not relevant there.

Change the test to only run on the images where it is appropriate and rely on
the multimedia demopack being available out-of-the-box in `/home/shared`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The "as-root" run of dbus-installed-test failed as it exceeded the time
limit by 0.04s. Increase the time limit from 15 mins to 20 mins to avoid
this happening.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

The check-dbus-services aims to test all services configured as bus
activatable, however such a test is proving to be too naive and requires
significant investment to keep in a passing state.

There are an increasing number of services that have needed to be
blacklisted as they don't pass with reasonable rationale:

* Some require the ability to display a GUI and recieve a response that
  isn't currently viable with the current automated test platform nor is
  a GUI present on many of the image types.
* Some require specific devices to be present for the underlying service
  to start that isn't present in on the test systems.
* Some aren't completely installed and operational in the smaller cut-down
  images.

It has been decided that this test lacks sufficent benefit and significant
overhead and is thus being removed.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Emanuele Aina authored 5 years ago and Luis Araujo committed 5 years ago

The `make-ssl-cert` script specifically targets bash, so let's install it
explicitly.

The `ssl-cert` package is brought in by `postfix`, which is needed by `quilt`,
suggested by `devscript`.

This fixes the error below:
    Setting up ssl-cert (1.0.39ba1) ...
    /var/lib/dpkg/info/ssl-cert.postinst: 48: /var/lib/dpkg/info/ssl-cert.postinst: make-ssl-cert: not found
    dpkg: error processing package ssl-cert (--configure):
     installed ssl-cert package post-installation script subprocess returned error exit status 127

The install error also caused the subsequent tests to be skipped due to the
inconsistent state.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Folks test cases are not so relevant anymore.

The apparmor-folks test case has not been executed for several releases
already (it was not executed for 18.12 and for 18.09 was only executed
for half the platforms with some failures too).

This commit removes the apparmor-folks test case.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This will allow the didcot test case page to be included in the test
case index page and test report page.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Some issues were reported in comments from T5439.
This patch fixes them.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Martyn Welch authored 5 years ago and Frédéric Dalleau committed 5 years ago

The regex for parsing the test result is incorrect, leading to the test
reporting a pass rather than a fail as it should be. Change the regex to
make it report the correct result.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

After completion of APERTIS-5594 the offline upgrade test has been changed.
Now it is not possible to use random update bundle hence we must to
prepare the system to be upgraded to the same commit.
Added the description how to do that into all AUM manual tests.

Fixes: APERTIS-5808

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Mark some manual tests as 'low' since there are automatic versions
but we need to keep the steps for manual testing.
Added links to automated versions of these tests.
But we still need 2 tests with USB HW for manual testing.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The test is currently being run on all the image types, update the testcase to
reflect that fact.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

There is no benefit to execute this unit test in LAVA and it is already
being executed in OBS during package build time.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Do not need to pass the test name to the 'run-test-in-systemd'.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Rename disk-fsck-boot-rootfs test case file to disk-rootfs-fsck.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

The apertis update manager tests should be executed only in public armhf
images at the moment.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Rework a parse pattern for these cases.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Check if update file with any combination of incorrect Collection ID,
binding refs and timestamp couldn't be used in running system.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Denis Pynkin authored 5 years ago and Denis Pynkin committed 5 years ago

Ensure the system is booted with properly configured Collection ID.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ritesh Raj Sarraf authored 5 years ago and Frédéric Dalleau committed 5 years ago

Install geoclue-2.0 binary packages provided in the distribution

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Frédéric Dalleau authored 6 years ago and Frédéric Dalleau committed 5 years ago

This commit add new tests for update manager of OStree images.
Upgrade test, API test, rollback test, power cut test and out of space test.
Refer to https://phabricator.apertis.org/T5684

 for informations

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This testcase is meant to test a custom libfolks feature we carried downstream
Aand that was proposed upstream and WONTFIX'ed in
https://bugzilla.gnome.org/show_bug.cgi?id=741643



We're thus considering that this testcase does not bring any value to the
project and is rather an hindrance since:
* the actual use-case for the feature is not really documented
* the implementation is problematic and has major architectural issues
* a proper solution would involve doing it in EDS rather than libfolks

No replacement is planned since contacts management via libfolks is not in the
current priorities.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This is in order to keep consistency between the test definition name
and the file name, at the same time of following the same standard for
the rest of the test cases.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

With the rebase to Buster, the tracker packaging ABI has been bumped up to 2.0.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

With the rebase to Buster, the grilo packaging ABI has been bumped up to 0.3.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The pre-condition steps generated by the `install`/`git-repos` directives
where only rendered for automated testcases. Arguably, manual testers are
more interested in those being rendered than LAVA, which usually considers
the YAML enough as-is.

This currently affected the WebKit manual testcases such as
webkit2gtk-ac-3d-rendering.yaml where no installation pre-condition was
rendered, omitting fundamental steps needed by manual testers.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

We have them, let's use them as much as possible.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit 108fb918 was a bit too eager in changing the branch names and also
changed the ones in the unit tests, breaking them since the reference files
weren't update accordingly.

Revert those changes so that unit tests pass again.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

In case if fsck has been started it savees 2 files in directory
'/run/initramfs':
- stamp file 'fsck-root' in case if fsck run was successful
- log file 'fsck.log' with filesystem check log

Since fsck is started on every boot it is enough to make sure we have
root file system check on initramfs stage.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

We are moving to per-release branches in every repository and this one is
no exception. :)

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Gustavo Noronha authored 6 years ago and Gustavo Noronha committed 6 years ago

With the move to upstream WebKit2GTK+, there is no longer a custom
launcher that uses Clutter, and MiniBrowser should be used instead.

We also take the opportunity to move away from the model of shipping
test data on packages, and add a new test data repository to gitlab
instead.