The check-dbus-services test aims to test all services configured as bus
activatable, however such a test is proving to be too naive and requires
significant investment to keep in a passing state.
There are an increasing number of services that have needed to be
blacklisted as they don't pass with reasonable rationale:

* Some require the ability to display a GUI and recieve a response that
  isn't currently viable with the current automated test platform nor is a
  GUI present on many of the image types.
* Some require specific devices to be present for the underlying service
  to start that isn't present in on the test systems.
* Some aren't completely installed and operational in the smaller cut-down
  images.

It has been decided that this test lacks sufficent benefit and significant
overhead and is thus being removed.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Folks test cases are not so relevant anymore.

The apparmor-folks test case has not been executed for several releases
already (it was not executed for 18.12 and for 18.09 was only executed
for half the platforms with some failures too).

This commit removes the apparmor-folks tests.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This commit enables the tracker-indexing-local-storage test case in the
minimal images.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Tracker reset now asks for confirmation before deleting index. This needs
to be answered if we want to successfully run the command and thus our
tests.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Fixed the syntax issue for new group added

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

The `cgroups-resource-control` and `dbus-dos-reply-time` tests are supposed to
be run on all our platforms, but they never got included in the rounds for the
minimal images. Neither 18.12 nor 18.09 have them in their test reports (and
likely older releases as well).

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The AppArmor tests are meant to run on all our platforms, but they never got
included in the rounds for the minimal images. Neither 18.12 nor 18.09 have
them in their test reports.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Denis Pynkin authored 6 years ago and Frédéric Dalleau committed 6 years ago

Use the same name as in test cases for 'aum-offline-upgrade-collection_id'.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

There is no benefit to execute this unit test in LAVA and it is already being
executed in OBS during package build time.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This renames the disk-fsck test case files from the lava job templates
to have the same name of the test case.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Update the test program from the Grilo 0.2 API to the 0.3 one shipped in the
Buster-based v2019dev0 release.

* replace Registry.load_plugin_by_id() with Registry.load_all_plugins() and
  Rregistry.activate_plugin_by_id()
* replace check of MediaBox subclassing with Media.is_container()

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

The error was reported by dch when generated the previous release
dch: warning: debian/changelog(l21): badly formatted trailer line

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Frédéric Dalleau authored 6 years ago and Frédéric Dalleau committed 6 years ago

The update manager tests report the different phases as
aum-*-phase-[1..x]. This is does not cope with the qa reporting
tool that expects a result with the same name than the test
yaml definition. Let's consider the last phase of the test to be
the final result. Any previous error would have been carried forward.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

The internal i.MX6 images have been switched to the same u-boot setup used by
the public uboot images, introduced here with commit acf72348.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Martyn Welch authored 6 years ago and Emanuele Aina committed 6 years ago

Newer kernels (>=4.18) are far more stringent regarding the generation of
random numbers. This results in boot taking much longer when there are
limited sources of good entropy.

Tweak the qemu configuration to add a source of entropy so that we aren't
starved of random numbers during boot.

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

Additional section causing the skip of previous tests definations
without syntax errors from LAVA side.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Use the name "ostree-wrong-collection-id" since previos was already
used in other test.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Check if update file with any combination of incorrect Collection ID,
binding refs and timestamp couldn't be used in running system.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

It got disabled when things were in flux, now it is time to enable it again.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Denis Pynkin authored 6 years ago and Denis Pynkin committed 6 years ago

Create a new group for common tests to b run only on ostree images.
Add initial ostree common test: check the Collection ID on booted image.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Ritesh Raj Sarraf authored 6 years ago and Frédéric Dalleau committed 6 years ago

The old geoclue-get-address tool was specific to geoclue-0.xx and some
of the interfaces that it used is no more available with GeoClue2.

The newer where-am-i tool is part of the GeoClue2 package

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

To make sure the test are executed by Jenkins, they need to be referenced
in the profile. The best way to do so is to make tests groups.

Those are not real groups but rather per test groups
This is due to the fact that lava has a bug that prevents
reflashing the same device twice when tftp is in use.
It can be identified by the following error:
Bug error: [Errno 2] No such file or directory: '/var/lib/lava/dispatcher/tmp/1547278/tftp-deploy-3mrphokm/ramdisk/ramdisk.cpio.gz.uboot'

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

When uboot detects that the bootcount exceeds the limit, it
should run the altbootcmd, but if the boot is interrupted, such
as on a serial port or when lava handles the boot, the altbootcmd is not used.
By using reset, we ensure lava does not interrupt the second boot.
This has a side effect: the bootcount is incremented by two each time
the os boots. So the test accounts for it.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

The overlay is composed of a long list of files and thus
generates an enormous output. Which slow down both downloading pages
and browsing them without any benefit.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

For apertis-update-manager tests, the reboot tpl is used to trigger a
reboot without using apertis-nfs and reflashing.
After a test successfully terminates, lava can execute the reboot
template to reboot before running the next test.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This test plan template includes a variable to
specify which test must be executed (aum_test).
The test plan contains a predefined number of steps.
Each of those steps is executed after a reboot and is implemented in
the apertis update manager tests.
The AUM tests maintains a state in /var to know which state to run.
These tests cannot be interleaved because they all store their state
in the same location.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

This testcase is meant to test a custom libfolks feature we carried downstream
Aand that was proposed upstream and WONTFIX'ed in
https://bugzilla.gnome.org/show_bug.cgi?id=741643



We're thus considering that this testcase does not bring any value to the
project and is rather an hindrance since:
* the actual use-case for the feature is not really documented
* the implementation is problematic and has major architectural issues
* a proper solution would involve doing it in EDS rather than libfolks

No replacement is planned since contacts management via libfolks is not in the
current priorities.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Check the AppArmor features exposed by the kernel to ensure AppArmor D-Bus
mediation is supported.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Ritesh Raj Sarraf authored 6 years ago and Emanuele Aina committed 6 years ago

Since our rebase to Debian, we do not carry any additional patches
for our webkit2gtk package

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Ensure the fsck is a part of initramfs and started for root
filesystem consistency check.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Ensure that the root file system check is enabled on boot.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

uboot profiles is not specific to a board.
It is still referencing mx6qsabrelite.
The board is used in my tests to find the correct ostree branch
and there is not apertis/v2019dev0/armhf-mx6qsabrelite/minimal branch.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

The mx6sabrelite-specific images which shipped u-boot as part as the images
have been dropped in favour of the generic uboot images which rely on u-boot
being available on the SPI NOR memory: update the templates to match the
previously-disabled profiles and use the appropriate bootloader command for the
new setup.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Move the boot-no-crashes and boot-performance testcases in the standalone group
so they get run on OSTree images as well.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The development images have been dropped a few cycles ago, we no longer need to
carry profiles for those.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The telepathy-ring testcase is the manual one, we need to point to the
telepathy-ring-automated one to get the LAVA one.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>