This should hopefully catch failures like T2040, where the launcher
crashed on boot. It checks for ANOM_ABEND lines in the audit log.

Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>

Differential Revision: https://phabricator.apertis.org/D3540

This avoids mistakes.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3490

This was installed but not packaged.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3489

This lets us run it from /usr/lib/apertis-tests/tumbler if desired.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3488

This test is run from the source tree, so there's no guarantee
that this query will return anything.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3487

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3480

This typically makes logs from automated tests more useful.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3479

Debian Policy §10.4 says "Every script should use set -e or check
the exit status of every command", which is just generally good
advice.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3478

We need the compiled LD_PRELOAD hack from the installed tree, but
the rest can come from the source, allowing for quicker test/fix
cycles.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3477

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3476

If we have apertis-tests checked out in ~/apertis-tests or something,
it's nice to be able to run the test uninstalled.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3475

We use systemd-run to schedule the pactl process to be run under
a vaguely realistic user-session. However, there's a chicken-and-egg
problem here: systemd-run uses either D-Bus or a private socket in
XDG_RUNTIME_DIR to communicate with systemd, and without setting some
environment variables we can't know either of those.

This is similar to the implementation of the same concept in
common/run-test-in-systemd. Unfortunately, the AppArmor tests
need to reinvent that bit, because they run as root (to be able to
manipulate AppArmor, which is a highly privileged action).

Bug-Apertis: https://phabricator.apertis.org/T1859



Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3449

The list of profiles and processes isn't all that long. If we're
going to make assertions about this information, we should
probably show it first.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3448

We were running commands like "pactl stat" and then ignoring their
nonzero exit status.

I've included support for ignoring failures, but in fact we never
actually run anything in this test that can legitimately fail, so
it's unused.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3447

As this test is not valid anymore for Wayland images, just move
it to be executed in SDK images.

Reviewed-by: Héctor Orón Martínez <hector.oron@collabora.co.uk>
Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3377

The bundle_stream variable will allow to easily set the
bundle stream from the profile file for the different
job templates.

This commit also sets the value of such a variable to
the new stream: /public/personal/apertis-validator/

Reviewed-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2684

Reviewed-by: Luis Araujo <luis.araujo@collabora.co.uk>
Signed-off-by: Héctor Orón Martínez <hector.oron@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2625

When raising an exception, it’s generally a good idea to actually
include the exception value — otherwise Python will re-raise the ‘last
exception which was active in the current scope’, which is not so
useful, especially when you want to print that exception.

Reviewed-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D1296

This introduces no behavioural changes.

Reviewed-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2310

Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>

The regexp is not bound to either end of the process name, so despite
the fact that the test script changed its effective process name to
‘ofonod_’, the regexp ‘ofonod’ still matches that, and hence the script
was killing itself, which was causing the systemd unit it was running as
to fail, and hence the overall test to fail.

Tighten the pkill regexp to match at the end of the process name to
avoid this.

Bug-Apertis: https://bugs.apertis.org/show_bug.cgi?id=681



Reviewed-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2284

If an AppArmor malicious test is run as a systemd system job (using
`run-test-in-systemd --system`), $HOME will explicitly not be set, which
results in the program trying to read (null)/.bash_history, rather than
the expected /home/user/.bash_history.

Fix that by hard-coding it to use /home/user/.bash_history if $HOME is
not set. If the username changes in future, the tests should start
failing, which will allow us to update it again.

Bug-Apertis: https://bugs.apertis.org/show_bug.cgi?id=681



Reviewed-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2283

This matches the command used in ofono.service (minus the --nodetach,
because we want the process to detach so we can do other things in the
test). This eliminates some spurious AppArmor failures caused by the RIL
code, and allows ofono to work (the RIL plugin does not work on the
i.MX6).

Bug-Apertis: https://bugs.apertis.org/show_bug.cgi?id=681



Reviewed-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2278

As explained in the new comment in the Tumbler mixin, it is possible for
the Error and Ready signals from tumblerd to race if one thumbnailer
fails fast while another succeeds (for the same URI) slowly. This
happened for me with the cover and gst thumbnailers on
big_buck_bunny_smaller.ogv — the cover thumbnailer failed fast because
the search terms ‘big buck bunny smaller’ return nothing on omdbapi.com,
while the gst thumbnailer eventually succeeded by grabbing a frame from
the video.

Reviewed-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D1299

We can't to anything with bluetooth until its enabled via connman
(otherwise everything else will blocked/soft rfkilled).

Enable it explicitely in the agent so the user doesn't have to first use
connmanctl.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2209

object_set_properties sets the properties one by one, but the bluez
interface doesn't allow setting Discoverable or Pairable unless the
adaptor is powered. So before setting those, power it on first.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2208

The simple-agent command line arguments were a bit weird, it can just
act as an agent by giving no arugments or the adaptor to use. However to
pair a specific device a adaptor was mandatory (which is pointless on
most systems with just one adaptor). And for extra fun adding a third
argument which (could be whatever).

Sanitize things by making the device an optional argument rather then a
positional one. Drop the recgonition of the third positional argument
(bluetoothctl can be used instead to remove pairings).

On top of that switch to argparse (The recommended argument parser in
python 3), fix outgoing pairings (with bluez5 you can only trigger a
pairing with devices found during scanning it seems) and add a --yes
argument to accept all incoming connection making it more convenient to
use in testing.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2207

This variable can be used to specify a base url for the
images url link, which will allow to set http credentials
for those profile requiring so.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Set the main adaptor to be discoverable when starting the simple agent
for incoming pairing requests.

Also set the agent to be the default to prevent an already running
system agent interfering with the test.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2203

Sometimes just setting the adaptor to be pairable and/or discoverable
as a one-shot action without change notifications is enough. Support
this.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2202

Add a symlink in the python 3 system path to maek apertis_tests_lib
available to the system.

Note that this is a somewhat ugly solution. To solve it properly the
build system needs to be a bit revamped to install the python libraries
(and really binaries) in the right spots in the system.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2201

Philip Withnall authored 9 years ago and Sjoerd Simons committed 9 years ago

This is effectively a complete rewrite of the tests. Some of the common
functionality has been split out to apertis_tests_lib. The external
interfaces and behaviour of the tests should have remained unchanged.

The tests needed to change considerably in structure because of the
difference between how BlueZ 4 and BlueZ 5 handle device objects. BlueZ
4 only creates a device object if an adapter is paired with that device.
BlueZ 5 creates it as soon as a device is discovered.

Apart from that, the main differences between the two APIs are in the
shift to standard D-Bus interfaces (Properties and ObjectManager) in
BlueZ 5.

While rewriting the tests, a lot of multi-second sleep() calls have been
removed in favour of waiting for the relevant completion signals from
the D-Bus API. This means some potentially unbounded waits if the
Bluetooth daemon misbehaves or ends up in a state the unit test does not
expect. But otherwise it means faster test execution and no failures due
to races between the sleep() and the Bluetooth daemon completing an
operation.

[ Small updates for review  comments ]

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D971

Reviewed-by: Mathieu Duponchelle <mathieu.duponchelle@opencreed.com>
Signed-off-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2194

Of the schedulers we compile, only cfq supports the block I/O weighting
feature that is exercised by test_blkio_weights_random_read and
test_blkio_weights_sequential_read.

Related to <https://bugs.apertis.org/show_bug.cgi?id=635

>

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2165

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2164

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2163

Leave a symlink behind for compatibility.

The test still has a dependency on chaiwala-tests for now, so that it
can be run against older versions of apertis-tests.

While I'm editing dependencies, add python, since the test runs it
explicitly.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2162

This makes it much easier to try new versions.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2161