Commits · 72eb8ac08615d7769e5f94e1c02d169f8a7974d3 · Frédéric Dalleau / apertis-tests

Sep 09, 2016

sanity-check: make this more usable as a manual test · 72eb8ac0

Simon McVittie authored 8 years ago


Previously, this test didn't provide an obvious indication that it
had all succeeded, and assumed that it was run as root with $(pwd) in
/usr/lib/apertis-tests or a git clone of apertis-tests.

To use this as a manual test:

    Prerequisites:
        - apertis-tests installed (sudo apt install apertis-tests)
    Steps:
        sudo /usr/lib/apertis-tests/common/sanity-check
    Successful result:
        Output ends with "# Sanity check successful"
    Failing result:
        Output ends with "Failed, see above for details" or ends with
        neither of the expected messages
    Diagnosing failures:
        Look for "TEST_RESULT:fail:" in the output (it may appear more
        than once), and look for warnings/errors shortly before each
        occurrence of that message

Reviewed-by: Luis Araujo <luis.araujo@collabora.co.uk>
Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D4284

72eb8ac0

Sep 08, 2016

sanity-check: stop allowing os-release ID to be ubuntu · 19ab43ef

Héctor Orón Martínez authored 8 years ago


os-release pretended we were Ubuntu for a while, to make packages
with different build steps for Debian and Ubuntu follow Ubuntu-like
code paths. This has now been fixed.

Reviewed-by: Simon McVittie <smcv@collabora.co.uk>
Signed-off-by: Héctor Orón Martínez <hector.oron@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D4272

19ab43ef

Aug 31, 2016

common: Double the timeout for common/telepathy-gabble.yaml again · e574e80c

Philip Withnall authored 8 years ago


It seems that importing pyopenssl now takes longer than it did before,
which affects each iteration of the telepathy-gabble tests, and makes
them take a little more than the 30 minute timeout on an i.MX6.

Double the timeout again to prevent test failures even when the tests
are making progress.

Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Reviewed-by: Luis Araujo <luis.araujo@collabora.com>
Differential Revision: https://phabricator.apertis.org/D4122

e574e80c

Jul 20, 2016

firewall: run test from source directory · c0b7dd2e

Simon McVittie authored 8 years ago


As with many previous commits, there's no point in installing this
simple script; running it from the git directory gives us faster
fix/test cycles.

Reviewed-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3766

c0b7dd2e

Jul 19, 2016

common: dereference $dbus_socket correctly · f30285e8

Simon McVittie authored 8 years ago

Regression in e562c672 (D3663).

Bug: https://phabricator.apertis.org/T2215



Reviewed-by: Luis Araujo <luis.araujo@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3764

f30285e8

Jul 08, 2016

common: Only use an existing session bus · e562c672

Sjoerd Simons authored 8 years ago

As soon as there is a dbus user session a session bus should be
available. Only verify that this exists (failling if it doesn't) rather
then trying to workaround things by autolaunching a new bus.

Reviewed-by: Luis Araujo <luis.araujo@collabora.co.uk>
Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3663

e562c672

sanity-check: show what we're doing before we do it · 592f87a8
Simon McVittie authored 8 years ago and Sjoerd Simons committed 8 years ago
```
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3651
```
592f87a8

sanity-check: assert that /etc/os-release NAME is correct · f89d2045

Simon McVittie authored 8 years ago and

Sjoerd Simons committed 8 years ago

NAME was fixed (in T133). Unfortunately, ID wasn't fixed (opens: T2117).

Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3650

f89d2045

sanity-check: make user-id run id as uid user, as was intended · f2b514bc

Simon McVittie authored 8 years ago and

Sjoerd Simons committed 8 years ago


Otherwise it's just a pointless duplicate of system-id.

Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3649

f2b514bc

sanity-check: print a process tree in terms of both ppid and cgroup · e7ae602b

Simon McVittie authored 8 years ago and

Sjoerd Simons committed 8 years ago


This should help to debug situations where a process has been
started the wrong way, for example in T1984.

Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3648

e7ae602b

sanity-check: split out into a separate shell script · 57242f89

Simon McVittie authored 8 years ago and

Sjoerd Simons committed 8 years ago


This makes it easier to run outside LAVA.

No functional changes intended, even where the checks I'm copying
are wrong (which will be fixed in subsequent commits).

Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3647

57242f89

Jun 30, 2016

common: Add a test to check whether any processes crashed on boot · 07683a3b

Philip Withnall authored 8 years ago

This should hopefully catch failures like T2040, where the launcher
crashed on boot. It checks for ANOM_ABEND lines in the audit log.

Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>

Differential Revision: https://phabricator.apertis.org/D3540

07683a3b

Jun 24, 2016

tumbler: run the test from the source tree · 6d57a062

Simon McVittie authored 8 years ago

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3476

6d57a062

Jun 22, 2016

Run x-out-of-screen-events-buttond test only in SDK image · bec747e1

Luis Araujo authored 8 years ago


As this test is not valid anymore for Wayland images, just move
it to be executed in SDK images.

Reviewed-by: Héctor Orón Martínez <hector.oron@collabora.co.uk>
Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3377

bec747e1

Mar 04, 2016

cgroups: run sub-tests separately, for better visibility in LAVA · 1b257577

Simon McVittie authored 9 years ago

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2163

1b257577

Move cgroups automated test from chaiwala-tests to apertis-tests{,-bin} · 73fbe326

Simon McVittie authored 9 years ago


Leave a symlink behind for compatibility.

The test still has a dependency on chaiwala-tests for now, so that it
can be run against older versions of apertis-tests.

While I'm editing dependencies, add python, since the test runs it
explicitly.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2162

73fbe326

cgroups-resource-control: run the test from git, not from chaiwala-tests · 97174691

Simon McVittie authored 9 years ago


This makes it much easier to try new versions.

Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2161

97174691

Mar 03, 2016

Remove all references to chaiwala-btrfs-tests · 6f6e6cb3

Mathieu Duponchelle authored 9 years ago and

Sjoerd Simons committed 9 years ago


We no longer ship chaiwala-btrfs.

Reviewed-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D2158

6f6e6cb3

grilo: Add python3 dependency · d90720fe

Xavier Claessens authored 9 years ago and

Sjoerd Simons committed 9 years ago


Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D962

d90720fe

Replace tracker-utils package dependency by package tracker · d4110657

Luis Araujo authored 9 years ago and

Sjoerd Simons committed 9 years ago

The tracker-utils package is now merged into the tracker package. This
commit updates the tracker-indexing-local-storage test definition
to use the new package name.

d4110657

grilo: remove old bash tests · 18b8b188

Xavier Claessens authored 9 years ago and

Sjoerd Simons committed 9 years ago

Differential Revision: https://phabricator.apertis.org/D536


Reviewed-by: Philip Withnall <philip.withnall@collabora.co.uk>

18b8b188

common.sh: remove unused code · 9408bdf8
Xavier Claessens authored 9 years ago and Sjoerd Simons committed 9 years ago
```
Differential Revision: https://phabricator.apertis.org/D533
```
9408bdf8
tumbler: inline a common sh file used only from one place · 38f5f68c
Xavier Claessens authored 9 years ago and Sjoerd Simons committed 9 years ago
```
Differential Revision: https://phabricator.apertis.org/D532
```
38f5f68c

common: Double the timeout for common/telepathy-gabble.yaml · 9cbc04a0

Philip Withnall authored 9 years ago and

Sjoerd Simons committed 9 years ago

On an i.MX6, this test can take around 15 minutes under normal
conditions, so any network delays are going to tip it over the edge and
cause test failures due to timeouts.

Double the timeout to 30 minutes to hopefully avoid this.

Bug: https://bugs.apertis.org/show_bug.cgi?id=476


Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Reviewed-by: Luis Araujo <luis.araujo@collabora.com>
Differential Revision: https://phabricator.apertis.org/D429

9cbc04a0

tracker tests: require python3 and python3-gi · af270a1b
Simon McVittie authored 9 years ago and Sjoerd Simons committed 9 years ago
```
Differential Revision: https://phabricator.apertis.org/D411
Reviewed-by: xclaesse
```
af270a1b

run-test-in-systemd: stop emitting STARTTC, ENDTC · 2064d89f

Simon McVittie authored 9 years ago and

Sjoerd Simons committed 9 years ago

`lava-test-case --shell` waits for LAVA_ACK from the dispatcher after
each of these[1], but run-test-in-systemd does not. This is a point of
incompatibility that could potentially cause or exacerbate issues in
LAVA's "screen-scraping" on the serial console, such as
<https://bugs.apertis.org/show_bug.cgi?id=396>.

[1] https://git.linaro.org/lava/lava-dispatcher.git/blob/HEAD:/lava_dispatcher/lava_test_shell/lava-test-case

Differential Revision: https://phabricator.apertis.org/D388
Reviewed-by: araujo

2064d89f

tracker: Fix dependencies · 31daa8f9
Xavier Claessens authored 9 years ago and Sjoerd Simons committed 9 years ago

31daa8f9

tracker: Rewrite tests in python · 67bc0739

Xavier Claessens authored 9 years ago and

Sjoerd Simons committed 9 years ago

Summary:
This is mostly a 1-1 translation. Tests are not smarter than
what they were before, to the exception that they don't include
sleep() of arbitrary duration everywhere.

Reviewers: smcv, pwith

Differential Revision: https://phabricator.apertis.org/D377

67bc0739

tracker: Use the new python tests in LAVA · ca96cc64
Xavier Claessens authored 9 years ago and Sjoerd Simons committed 9 years ago

ca96cc64

run-test-in-systemd: watch output in real-time · 20e93304

Simon McVittie authored 9 years ago and

Sjoerd Simons committed 9 years ago

This is good for usability, and might also prevent it from detecting
inactivity and killing our tests, which upstream developer Neil Williams
warned me might happen (although I haven't seen it on our installation).

Differential Revision: https://phabricator.apertis.org/D372
Reviewed-by: pwith

20e93304

Move D-Bus tests from common/ to new dbus directory · ff2b58bb

Simon McVittie authored 9 years ago and

Sjoerd Simons committed 9 years ago

D-Bus is a fairly major piece of Apertis infrastructure, so it seems
worth splitting out.

Also rename dbus.yaml (the upstream installed-tests) to
installed-tests.yaml, in preparation for adding other D-Bus tests.

Differential Revision: https://phabricator.apertis.org/D358
Reviewed-by: pwith

ff2b58bb

In tests that use run-aa-test, add a prefix to test names · eb1578f0

Simon McVittie authored 9 years ago and

Sjoerd Simons committed 9 years ago

In many tests, we run a scenario twice, once with a fake "malicious"
LD_PRELOAD and one without. Prefix the tests so we get

normal_test1: pass
normal_test2: pass
normal.expected_underlying_tests: pass
normal.expected: pass
malicious_test1: fail
malicious_test2: pass
malicious.expected_underlying_tests: fail
malicious.expected: pass

instead of having "duplicate" results for the underlying tests:

test1: pass
test2: pass
normal.expected_underlying_tests: pass
normal.expected: pass
test1: fail
test2: pass
malicious.expected_underlying_tests: fail
malicious.expected: pass

Differential Revision: https://phabricator.apertis.org/D282


Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Reviewed-by: xclaesse

eb1578f0

Initial commit · e6c93c2b
Sjoerd Simons authored 9 years ago

e6c93c2b