Use specific prompts to match instead of just a single `#`.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This commit changes the prompt that LAVA needs to match in the images
during tests execution.

For SDK images the 'user@apertis:' prompt is the initial prompt after
login and 'root@apertis:' is used after becoming root.

And for the rest of the images '\$ ' is the initial prompt after login,
the '/home/user #' is used right after becoming root, and '/tmp #' is
used during test execution.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

With latest 5.4 kernel there's a message with a '# ' in it
(pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold) which is
detected by Lava as a prompt and triggers send of the username before login
prompt appeared.
Adding a space in front of # fixes it.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Apertis-tests for geoclue fails with the following error:
/usr/share/chaiwala-tests/geoclue/automated/run-test.sh: 260: cd: can't cd to /usr/share/chaiwala-tests/geoclue/automated/../../resources

In this case /usr/share/chaiwala-tests/geoclue/automated is a link to
../../../lib/apertis-tests/geoclue/automated.

Force 'cd' to resolve symbolic links to their values.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Denis Pynkin authored 5 years ago and Emanuele Aina committed 5 years ago

The board name is used in ostree-based images for pointing
the correct ostree branch name.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This reverts commit d8db26c9 which was
introduced to see how things would work. Apparently we still have some road to
do before being able to properly turn on the tests on all the platforms
since they were all failing.

Among others, https://phabricator.apertis.org/T6446 and
https://phabricator.apertis.org/T6444

 have been identified during this
experiment.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This flag is no longer available for the timeout subcommand, making the
dbus test to fail, so remove it.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Use correct boot commands for Amd64 since they are used
in all AUM tests during reboot.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Enable tests for ostree and Apertis Update Manager for all supported
architectures.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add testing of signature verification of commits in offline updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Recent ARM internal jobs fails with a timeout on unpacking the overlay
into the image.

This commit increases the default timeout to the action transfering the
overlay in the LAVA job to 5 minutes for all jobs types.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This test case has been failing for a long a time since the required
telepathy packages are missing from the images.

The test case is also not so relevant anymore and makes current SDK lava
tests jobs to fail, so this commit removes the test case.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Except for the SDK image, this test is failing in the remaining images.

This test is also not relevant anymore, so this commit removes it along
with the group-folks template file, so LAVA won't deploy and boot an
image in order to execute just one single test.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

All telepathy folks tests currently fail to execute because the
telepathy-gabble package is not available in the images.

Support for folks and telepathy libs also needs to be revised, so
this commit disables all the folks telepathy tests in the meantime.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Sysprof moved to a binary capture log, so the current machinery parsing
the old XML-based format no longer works.

Since this is a smoke test, just check that the capture file exist and is
not empty.

This also avoid printing large amount of binary contents to the
LAVA log, which makes the log unviewable from the UI and may cause
other issues.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Update the `run-aa-test` script to use the new shell/awk based
`aa_log_extract_tokens.sh` implementation instead of the old Perl-based one.

This affects the follwing testcases:
* apparmor-geoclue
* apparmor-ofono
* apparmor-tracker

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The actual type for "should-be-denied-but-in-complain-mode-so-go-ahead" audit
messages is `ALLOWED`, not `PERMIT`.

See https://github.com/torvalds/linux/blob/v4.19/security/apparmor/audit.c#L32



Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This commit checks for any failed systemd units in the sanity-check
test.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

These tests are not meant to be run on minimal images:
* apparmor-gstreamer1-0
* apparmor-pulseaudio
* apparmor-tumbler
* apparmor-tracker

Move them out of the minimal group to the common one.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit da75fe5a "apparmor: Drop bluez/, now shipped by tests/apparmor-bluez"
broke the debian package build by removing the `apparmor/bluez` folder but not
dropping the debian package which used to ship its contents:

    dh_install --fail-missing
    dh_install: Please use dh_missing --list-missing/--fail-missing instead
    dh_install: This feature will be removed in compat 12.
    dh_install: Cannot find (any matches for) "apparmor/bluez/*.expected" (tried in ., debian/tmp)

    dh_install: apertis-tests-apparmor-bluez missing files: apparmor/bluez/*.expected
    dh_install: Cannot find (any matches for) "apparmor/bluez/*.sh" (tried in ., debian/tmp)

    dh_install: apertis-tests-apparmor-bluez missing files: apparmor/bluez/*.sh
    dh_install: Cannot find (any matches for) "apparmor/bluez/test-bluez" (tried in ., debian/tmp)

    dh_install: apertis-tests-apparmor-bluez missing files: apparmor/bluez/test-bluez
    dh_install: missing files, aborting
    make[1]: *** [debian/rules:13: override_dh_install] Error 25

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit bbf36571 "apparmor: Drop automated/, now shipped by
tests/apparmor-basic-profiles" broke `make install` and thus the debian
package build:

    dh_auto_install --destdir=debian/tmp
    	make -j1 install DESTDIR=/tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp AM_UPDATE_INFO_DIR=no
    make[2]: Entering directory '/tmp/build-snapshot.x7qrtWRuJX/s'
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp -a apparmor/apparmor-profile-sanity-check.sh /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp -a apparmor/apparmor-profile-sanity-check.yaml /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/
    cp -a apertis_tests_lib /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp -a apparmor/automated /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp: cannot stat 'apparmor/automated': No such file or directory
    make[2]: *** [Makefile:61: install-apparmor/automated] Error 1

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

AppArmor info are accessible through jounalctl.
Replace the aa_log_extract_tokens.pl by shell version, removing
dependencies on auditd and libapparmor-perl packages.

Fixes: APERTIS-6233

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Point the test repositories from the job templates to the new branch
apertis/v2020dev0.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Frédéric Dalleau authored 5 years ago and Emanuele Aina committed 5 years ago

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

The image type is used in the QA report to distinguish the
test results.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Emanuele Aina authored 5 years ago and Peter Senna Tschudin committed 5 years ago

Add a testcase to ensure that the sample image recipes can be built on the SDK
and that the commands in the documentation work correctly.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Having a standard status line at the end of the test is extremely valuable
for manual runs, as it provides a clear, unambiguous, consistent indicator
to the tester.

Add it to `run-test-in-systemd` so it can be used to wrap manual tests just
like it wraps automated tests in LAVA.

 foobar: test failed (exit=1)


 foobar: test succeeded

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The nfsroot nfsuser has been renamed user.
The nfsroot apertis-nfs machine name changed to apertis.
Only the group-nfs-tpl.yaml test changed because other
tests are still using a 18.09 release.
When we update lava, it will be time to update other tests.

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Currently the lava artefacts are built separately than the Apertis
standard images. The image testing profiles can't be used because
the lava artefacts may not be built already when the image tests
are submitted.
Add specific profiles for the lava artefacts tests.
Create one lava profile per arch which is based on the
apertis-minimal-<arch>-<board>

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The current code fails because it tries to check out the sample app code twice
in the same place, and git complains.

Ensure that the repository is deleted between runs to avoid the issue.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Rely on `set -e` to make the scripts immediately exit on failure and on
`run-test-in-systemd` as called by the YAML testcase to signal failures to LAVA
appropriately.

This simplifies the code and also avoids inconsistencies between the testcase
names used inside the scripts and by `run-test-in-systemd`.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 5 years ago and Frédéric Dalleau committed 5 years ago

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 6 years ago and Frédéric Dalleau committed 5 years ago

The tests/apparmor-bluez repository now ships the contents of the
`bluez/` folder, drop the obsolete duplicate.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 6 years ago and Frédéric Dalleau committed 5 years ago

The tests/apparmor-basic-profiles repository now ships the contents of the
`automated/` folder, drop the obsolete duplicate.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>