All the release channels now use the same first stage based on the v2020
NFS artifacts rather than the old ones based on 18.12.

This means that tests are currently broken because LAVA fails to detect
the login prompt as the string changed.

Update the config using what is being used by v2021dev3.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Earlier, we were using an old kernel and rootfs from 18.09.xx.

We've now switched to using a much newer one from v2020, which is where
it requires an update of our assumpt login prompts

This fix is a cherry-pick of commit id: 7a5c59f4292dccc5e02416f792fd3d96bc3c1488
which has already been applied in apertis-test-cases repository against
v2021dev3

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

This reverts commit 48297f0f.
Need to disable OTA update tests for internal minimal armhf since their
behavior aren't stable and there is no value to test AUM on this platform.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Enable OTA tests for SabreLite minimal internal images

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Enable OTA tests for SabreLite minimal public images

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Added templates for testing OTA updates:
- API
- auto
- diskfull
- rollback/blacklist
- signed/unsigned update

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Some components, like Mildenhall ones, generate unrelated AppArmor
complaints.
This new parameter allows to pass a list, in `sed` format, to remove these
complaints.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Andrew Lee (李健秋) authored 5 years ago and Frederic Danis committed 5 years ago

Signed-off-by: Andrew Lee (李健秋) <ajqlee@debian.org>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Use specific prompts to match instead of just a single `#`.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This commit changes the prompt that LAVA needs to match in the images
during tests execution.

For SDK images the 'user@apertis:' prompt is the initial prompt after
login and 'root@apertis:' is used after becoming root.

And for the rest of the images '\$ ' is the initial prompt after login,
the '/home/user #' is used right after becoming root, and '/tmp #' is
used during test execution.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

With latest 5.4 kernel there's a message with a '# ' in it
(pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold) which is
detected by Lava as a prompt and triggers send of the username before login
prompt appeared.
Adding a space in front of # fixes it.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Apertis-tests for geoclue fails with the following error:
/usr/share/chaiwala-tests/geoclue/automated/run-test.sh: 260: cd: can't cd to /usr/share/chaiwala-tests/geoclue/automated/../../resources

In this case /usr/share/chaiwala-tests/geoclue/automated is a link to
../../../lib/apertis-tests/geoclue/automated.

Force 'cd' to resolve symbolic links to their values.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Denis Pynkin authored 5 years ago and Emanuele Aina committed 5 years ago

The board name is used in ostree-based images for pointing
the correct ostree branch name.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

This reverts commit d8db26c9 which was
introduced to see how things would work. Apparently we still have some road to
do before being able to properly turn on the tests on all the platforms
since they were all failing.

Among others, https://phabricator.apertis.org/T6446 and
https://phabricator.apertis.org/T6444

 have been identified during this
experiment.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This flag is no longer available for the timeout subcommand, making the
dbus test to fail, so remove it.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Use correct boot commands for Amd64 since they are used
in all AUM tests during reboot.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Enable tests for ostree and Apertis Update Manager for all supported
architectures.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add testing of signature verification of commits in offline updates.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Recent ARM internal jobs fails with a timeout on unpacking the overlay
into the image.

This commit increases the default timeout to the action transfering the
overlay in the LAVA job to 5 minutes for all jobs types.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

This test case has been failing for a long a time since the required
telepathy packages are missing from the images.

The test case is also not so relevant anymore and makes current SDK lava
tests jobs to fail, so this commit removes the test case.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Except for the SDK image, this test is failing in the remaining images.

This test is also not relevant anymore, so this commit removes it along
with the group-folks template file, so LAVA won't deploy and boot an
image in order to execute just one single test.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

All telepathy folks tests currently fail to execute because the
telepathy-gabble package is not available in the images.

Support for folks and telepathy libs also needs to be revised, so
this commit disables all the folks telepathy tests in the meantime.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Sysprof moved to a binary capture log, so the current machinery parsing
the old XML-based format no longer works.

Since this is a smoke test, just check that the capture file exist and is
not empty.

This also avoid printing large amount of binary contents to the
LAVA log, which makes the log unviewable from the UI and may cause
other issues.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Update the `run-aa-test` script to use the new shell/awk based
`aa_log_extract_tokens.sh` implementation instead of the old Perl-based one.

This affects the follwing testcases:
* apparmor-geoclue
* apparmor-ofono
* apparmor-tracker

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The actual type for "should-be-denied-but-in-complain-mode-so-go-ahead" audit
messages is `ALLOWED`, not `PERMIT`.

See https://github.com/torvalds/linux/blob/v4.19/security/apparmor/audit.c#L32



Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This commit checks for any failed systemd units in the sanity-check
test.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

These tests are not meant to be run on minimal images:
* apparmor-gstreamer1-0
* apparmor-pulseaudio
* apparmor-tumbler
* apparmor-tracker

Move them out of the minimal group to the common one.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit da75fe5a "apparmor: Drop bluez/, now shipped by tests/apparmor-bluez"
broke the debian package build by removing the `apparmor/bluez` folder but not
dropping the debian package which used to ship its contents:

    dh_install --fail-missing
    dh_install: Please use dh_missing --list-missing/--fail-missing instead
    dh_install: This feature will be removed in compat 12.
    dh_install: Cannot find (any matches for) "apparmor/bluez/*.expected" (tried in ., debian/tmp)

    dh_install: apertis-tests-apparmor-bluez missing files: apparmor/bluez/*.expected
    dh_install: Cannot find (any matches for) "apparmor/bluez/*.sh" (tried in ., debian/tmp)

    dh_install: apertis-tests-apparmor-bluez missing files: apparmor/bluez/*.sh
    dh_install: Cannot find (any matches for) "apparmor/bluez/test-bluez" (tried in ., debian/tmp)

    dh_install: apertis-tests-apparmor-bluez missing files: apparmor/bluez/test-bluez
    dh_install: missing files, aborting
    make[1]: *** [debian/rules:13: override_dh_install] Error 25

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Commit bbf36571 "apparmor: Drop automated/, now shipped by
tests/apparmor-basic-profiles" broke `make install` and thus the debian
package build:

    dh_auto_install --destdir=debian/tmp
    	make -j1 install DESTDIR=/tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp AM_UPDATE_INFO_DIR=no
    make[2]: Entering directory '/tmp/build-snapshot.x7qrtWRuJX/s'
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp -a apparmor/apparmor-profile-sanity-check.sh /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp -a apparmor/apparmor-profile-sanity-check.yaml /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/
    cp -a apertis_tests_lib /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/
    install -d /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp -a apparmor/automated /tmp/build-snapshot.x7qrtWRuJX/s/debian/tmp/usr/lib/apertis-tests/apparmor/
    cp: cannot stat 'apparmor/automated': No such file or directory
    make[2]: *** [Makefile:61: install-apparmor/automated] Error 1

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

AppArmor info are accessible through jounalctl.
Replace the aa_log_extract_tokens.pl by shell version, removing
dependencies on auditd and libapparmor-perl packages.

Fixes: APERTIS-6233

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>

Point the test repositories from the job templates to the new branch
apertis/v2020dev0.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Frédéric Dalleau authored 5 years ago and Emanuele Aina committed 5 years ago

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>