tests_: Add integration tests for the entire system

Add a basic integration test which runs as a client on the system, as
installed. It checks basic functionality against the mock backends.

The integration test is written in Python, and exercises some of the
Python bindings for librhosydd. It is executed by autopkgtest, and is
intended to be executed as an installed-test in future, although this is
not currently the case due to it needing to be run as root in order to
get authentication to access the daemon from polkit.

Reviewed-by: Simon McVittie <simon.mcvittie@collabora.co.uk>
Signed-off-by: Philip Withnall <philip.withnall@collabora.co.uk>
Differential Revision: https://phabricator.apertis.org/D3145

Makefile.am

 # DIST_SUBDIRS needs to be specified with the root directory last so that
 # maintainer-clean does not delete build-aux/* before recursing, since it is
 # needed for any automake invocation.
-SUBDIRS = . librhosydd/tests libcroesor/tests po
-DIST_SUBDIRS = librhosydd/tests libcroesor/tests po
+SUBDIRS = . librhosydd/tests libcroesor/tests tests po
+DIST_SUBDIRS = librhosydd/tests libcroesor/tests tests po
 
 ACLOCAL_AMFLAGS = -I m4
 

configure.ac

@@ -191,6 +191,7 @@ librhosydd/librhosydd-$RSD_API_VERSION.pc:librhosydd/librhosydd.pc.in
 librhosydd/version.h
 librhosydd/tests/Makefile
 po/Makefile.in
+tests/Makefile
 ],[],
 [RSD_API_VERSION='$RSD_API_VERSION';CSR_API_VERSION='$CSR_API_VERSION'])
 AC_OUTPUT

debian/control

@@ -199,6 +199,20 @@ Description: Sensors and actuators service - mock backends
  .
  This package contains mock backends for integration testing only.
 
+Package: rhosydd-tests
+Section: misc
+Architecture: any
+Depends:
+ gir1.2-rhosydd-0,
+ python3-gi,
+ rhosydd,
+ ${misc:Depends},
+ ${shlibs:Depends},
+Description: Sensors and actuators service - tests
+ Rhosydd mediates access to vehicle sensors and actuators.
+ .
+ This package contains integration tests.
+
 Package: rhosydd-tools
 Section: misc
 Architecture: any

debian/rhosydd-tests.install

+usr/lib/*/installed-tests/rhosydd-0

debian/tests/control

@@ -18,3 +18,9 @@ Depends:
 Tests: rhosydd-client
 Depends:
  rhosydd-tools,
+
+Tests: integration
+Depends:
+ dpkg-dev,
+ python3,
+ rhosydd-tests,

debian/tests/integration

+#!/bin/sh
+
+# Smoke-test for rhosydd-client: can it connect to the running daemon?
+
+# Copyright © 2016 Collabora Ltd.
+#
+# SPDX-License-Identifier: MPL-2.0
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+set -e
+
+exec 2>&1
+
+case "$(id -n -u)" in
+    (root)
+        as_root=
+        ;;
+
+    (user)
+        if ! sudo --non-interactive --validate; then
+            echo "SKIP: cannot use sudo without password here"
+            exit 0
+        fi
+        as_root="sudo --non-interactive"
+        ;;
+
+    (*)
+        echo "SKIP: neither root nor user; is this really Apertis?"
+        exit 0
+        ;;
+esac
+
+$as_root systemctl status rhosydd.service || :
+
+# Use sudo to get authorisation. This will probably end up listing zero
+# vehicles, as no backends are running. But it should at least do so
+# successfully.
+$as_root /usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/installed-tests/rhosydd-0/integration.py
+
+$as_root  systemctl status rhosydd.service

tests/Makefile.am

+include $(top_srcdir)/glib-tap.mk
+
+installed_testdir = $(libexecdir)/installed-tests/rhosydd-@RSD_API_VERSION@
+installed_test_metadir = $(datadir)/installed-tests/rhosydd-@RSD_API_VERSION@
+
+# This is an ‘extra’ script for now, as it can’t be run as a test case because
+# it requires root (and also doesn’t support TAP output yet). However, it can
+# be run manually, and can be run by autopkgtest. See debian/tests/control.
+dist_installed_test_extra_scripts = \
+	integration.py \
+	$(NULL)
+
+-include $(top_srcdir)/git.mk

tests/integration.py

+#!/usr/bin/python3
+# -*- coding: utf-8 -*-
+
+# Copyright © 2016 Collabora Ltd.
+#
+# SPDX-License-Identifier: MPL-2.0
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+"""Integration tests for the daemon and mock backends."""
+
+import gi
+from gi.repository import (GLib, Gio)
+
+gi.require_version('Rhosydd', '0')
+from gi.repository import Rhosydd  # pylint: disable=no-name-in-module
+
+import unittest
+import xml.etree.ElementTree as ElementTree
+
+
+class TestWithMockBackend(unittest.TestCase):
+    """Integration test using the mock backends.
+
+    It relies on service activation for the daemon and mock backends
+    (rhosydd-mock-backend.service and rhosydd-speedo-backend.service) to be
+    started, which means that the mock backends must be installed.
+
+    It can only be run when installed, due to requiring all the polkit
+    policy files, systemd unit files, and D-Bus service files to be in place
+    for the system bus.
+
+    It currently needs to be run as root in order for all the polkit
+    authorisation checks to succeed, as no .rules file is installed to
+    give it authorisation.
+    """
+
+    def setUp(self):
+        self.conn = Gio.bus_get_sync(Gio.BusType.SYSTEM, None)
+        self.vehicle_manager = None
+        self.main_context = GLib.main_context_default()
+        self.__result = None
+
+        Rhosydd.VehicleManager.new_async(connection=self.conn,
+                                         cancellable=None,
+                                         callback=self._block_on_result_cb,
+                                         user_data=self)
+        self.vehicle_manager = \
+            Rhosydd.VehicleManager.new_finish(self._block_on_result())
+
+    def tearDown(self):
+        # Check there are no more sources pending dispatch.
+        self.assertIsNone(self.__result)
+        self.assertFalse(self.main_context.iteration(False))
+        self.main_context = None
+
+        self.vehicle_manager = None
+        self.conn = None
+
+    def _block_on_result(self):
+        """Block on receiving an async result.
+
+        Pass callback=self._block_on_result_cb to the asynchronous function,
+        with user_data=self, then call this function in order to block on
+        retrieving the Gio.AsyncResult from an asynchronous operation, which
+        will be returned. You can then call the appropriate finish() function
+        on it.
+        """
+        while self.__result is None:
+            self.main_context.iteration(True)
+
+        result = self.__result
+        self.__result = None
+
+        return result
+
+    def _block_on_result_cb(self, source_object, result, user_data):
+        """Callback to use with _block_on_result()."""
+        self.__result = result
+
+    def test_list_vehicles(self):
+        """Test that the mock backend’s vehicles are listed correctly."""
+        vehicles = self.vehicle_manager.list_vehicles()
+
+        ids = [v.get_id() for v in vehicles]
+        self.assertCountEqual(ids, ['mock', 'speedo'])
+
+    def __get_mock_vehicle(self):
+        """Helper function to get the ‘mock’ vehicle, or None."""
+        vehicles = self.vehicle_manager.list_vehicles()
+
+        for vehicle in vehicles:
+            if vehicle.get_id() == 'mock':
+                return vehicle
+
+        return None
+
+    def test_vehicle_zones(self):
+        """Test that the mock backend’s only vehicle’s zones are listed."""
+        mock_vehicle = self.__get_mock_vehicle()
+
+        zones = mock_vehicle.get_zones()
+        self.assertEqual(len(zones), 1)
+        root_zone = zones[0]
+
+        self.assertEqual(root_zone.get_path(), '/')
+        self.assertEqual(root_zone.get_parent_path(), '')
+        self.assertEqual(root_zone.get_tags(), [])
+
+    def test_vehicle_all_metadata(self):
+        """Test getting all metadata from a vehicle."""
+        mock_vehicle = self.__get_mock_vehicle()
+        root_zone = Rhosydd.StaticZone.new('/')
+
+        mock_vehicle.get_all_metadata_async(
+            zone=root_zone,
+            callback=self._block_on_result_cb,
+            user_data=self)
+        metadata = \
+            mock_vehicle.get_all_metadata_finish(self._block_on_result())
+
+        # The mock backend currently exports 70 attributes.
+        self.assertEqual(len(metadata), 70)
+
+        # Pick a particular attribute to spot-check.
+        for meta in metadata:
+            if meta.name != 'lightStatus.automaticHeadlights':
+                continue
+
+            self.assertEqual(meta.name, 'lightStatus.automaticHeadlights')
+            self.assertEqual(meta.zone_path, '/')
+            self.assertEqual(meta.availability,
+                             Rhosydd.AttributeAvailability.AVAILABLE)
+            self.assertEqual(meta.flags, Rhosydd.AttributeFlags.READABLE)
+
+    def test_vehicle_all_attributes(self):
+        """Test getting all attributes from a vehicle."""
+        mock_vehicle = self.__get_mock_vehicle()
+        root_zone = Rhosydd.StaticZone.new('/')
+
+        mock_vehicle.get_all_attributes_async(
+            zone=root_zone,
+            callback=self._block_on_result_cb,
+            user_data=self)
+        attributes = \
+            mock_vehicle.get_all_attributes_finish(self._block_on_result())
+
+        # The mock backend currently exports 70 attributes.
+        self.assertEqual(len(attributes), 70)
+
+        # Pick a particular attribute to spot-check.
+        for attr in attributes:
+            if attr.metadata.name != 'lightStatus.automaticHeadlights':
+                continue
+
+            self.assertEqual(attr.attribute.accuracy, 0.0)
+            self.assertEqual(attr.attribute.last_updated, 0)
+            self.assertEqual(attr.attribute.value, GLib.Variant('b', True))
+
+            self.assertEqual(attr.metadata.name,
+                             'lightStatus.automaticHeadlights')
+            self.assertEqual(attr.metadata.zone_path, '/')
+            self.assertEqual(attr.metadata.availability,
+                             Rhosydd.AttributeAvailability.AVAILABLE)
+            self.assertEqual(attr.metadata.flags,
+                             Rhosydd.AttributeFlags.READABLE)
+
+    def test_vehicle_metadata(self):
+        """Test getting metadata for a specific attribute from a vehicle."""
+        mock_vehicle = self.__get_mock_vehicle()
+        root_zone = Rhosydd.StaticZone.new('/')
+
+        mock_vehicle.get_metadata_async(
+            zone=root_zone,
+            attribute_name='lightStatus.automaticHeadlights',
+            callback=self._block_on_result_cb,
+            user_data=self)
+        metadata = mock_vehicle.get_metadata_finish(self._block_on_result())
+
+        self.assertEqual(metadata.name, 'lightStatus.automaticHeadlights')
+        self.assertEqual(metadata.zone_path, '/')
+        self.assertEqual(metadata.availability,
+                         Rhosydd.AttributeAvailability.AVAILABLE)
+        self.assertEqual(metadata.flags, Rhosydd.AttributeFlags.READABLE)
+
+    def test_vehicle_attribute(self):
+        """Test getting a specific attribute from a vehicle."""
+        mock_vehicle = self.__get_mock_vehicle()
+        root_zone = Rhosydd.StaticZone.new('/')
+
+        mock_vehicle.get_attribute_async(
+            zone=root_zone,
+            attribute_name='lightStatus.automaticHeadlights',
+            callback=self._block_on_result_cb,
+            user_data=self)
+        attribute = mock_vehicle.get_attribute_finish(self._block_on_result())
+
+        self.assertEqual(attribute.attribute.accuracy, 0.0)
+        self.assertEqual(attribute.attribute.last_updated, 0)
+        self.assertEqual(attribute.attribute.value, GLib.Variant('b', True))
+
+        self.assertEqual(attribute.metadata.name,
+                         'lightStatus.automaticHeadlights')
+        self.assertEqual(attribute.metadata.zone_path, '/')
+        self.assertEqual(attribute.metadata.availability,
+                         Rhosydd.AttributeAvailability.AVAILABLE)
+        self.assertEqual(attribute.metadata.flags,
+                         Rhosydd.AttributeFlags.READABLE)
+
+    def test_exported_objects(self):
+        """Test the SDK API only exports the objects we expect.
+
+        If it’s exporting more, that could leak confidential information from
+        the daemon.
+
+        Do this by introspecting the root object.
+        """
+        paths = self.__introspect_object_paths('org.apertis.Sensors1',
+                                               '/org/apertis/Rhosydd1')
+
+        self.assertCountEqual(paths, [
+            '/org/apertis/Rhosydd1',
+            '/org/apertis/Rhosydd1/mock',
+            '/org/apertis/Rhosydd1/speedo',
+        ])
+
+        # For some reason we end up with some undispatched sources in the main
+        # context. Handle those before finishing.
+        while self.main_context.iteration(False):
+            pass
+
+    def __introspect_object_paths(self, service, object_path):
+        """List objects beneath a given root.
+
+        Recursively build a list of all the objects available beneath (and
+        including) object_path on the well-known name, service.
+        """
+        object_paths = [object_path]
+        Gio.DBusProxy.new(connection=self.conn,
+                          flags=Gio.DBusProxyFlags.NONE,
+                          info=None,
+                          name=service,
+                          object_path=object_path,
+                          interface_name='org.freedesktop.DBus.Introspectable',
+                          cancellable=None,
+                          callback=self._block_on_result_cb,
+                          user_data=self)
+        proxy = Gio.DBusProxy.new_finish(self._block_on_result())
+
+        # Introspect the object to find its immediate child nodes.
+        proxy.Introspect(result_handler=self._block_on_result_cb,
+                         user_data=self)
+        introspection_data = self._block_on_result()
+        if isinstance(introspection_data, Exception):
+            raise introspection_data  # pylint: disable=raising-bad-type
+
+        for node in ElementTree.fromstring(introspection_data):
+            if node.tag == 'node':
+                if object_path == '/':
+                    object_path = ''
+                node_path = '/'.join((object_path, node.attrib['name']))
+                node_child_paths = self.__introspect_object_paths(service,
+                                                                  node_path)
+                object_paths += node_child_paths
+
+        return object_paths
+
+    def test_no_backend_access(self):
+        """Test that we cannot access backends directly.
+
+        We expect this to raise an AccessDenied error.
+        """
+        self.conn.call(bus_name='org.apertis.Rhosydd1.Backends.Mock',
+                       object_path='/org/apertis/Rhosydd1/Backends/Mock/mock',
+                       interface_name='org.apertis.Rhosydd1.Vehicle',
+                       method_name='GetAllAttributesMetadata',
+                       parameters=GLib.Variant('(s)', ('/',)),
+                       reply_type=GLib.VariantType('(a(ss(uu)))'),
+                       flags=Gio.DBusCallFlags.NONE,
+                       timeout_msec=-1,
+                       cancellable=None,
+                       callback=self._block_on_result_cb,
+                       user_data=self)
+
+        with self.assertRaises(GLib.Error) as err:
+            self.conn.call_finish(self._block_on_result())
+
+        self.assertEqual(err.exception.domain,
+                         GLib.quark_to_string(Gio.DBusError.quark()))
+        self.assertEqual(err.exception.code, Gio.DBusError.ACCESS_DENIED)
+
+
+if __name__ == '__main__':
+    unittest.main()