Add designs from designs.a.o

Add the designs from designs.a.o with the minimum changes to get the
site to build. This essentially required to trailing marker for the
metadata to be changed from "..." to "---".

Signed-off-by: Martyn Welch <martyn.welch@collabora.com>

content/designs/application-bundle-metadata.md

+---
+title: Application bundle metadata
+short-description: Associating arbitrary metadata with entire app-bundles (implemented)
+authors:
+    - name: Simon McVittie
+---
+
+# Application bundle metadata
+
+This document extends the Apertis [Applications concept design]
+to cover metadata about [application bundles][Application bundle] (app-bundles).
+
+## Terminology and concepts
+
+See the [Apertis glossary] for background information on terminology.
+Apertis-specific jargon terms used in this document are hyperlinked
+to that glossary.
+
+## Use cases
+
+These use-cases are not exhaustive: we anticipate that other uses will
+be found for per-application-bundle metadata. At the time of writing,
+this document concentrates on use-cases associated with assigning priorities
+to requests from an app-bundle to a platform service.
+
+### Audio management priorities
+
+Assume that the Apertis audio management component (which is outside
+the scope of this document) assigns priorities to audio streams based
+on [OEM]-specific rules, potentially including user configuration.
+
+Suppose the author of an app-bundle has a legitimate reason to have their
+audio streams played with an elevated priority, for example because their
+app-bundle receives voice calls which should take precedence over music
+playback.
+
+Also suppose a different, malicious app-bundle author wishes to interrupt
+the driver's phone call to play an advertisement or other distracting sound
+as an audio stream.
+
+The Apertis system must be able to distinguish between the two app-bundles,
+so that requests for an elevated priority from the first app-bundle
+can be obeyed, while requests for an elevated priority from the second
+app-bundle are rejected.
+
+We assume that the app-bundles have been checked by an app-store
+curator before publication, and that the first app-bundle declares a special
+[permission][Permissions] in its app manifest, resulting in the app framework
+allowing it to flag its audio stream in ways that will result in it being
+treated as important, and hence superseding less important audio.
+Conversely, if the second app-bundle had declared that permission, we
+assume that the app-store curator would have recognised this as inappropriate
+and reject its publication.
+
+### Notification and dialog priorities
+
+Assume that the Apertis compositor (which is outside the scope of this
+document) assigns priorities to notifications based on [OEM]-specific rules,
+potentially including user configuration. Depending on the OEM's chosen
+UX design, app-modal and system-modal dialogs might be treated as
+visually similar to notifications; if they are, the compositor author
+might also wish to assign priorities from the same ranges to dialogs.
+
+Similar to the [][Audio management priorities] use case, app-bundles that
+have a legitimate reason for their notifications or dialogs to be
+high-priority must be able to achieve this, but malicious app-bundles
+whose authors aim to misuse this facility must not be able to achieve
+an elevated priority.
+
+### App-bundle labelling
+
+A UX designer might wish to arrange for all user interface elements
+associated with a particular app-bundle (including notifications, windows,
+its representation in lists of installed app-bundles, and so on)
+to be marked with an unambiguous indication of the app-bundle that created
+them, such as its name and icon.
+
+In particular, the Compositor Security concept design
+(which is [work in progress][Compositor Security design] at the time of
+writing) calls for windows and notifications to be visually associated
+with the app-bundle that created them, so that malicious app-bundle authors
+cannot make the user believe that information presented by the
+malicious app-bundle came from a different app-bundle (*output integrity*),
+and also cannot convince the user to enter input into the malicious
+app-bundle that they had only intended to go to a different app-bundle
+(a *trusted input path*, providing *input confidentiality* for the
+non-malicious app-bundle).
+
+Note this mechanism will not be effective unless either the app-store
+curator avoids accepting app-bundles with the same or confusingly similar
+names or icons, or the UX designer disambiguates app-bundles using
+something that is guaranteed to be unique, such as the app-bundle ID
+(which is not necessarily a desirable or user-friendly UX). This applies
+wherever app-bundles are listed, such as the app store's on-device user
+interface, the app-store's website, or a list of installed app-bundles
+in the device's equivalent of Android's Settings → Apps view.
+
+## Requirements
+
+### App-bundle metadata
+
+An Apertis platform library to read app bundle metadata must be made
+available to platform components, featuring at least these API calls:
+
+* given a bundle ID, return an object representing the metadata
+* list all installed bundles (either built-in or store) with their
+  IDs and metadata
+* emit a signal whenever the list of installed bundles changes,
+  for example because a store app bundle was installed, removed,
+  upgraded or rolled back (simple change-notification)
+
+### Labelling requirements
+
+Each app-bundle must contain a human-readable name in international English.
+It must also be possible for an app-bundle to contain translated versions
+of this name for other languages and locales, with the international English
+version used in locales where a translation is not provided.
+
+Each app-bundle must be able to contain the name of the authoring company
+or individual.
+
+Each app-bundle must contain a version number.
+To let the application manager make appropriate decisions, all
+application bundles must a format for their version strings that can
+be compared in a standard way. How an application developer chooses to set
+the version numbers is ultimately their decision, but Apertis must be
+able to determine whether one version number is higher than another.
+
+Collabora recommends requiring version numbers to be dotted-decimal
+(one or more decimal integers separated by single dots), with
+“major.minor.micro” (for example `3.2.4`) recommended but not strictly
+required.
+
+There will be a “store version” appended to the
+version string after a dash, similar to the versioning scheme used by
+`dpkg`; for example, in the version string `3.2.4-1`, the `1` is the store
+version. The store version allows the store to push an update even if
+the application version hasn't changed, and it will be the lowest
+significant figure. For example, version `2.2.0-1` is newer than version
+`2.1.99-4`. The store version will re-start at 1 any time the application
+version is increased, and will be incremented if a new intermediate
+release is required.
+
+### Secure identification
+
+Apertis [platform] services that receive requests from an unknown process
+must be able to identify which app-bundle the process belongs to. To support
+this, the request must take place via a channel that guarantees integrity
+for that process's identification: it must not be possible for a
+malicious process to impersonate a process originating from a different
+app-bundle.
+
+### Audio stream and notification requirements
+
+The information required by the audio manager must be represented as
+one or more metadata key-value pairs that can be read from the app bundle
+metadata.
+
+The information required by the notification implementation must be
+represented as one or more metadata key-value pairs that can be read
+from the app bundle metadata.
+
+We anticipate that audio management and notifications will not always
+assign the same priority to each app-bundle, therefore it must be
+possible for the metadata keys used by audio management and those
+used by notifications to be distinct.
+
+### App-store curator oversight
+
+It must be straightforward for an app-store curator to inspect the
+metadata that is present in an app-bundle, for example so that they can
+refuse to publish app-bundles that ask for audio or notification priorities
+that they have no legitimate reason to use, or for which the name,
+icon or other information used for [][App-bundle labelling] is misleading.
+
+### Store app-bundle confidentiality
+
+Ordinary unprivileged programs in store app-bundles must not be able
+to use these API calls to enumerate other installed store app-bundles.
+For example, if those API calls are implemented in terms of a D-Bus
+service, it must reject method calls from store app-bundles, or if those
+API calls are implemented in terms of reading the filesystem directly,
+store app-bundles' AppArmor profiles must not allow reading the necessary
+paths.
+
+*Non-requirement*: it is acceptable for ordinary unprivileged programs to
+be able to enumerate installed built-in app-bundles. Built-in app-bundles
+are part of the platform, so there is no expectation of confidentiality
+for them.
+
+### Extension points
+
+We anticipate that vendors will wish to introduce non-standardized
+metadata, either as a prototype for future standardization or to support
+vendor-specific additional requirements. It must be possible to include
+new metadata fields in an app-bundle, without coordination with a central
+authority. For example, this could be achieved by namespacing new
+metadata fields using a DNS name ([as is done in D-Bus][D-Bus names]),
+namespacing them with a URI ([as is done in XML][Namespaces in XML]),
+or using the [`X-Vendor-NewMetadataField` convention][X-Vendor] (as is done
+in email headers, HTTP headers and
+[freedesktop.org `.desktop` files][Desktop Entry Specification]).
+
+### Future directions
+
+#### Platform API requirements
+
+The application bundle metadata should include a minimum system version
+(API version) required to run the application, for example to prevent the
+installation of an application that requires at least Apertis 16.12 in an
+Apertis 16.09 environment. A specific versioning model for the Apertis API has
+not yet been defined.
+
+#### Declarative permissions
+
+The application bundle metadata should include simple, declarative
+[permissions] which can be used to generate an AppArmor profile
+in an automated way. The [Permissions concept design] tracks this work.
+
+#### Declaring system extensions
+
+The [Applications concept design] calls for app-bundle metadata to describe
+the types of [system extension][] (themes, addons, plugins, etc.) provided
+by an app-bundle. There is currently no detailed specification for this.
+
+AppStream upstream XML already supports declaring that a component
+(app-bundle) is an addon to another component (via the `addon` type)
+or to the system as a whole (via the `<provides>` element). There is no
+specific metadata to describe themes; discussion has been started in
+[AppStream issue 67](https://github.com/ximion/appstream/issues/67).
+
+#### Declaring where applications store non-essential files
+
+The [Applications concept design] suggests that application bundle metadata
+might declare where applications store non-essential files, so that the
+system can delete those files when disk space runs out.
+
+#### Placing symbolic links in centralized locations
+
+The [Applications concept design] suggests that for applications not
+originally designed for Apertis, which might write to locations like
+`~/.someapp`, application bundle metadata
+might declare where the platform must create symbolic links to cause
+those applications to read and write more appropriate locations on Apertis,
+for example
+`~/.someapp → /var/Applications/com.example.SomeApp/users/${UID}/data`.
+
+#### Declaring an EULA
+
+App-bundle metadata should include a way to specify an EULA which the
+user must agree with before the application bundle will be installed.
+See [AppStream issue 50](https://github.com/ximion/appstream/issues/50)
+for work on this topic in the AppStream specification.
+
+Other files in the
+license directory of the bundle but not mentioned in this way will
+still be copied the device, and the HMI components must provide some
+way to view that information later.
+
+#### Placeholder icons
+
+Since the installation process is not instant, a placeholder icon should be
+provided and specified in the version of the application bundle metadata
+that is downloaded from the application store. This icon will be copied into
+the store directory by the application store during publication. It will
+be displayed by the application manager instead of the application until
+the installation is completed. The application launcher will also be
+able to display a progress indicator or – if multiple applications are
+being installed – a position in the install queue.
+
+#### Platform component metadata
+
+Although it is not a requirement at this stage, we anticipate that it
+might be useful in the future to be able to associate similar metadata
+with platform components, such as the Newport download manager.
+
+## Other systems
+
+This section contains a very brief overview of the analogous functionality
+in other open-source platforms.
+
+### freedesktop.org AppStream
+
+Several open-source desktop platforms such as GNOME and KDE, and Linux
+distributions such as Ubuntu and Fedora, have adopted [AppStream]
+as a shared format for software component metadata, complementing the
+use of [`.desktop` files][Desktop Entry Specification] for
+[entry points].
+
+The AppStream specification refers to *components*, which are a
+generalization of the same concept as Apertis app-bundles, and can
+include software from various sources, including traditional distribution
+packages and bundling technologies such as [][Flatpak].
+
+#### Flatpak
+
+The [Flatpak] framework provides user-installable applications
+analogous to Apertis app-bundles. It uses [AppStream] for app-bundle
+metadata, together with [`.desktop` files][Desktop Entry Specification] for
+entry points.
+
+### Snappy
+
+Ubuntu [Snappy] packages (snaps) are also analogous to Apertis app-bundles.
+[Their metadata][Snappy metadata] consists of a Snappy-specific YAML
+file describing the snap, again together with
+[`.desktop` files][Desktop Entry Specification]
+describing entry points.
+
+### Android
+
+Android *apps* are its equivalent of Apertis app-bundles. Each app has a
+single [App manifest][Android App Manifest] file,
+which is an XML file with Android-specific contents, and describes both
+the app itself, and any *activities* that it provides (activities
+are analogous to Apertis [entry points]).
+
+## Design recommendations
+
+The [Apertis Application Bundle Specification] describes the metadata
+fields that can appear in an application bundle and are expected to
+remain supported long-term. This document provides rationale for those
+fields, suggested future directions, and details of functionality that
+is not necessarily long-term stable.
+
+### App-bundle metadata design
+
+We anticipate that other designs involving app-bundles will frequently
+require other metadata beyond the use-cases currently present in this
+document, for example categories.
+As such, we recommend introducing a general metadata file into built-in
+and store app-bundles.
+
+This metadata file could have any syntax and format that is readily parsed.
+To minimize duplicate effort, we recommend using [AppStream XML][AppStream],
+a format designed to be shared between desktop environments such as GNOME
+and KDE, and between Linux distributions such as Ubuntu and Fedora.
+
+Each built-in app bundle should install an [AppStream upstream XML]
+metadata file. If the built-in app bundle has [entry points],
+then its metadata file must be made available as
+`/usr/share/metainfo/${bundle_id}.appdata.xml` (where `${bundle_id}`
+represents its bundle ID), and its `<id>` must be
+`<id type="desktop">${entry_point_id}.desktop</id>` where `${entry_point_id}`
+represents its primary entry point (typically the same as the bundle ID).
+`/usr/share/metainfo/${bundle_id}.appdata.xml` will typically be a symbolic
+link to
+`/usr/Applications/${bundle_id}/share/metainfo/${bundle_id}.appdata.xml`.
+
+If the built-in app bundle has no entry points (for example a theme),
+then its metadata file must be available as
+`/usr/share/metainfo/${bundle_id}.metainfo.xml` (where `${bundle_id}`
+represents its bundle ID), and its `<id>` must be the same as its bundle ID.
+Again, this would typically be a symbolic link to a corresponding path
+in `/usr/Applications/${bundle_id}`.
+
+Each store app bundle should install an AppStream upstream XML metadata
+file into `/Applications/${bundle_id}/share/metainfo/${bundle_id}.appdata.xml`
+or `/Applications/${bundle_id}/share/metainfo/${bundle_id}.metainfo.xml`
+(depending on whether it has entry points), with contents
+corresponding to those specified for built-in app bundles.
+For [][Store app-bundle confidentiality], a store app-bundle's
+AppArmor profile must not allow it to read the contents of a different
+store app-bundle, and in particular its AppStream metadata.
+
+AppStream upstream XML is normally also searched for in the
+[deprecated path][deprecated AppStream path]
+`/usr/share/appdata`, but for simplicity, we do not require
+the `share/appdata/` directory to be processed for application bundles.
+Since existing application bundles do not contain it, this does not create
+a compatibility problem.
+
+For [][App-store curator oversight], if the implementation reads
+other sources of
+metadata from a store app-bundle (for example the `.desktop` entry points
+provided by the app-bundle), then the implementation must document those
+sources. The app-store curator must inspect all of those sources.
+This requirement does not apply to built-in app-bundles, which are assumed
+to have been checked thoroughly by the platform vendor at the time the
+built-in app-bundle was integrated into the platform image.
+
+The Apertis platform must provide cache files whose timestamps
+change whenever there is a change to the set of store or built-in app
+bundles, or to those bundles' contents.
+These cache files should be monitored by the
+[libcanterbury-platform][Canterbury] library, using the standard
+`inotify` mechanism. Any cache files that contain store app-bundles must
+not be readable by a store app-bundle, to preserve
+[][Store app-bundle confidentiality].
+
+The other APIs that are required are straightforward to implement in the
+[libcanterbury-platform][Canterbury] library by reading from the cache files.
+Because this is done in a library (as opposed to a D-Bus service),
+the implementation of these APIs will run with the privileges of the
+process that is requesting the information: in particular, if an
+unprivileged process attempts to read the cache files, this will be
+prevented by its AppArmor profile, regardless of whether it is using
+libcanterbury-platform or reading the files directly.
+
+We recommend that store app-bundles and built-in app-bundles appear in
+separate cache files, for several reasons:
+
+* In the current design for Apertis operating system upgrades,
+  the metadata files for built-in app-bundles and platform components
+  in `/usr/Applications/*/share/*` and `/usr/share/*` are
+  only updated during an operating system upgrade, by either `dpkg` or
+  by unpacking a new OS filesystem hierarchy that will be activated
+  after the next reboot. In the `dpkg` case, it is sufficient to have a
+  `dpkg` trigger monitoring these directories, and update the built-in
+  app-bundle cache when they have changed, leaving the store app-bundle
+  cache unchanged. Similarly, in the whole-OS upgrade case, the built-in
+  app-bundle cache can be provided in the new OS filesystem or rebuilt
+  during startup, again leaving the store app-bundle cache unchanged.
+
+* Conversely, the metadata files for store app-bundles are updated by
+  the Ribchester subvolume manager when it installs a new store app-bundle,
+  which can occur at any time. When it does this, it is sufficient to update
+  the store app-bundle cache, leaving the built-in app-bundle cache unchanged.
+
+* If Apertis moves to a more static model for deployment of the platform
+  (for example using disk images or OSTree to deploy pre-built filesystem
+  hierarchies), the built-in app-bundle cache would be entirely static and
+  could be included in the pre-built filesystem hierarchy.
+
+* Using separate caches makes it straightforward to ensure that if
+  a store app-bundle with the same name as a built-in app-bundle is
+  somehow installed, the built-in app-bundle takes precedence.
+
+Any metadata keys and values that have not been standardized by the AppStream
+project (for example audio roles that might be used to determine a bundle's
+audio priority) must be represented using [][Extension points] within the
+AppStream metadata. The formal [AppStream specification][AppStream] does
+not provide an extension point, but the
+[reference implementation][libappstream]
+and [appstream-glib] both provide support for a `<custom>` element
+with `<value>` children. We recommend using that element for extension
+points. See the [Apertis Application Bundle Specification] for
+details.
+
+When a store or built-in app-bundle is added, removed or changed,
+the Apertis platform must update the corresponding cache file.
+
+#### Future directions
+
+AppStream XML is equally applicable to platform components, which can
+install metadata in `/usr/share/metainfo` in the same way as built-in
+app-bundles.
+
+Because built-in app-bundles and platform components have the same update
+schedule and are managed by the same vendor (they are both part of the
+platform), we anticipate that platform components should use the same cache
+file as built-in app-bundles.
+
+### Secure identification design
+
+Consumers of requests from app-bundles, such as the audio manager or the
+notifications implementation, must receive the bundle ID alongside the
+request using a trusted mechanism. If the request is received via D-Bus,
+the bundle ID must be retrieved by using
+the [GetConnectionCredentials] method call to receive the AppArmor context,
+then parsing the context to get the bundle ID and whether it is a
+store or built-in app-bundle. If the request takes the form
+of a direct `AF_UNIX` socket connection, the bundle ID must be retrieved
+by reading the `SO_PEERCRED` socket option, then parsed in the same way.
+Consumers of app-bundle priorities should do this by using the
+[CbyProcessInfo] objects provided by [libcanterbury][Canterbury].
+
+Because the Apertis [Security concept design] does not place a security
+boundary between different processes originating from the same app-bundle,
+all identification of app-bundles should be carried out using their
+bundle IDs. In particular, consumers of requests from app-bundles should
+only use the requester's AppArmor label to derive its bundle ID and
+whether it is a store or built-in app-bundle, and must not use the
+complete AppArmor label, the complete path of the executable or the
+name of the corresponding [entry point][entry points] in access-control
+decisions.
+
+### Labelling design
+
+[AppStream upstream XML] already contains standardized metadata
+fields for a name, author name etc.
+
+The name (and several other metadata fields) can be translated via the
+`xml:lang` attribute. For example, GNOME Videos (Totem) has many
+language-specific names, starting with:
+
+---
+  <name>Videos</name>
+  <name xml:lang="af">Video's</name>
+  <name xml:lang="ar">فيديو</name>
+  <name xml:lang="as">ভিডিঅ'সমূহ</name>
+  <name xml:lang="be">Відэа</name>
+---
+
+AppStream upstream XML does not include an icon, although the derived
+[AppStream collection XML] format published by redistributors does. We recommend
+that the app-bundle should contain a PNG icon whose name matches its bundle ID,
+installed to its `share/` directory as part of the `hicolor` fallback theme.
+
+> The reserved icon theme name `hicolor` is used as the fallback whenever
+> a specific theme does not have the required icon, as specified in the
+> [freedesktop.org Icon Theme specification]. The name `hicolor` was chosen
+> for historical reasons.
+
+For example, `com.example.ShoppingList` would include
+`/Applications/com.example.ShoppingList/share/icons/hicolor/64x64/apps/com.example.ShoppingList.png`.
+If the app-store uses AppStream collection XML, then the process used to
+build AppStream collection XML from individual applications' AppStream upstream
+XML files should assume this icon name and include it in the collection XML.
+
+**Open question:** We should require a specific size for the icon, to avoid
+blurry or blocky app icons caused by resizing. GNOME Software uses 64×64
+as its baseline requirement, but recommends larger icons, for example 256×256.
+[iOS][iOS icon sizes] uses 1024×1024 for the App Store and ranges from 60×60
+to 180x180 for on-device icons. [Android][Android icons sizes] uses 512×512
+for the Google Play Store and ranges from 36×36 to 96×96 for on-device icons.
+What are our preferred sizes?
+
+#### Future directions
+
+Platform components that are not part of an app-bundle do not have bundle IDs.
+We anticipate that [][Platform component metadata] might be identified by
+a separate identifier in the same reversed-DNS namespace, and that the
+consumer of requests might derive the platform component identifier by
+looking for components that declare metadata fields matching the
+requester's AppArmor label (part of the AppArmor context).
+
+## Summary
+
+* [][app-bundle metadata] is read from the cache that summarizes
+  built-in and store app-bundles. The [libcanterbury-platform][Canterbury]
+  library provides the required APIs; in particular, change notification
+  can be done using `inotify`.
+* [][Secure identification] is provided by
+  [parsing the requesting process's AppArmor
+  context][Secure identification design].
+* The [][Audio stream and notification requirements]
+  are addressed by providing their desired metadata in the app-bundle
+  metadata, in the form of arbitrary key/value pairs.
+* [][App-store curator oversight] is facilitated by documenting all
+  of the sources within a store app-bundle from which the implementation
+  gathers metadata to populate its cache.
+* [][Store app-bundle confidentiality] is provided by storing the cache
+  file describing installed store app-bundles in a location where store
+  app-bundles cannot read it, and by avoiding the need to introduce a
+  D-Bus service from which they could obtain the same information.
+* The [appstream-glib] library supports [][Extension points] in
+  AppStream XML.
+
+<!-- External references -->
+[Android App Manifest]: https://developer.android.com/guide/topics/manifest/manifest-intro.html
+[Android icon sizes]: https://developer.android.com/guide/practices/ui_guidelines/icon_design_launcher.html
+[Apertis Application Bundle Specification]: https://appdev.apertis.org/documentation/bundle-spec.html
+[Apertis Glossary]: https://wiki.apertis.org/Glossary
+[Applications concept design]: applications.md
+[Application bundle]: https://wiki.apertis.org/Glossary#application-bundle
+[AppStream]: https://www.freedesktop.org/software/appstream/docs/
+[AppStream collection XML]: https://www.freedesktop.org/software/appstream/docs/chap-CollectionData.html
+[AppStream extension points]: https://lists.freedesktop.org/archives/appstream/2016-July/000048.html
+[AppStream upstream XML]: https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html
+[AppStream-Glib]: https://github.com/hughsie/appstream-glib/
+[Canterbury]: https://gitlab.apertis.org/appfw/canterbury
+[CbyProcessInfo]: https://gitlab.apertis.org/appfw/canterbury/blob/master/canterbury/process-info.h
+[Compositor Security design]: https://wiki.apertis.org/Compositor_security
+[D-Bus names]: https://dbus.freedesktop.org/doc/dbus-specification.html#message-protocol-names
+[Deprecated AppStream path]: https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#spec-component-location
+[Desktop Entry Specification]: https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html
+[Entry points]: https://wiki.apertis.org/Application_Entry_Points
+[Flatpak]: http://flatpak.org/
+[freedesktop.org Icon Theme Specification]: http://standards.freedesktop.org/icon-theme-spec/icon-theme-spec-latest.html
+[GetConnectionCredentials]: https://dbus.freedesktop.org/doc/dbus-specification.html#bus-messages-get-connection-credentials
+[iOS icon sizes]: https://developer.apple.com/library/safari/documentation/UserExperience/Conceptual/MobileHIG/IconMatrix.html
+[libappstream]: https://www.freedesktop.org/software/appstream/docs/api/index.html
+[OEM]: https://wiki.apertis.org/Glossary#oem
+[Namespaces in XML]: https://www.w3.org/TR/REC-xml-names/
+[Permissions]: permissions.md
+[Permissions concept design]: permissions.md
+[Platform]: https://wiki.apertis.org/Glossary#platform
+[Security concept design]: security.md
+[Snappy]: http://snapcraft.io/
+[Snappy metadata]: https://developer.ubuntu.com/en/snappy/guides/meta/
+[System extension]: applications.md#system-extensions
+[X-Vendor]: https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html#extending

content/designs/automated-license-compliance.md

+---
+title: Automated License Compliance
+short-description: Automated process for OSS compliance in Apertis
+authors:
+    - name: Martyn Welch
+---
+
+# Automated License Compliance
+
+A Linux system such as those assembled by Apertis contain components licensed under many different licenses.
+These various licenses impose different conditions and it is important to understand to a good degree of fidelity the terms under which each component is provided.
+We are proposing to implement an automated process to generate software Bills Of Materials (BOMs) which detail both the components used in Apertis and the licensing that applies to them.
+Licensing isn't static, nor is it always as simple as all the components from a given source package deriving the same license.
+Packages have been known to change licenses and/or provide various existing or new components under different terms.
+Either now or at some point in the future, the licenses of some of the components in Apertis may start to be provided under [terms that Apertis may wish to avoid](https://designs.apertis.org/latest/license-expectations.html).
+For example, by default Apertis is careful not to include components to be used in the target system that are licensed under the GPL version 3, the licensing terms wouldn't be acceptable in Apertis' target markets.
+
+In order to take advantage of new functionality and support being developed in the software community, Apertis needs to incorporate newer versions of existing software packages and replace some with alternatives when better or more suitable components are created.
+To ensure that the licensing conditions remain favorable for the use cases targeted by Apertis, it is important to continually validate that the licensing terms under which these components are provided.
+These licensing terms should be documented in a way that is accessible to Apertis' users.
+
+Debian packages by default track licensing on a per source package level.
+The suitability of a package is decided at that level before it is included in Debian, which meets the projects [licensing goals](https://www.debian.org/social_contract.html#guidelines).
+Apertis will continue to evaluate licensing before the inclusion of source packages in the distribution, but also wishes to take a more nuanced approach, tracking licensing for each file in each of it's binary packages.
+By tracking licensing to this degree we can look to exclude components with unsatisfactory licensing from the packages intended for distributed target systems, whilst still packaging them separately so they may be utilized during development.
+A good example of this situation is the `gcc` source package and the `libgcc1` binary package produced by it.
+Unlike the other artifacts produced by the GCC source package, the libgcc1 binary package is not licensed under the stock GPLv3 license, a [run time exception](https://designs.apertis.org/latest/license-exceptions.html#gcc8) is provided and it is thus fine to ship it on target devices.
+The level of tracking we are proposing will detect such situations and will offer a straight forward way to resolve them, maintaining compliance with the licensing requirements.
+
+To achieve this 2 main steps need to be taken:
+
+- Record the licensing of the project source code, per file
+- Determine the mapping between source code files and the binary/data files in each binary package
+
+We recommend to integrate these steps into our CI pipelines to provide early detection of any change to the licensing status of each package. Extending our CI pipelines will also enable developers to learn about new issues and to solve them during the merge request development flow.
+
+## License scanners
+
+There are various proprietary and open source tools which can help with tracking the licensing terms that apply to the pieces of software from which Apertis is built.
+The following tools are examples of those that can help to achieve the first of the steps outlined above:
+
+- [Dependency-Track](https://dependencytrack.org/): Open source, higher level tool for presenting data from BOMs generated by other software.
+- [FOSSA](https://fossa.com/): Proprietary suite for license compliance tracking and management.
+- [FOSSID](https://fossid.com/): Proprietary license scanning tool.
+- [FOSSology](https://www.fossology.org/): Open source, server based tool, utilizing a number of techniques to extract licensing information from source and binary artifacts.
+- [Licensecheck](https://metacpan.org/pod/App::Licensecheck): A simple open source license checker.
+- [Licensee](https://github.com/licensee/licensee): Open source tool, limited to scanning for license files.
+- [Ninka](http://ninka.turingmachine.org/): Very limited, light weight, open source tool, developed as a research project aimed at identifying licenses in source code.
+- [Protex](https://www.integrauae.com/datasheets/Protex_UL.pdf): Part of the Black Duck suite of proprietary tools for managing open source compliance.
+- [ScanCode](https://www.nexb.com/): Suite of open source tools, which provide a foundation on which the company developing them provides it's proprietary enterprise solution.
+- [WhiteSource](https://www.whitesourcesoftware.com/): Proprietary suite for open source component management.
+
+Due to the open source nature of the Apertis project, we intend to utilize an open source tool for license compliance rather than a proprietary solution.
+Given the traction, community, and Linux Foundation involvement, our suggestion of open source tool for license scanning is FOSSology.
+
+### FOSSology
+
+FOSSology is a server based tool which provides a web front-end that is able to scan through source code (and to a degree binaries) provided to it, finding license statements and texts.
+To achieve this FOSSology employs a number of different scanning techniques to identify potential licenses, including using matching to known license texts and keywords.
+The scanning process errs on the side of caution, generating false positives over missing potential licensing information, as a result it will be necessary to "clear" the licenses that are found, deciding whether the matches are valid or not.
+This is likely to be a very time consuming process, though bulk recognition of identical patterns may provide some efficiencies.
+Once completed, FOSSology will record the licensing decisions and can apply this information to updated scans of the source.
+It is anticipated that, after an initial round of verification, FOSSology will only require additional clearing of license information should the scan detect new sources of potential licensing information in an updated projects source or when new packages are added to Apertis.
+It is possible to export and import reports which contain the licensing decisions that have previously been made, if a trusted source of reports can be found then these could also be imported, potentially reducing the work required.
+
+FOSSology is backed by the Linux Foundation, it appears to have an active user and developer base and a significant history.
+As such, it is felt that this tool is likely to be maintained for the foreseeable future and thus a good choice for integration into the Apertis workflow.
+
+### CI Pipeline integration
+
+In order to avoid manual tasks the license detection should be integrated into the CI process.
+FOSSology provides a [REST API](https://www.fossology.org/get-started/basic-rest-api-calls/) to enable such integration.
+
+FOSSology is able to consume branches of git repositories, thus allowing scanning of the given source code straight from GitLab.
+It is suggested that this should be triggered after normal build checks have been successfully performed.
+A report will be generated and retrieved, using the REST API, which describes (among other things) the licensing status of each file.
+The report can be generated in a number of formats, including various SPDX flavors that are easily machine parsable, which will be the preferred option.
+It is suggested that each component should require a determination of the licensing to have been made for every file in the project.
+Due to the large volume of licensing matches that will result from the initial licensing scan, we recommend that the absence of license information initially generates a warning.
+In some cases, to achieve the fine grained licensing information desired, the licensing of some files may need to be clarified with the components author(s).
+Once an initial pass of all Apertis components had been made we would expect missing license information to result in an error, as such errors would be as a result of new matches being found, which would need to be resolved in FOSSology before CI would complete without an error.
+The generated report should be saved in the Debian metadata archive so that it is available for the following processing.
+
+## Binary to source file mapping
+
+Now that we have a way to determine the licensing of the source files, we need a way to determine which of these source files were used in each binary.
+Compilers store information in the binaries it outputs, that can be used by a debugger to pause execution of a process at a point corresponding to a selected line of source code.
+This information provides a mapping between the lines of source code and the compiled machine code operations.
+Executable binaries in Linux are generally stored in the [Executable and Linkable Format](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) (ELF), the associated [DWARF](https://en.wikipedia.org/wiki/DWARF) debugging data format is generally used to store this debugging information inside the ELF in specific "debug" sections.
+
+By parsing this information, the source files that were used to generate each binary can be determined.
+Combining this with the licensing information provided in the licensing report, a mapping can be made between each binary and it's associated licenses.
+
+### CI Pipeline integration
+
+Apertis uses the Open Build Service (OBS) platform to build the binary packages in a controlled manner across several architectures and releases.
+OBS utilizes `dpkg-buildpackage` behind the scenes to build each package.
+This utility will have access to the source licensing report as it is contained in the Debian metadata archive.
+As well as the source licensing, the Debian metadata archive contains configuration to help `dpkg-buildpackage` determine how to build the source.
+This is typically done with the help of [`debhelper`](https://manpages.debian.org/jessie/debhelper/debhelper.7.en.html), which provides helpers that simplify this process.
+We plan to extend `debhelper` to include a command to perform the mapping between the binary files produced by the build and the license of the associated source files, using the process laid out above, and recording this for each of the binary packages to be made.
+In addition, this helper should record the licensing attached to any other files that will be packaged as well.
+Typically the binaries are striped (using a debhelper command called `dh_strip`) prior to packaging, removing the debug symbols from the binary and reducing its size.
+We suggest that it would be easier to perform the license mapping prior to this step.
+Whilst the debug symbols are kept, packaged separately in the `dbgsym` package, it's easier to perform the mapping before this is done.
+A report should be saved in each binary package covering the files shipped in that package.
+The report should be saved in `/usr/share/doc/<package>/`in a machine parsable SPDX format.
+
+The new debhelper command will need to be added to the build rules for each package.
+Whilst most packages make use of debhelper, many do so via higher level helpers that factor out common functionality, such as `dh` and `CDBS` and this will add complexity to this task.
+There may be packages in Apertis that do not make use of debhelper, these packages will need special handling to ensure that the required steps are completed.
+
+As these reports are provided by each binary package, the reports from installed packages can be accessed at image build time and amalgamated into an image wide report at that point should it be required.
+As a binary can be built from multiple sources, each with differing licenses, it will be necessary for the report to detail each file that is used to create each binary and the licensing under which it is provided.
+In some circumstances dual licensed source code may allow for a binary to be effectively licensed under the terms of a single license, that is the user has the option to pick a license that results in the whole binary being able to be provided under the terms of a single license.
+Where dual licensed source code isn't used, the terms of all applicable licenses should be declared.
+The terms of the various licenses may be considered [compatible](https://en.wikipedia.org/wiki/License_compatibility), allowing the binary to effectively be managed under the terms of the more restrictive license.
+For example, a binary derived from source code licensed with the GPLv2 license and other source code licensed with the MIT license, the terms of both apply to the binary, though as the terms of the MIT license will be met if the binary is used in accordance with the terms of the GPLv2, then handling the binary as though it was licensed under the GPLv2 will ensure the terms of both are met.
+Not all possible combinations of licenses work out this way and thus why it is important to ensure that licensing is properly tracked.
+
+## Binary Licensing Reporting
+
+The approach each project using Apertis takes with regards to the reporting of licensing information should be driven by how this information is to be utilized, i.e. some projects may wish to parse the license information and present it in a single BOM file in HTML, XML or human readable text.
+
+For the images provided by the Apertis project, we plan to combine the reports saved in `/usr/share/doc/<package>/` into a single parsable file.
+Should it be required to provide some tool with which to interrogate the licensing which applies to the binary packages, the SPDX files can be imported into FOSSology.
+
+### CI Pipeline integration
+
+Apertis utilizes [Debos](https://github.com/go-debos/debos) in its image generation pipeline.
+There is an existing tool available for the merging of SPDX documents.
+The generation of a combined BOM can be realized by utilizing this tool in a script to be run at the appropriate time during the image build process by integrating the script into the Debos recipes.
+Integrating scripts into the Debos recipes is an approach we have taken when generating the list of installed packages and list of files.
+It reduces the overhead and potential complexity of decompressing and mounting the images that would be necessary should the BOM be generated in a separate step.

content/designs/development.md

+# Apertis Development