Skip to content
Snippets Groups Projects
Commit a12d059f authored by Martyn Welch's avatar Martyn Welch Committed by Emanuele Aina
Browse files

Move guides to guides


These documents can be moved to guides more or less as they are.

Signed-off-by: default avatarMartyn Welch <martyn.welch@collabora.com>
parent b9475721
No related branches found
No related tags found
1 merge request!42First chunk of guidelines rework
......@@ -162,7 +162,7 @@ D-Bus interface descriptions contain documentation comments, and these
can be extracted from the XML using `gdbus-codegen`, and turned into
DocBook files to be included by gtk-doc. Generate the documentation
using the rules described in the [D-Bus services
guidelines]( {{< ref "/guidelines/d-bus_services.md" >}} ).
guidelines]( {{< ref "d-bus_services.md" >}} ).
The DocBook files can be included in the main `*-docs.xml` file using:
......
......@@ -41,7 +41,7 @@ allows for greater interaction between the parent process and the task.
Subprocesses should not be used to implement functionality which could
be implemented by [D-Bus service
daemons]( {{< ref "/guidelines/d-bus_services.md" >}} ).
daemons]( {{< ref "d-bus_services.md" >}} ).
When using subprocesses, closed-loop management should be used, by
monitoring the child process’ PID after sending it kill signals, and
......
......@@ -13,28 +13,29 @@ This page describes some general API design principles which are useful
to bear in mind when designing APIs (whether those APIs are for C, C++,
JavaScript, D-Bus, or something else).
## Summary
- API designs must make sense from the point of view of a third-party
app developer ([start by designing high-level
APIs](#high-level-api), [only add daemons if it is
necessary](#fewer-daemons))
- Interfaces that don't have to be API should not be API ([minimize
surface area](#surface-area))
- [Follow GNOME/GObject conventions](#gnome-conventions),
so that we get JavaScript bindings automatically
- [Use existing frameworks](#reuse) where we can; if we
can't use them directly, learn from their design
- [Identify privilege boundaries](#privilege-boundary), do
not trust less-privileged components, and [consider whether some
features should be restricted](#restricted-features)
## <span id="surface-area">Minimize "surface area"</span>
The "SDK API" is intended to remain stable/compatible over time, which
means that we are committing to interfaces in the "SDK API" continuing
to work in future: third-party code that uses stable Apertis APIs must
continue to work in future, without needing changes.
# Summary
- API designs must make sense from the point of view of a third-party app
developer
([start by designing high-level APIs]( {{< ref "#start-from-the-api-that-the-app-developer-will-use" >}} ),
[only add daemons if it is necessary]( {{< ref "#have-as-few-daemons-as-possible-but-no-fewer" >}} ))
- Interfaces that don't have to be API should not be API
([minimize surface area]( {{< ref "#minimize-surface-area" >}} ))
- [Follow GNOME/GObject conventions]( {{< ref "#follow-gnome-conventions" >}} ), so
that we get JavaScript bindings automatically
- [Use existing frameworks]( {{< ref "#use-existing-software-or-if-we-cant-learn-from-it" >}} ) where we can; if we can't
use them directly, learn from their design
- [Identify privilege boundaries]( {{< ref "#be-aware-of-where-the-privilege-boundaries-are" >}} ), do not
trust less-privileged components, and
[consider whether some features should be restricted]( {{< ref "#think-about-how-much-we-want-the-app-to-be-allowed-to-do" >}} )
# Minimize "surface area"
The "SDK API" is intended to remain stable/compatible over time, which means
that we are
[committing to interfaces](https://designs.apertis.org/latest/supported-api.html#api-support-levels)
in the "SDK API" continuing to work in future: third-party code that uses
stable Apertis APIs should continue to work in future, without needing changes.
As a result, one of the most important questions to ask about new public
interfaces is: does this *need* to be public API right now? If it
......@@ -50,33 +51,34 @@ regret it in future.
Some examples of applying that principle:
- hiding struct contents by using a MyObjectPrivate struct instead of
putting members in the MyObject struct (in GObject, use
[G_ADD_PRIVATE()](https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-ADD-PRIVATE:CAPS)
or
[G_DEFINE_TYPE_WITH_PRIVATE()](https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DEFINE-TYPE-WITH-PRIVATE:CAPS))
- considering the D-Bus API between a built-in or otherwise special
app, and the system components it uses, to be private
- if in doubt, making things private initially, and making them public
if it later proves to be necessary
- hiding struct contents by using a `MyObjectPrivate` struct instead of putting
members in the `MyObject` struct (in GObject, use
[G_ADD_PRIVATE()](https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-ADD-PRIVATE:CAPS)
or
[G_DEFINE_TYPE_WITH_PRIVATE()](https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DEFINE-TYPE-WITH-PRIVATE:CAPS))
- considering the D-Bus API between a built-in or otherwise special app, and
the system components it uses, to be private
## <span id="fewer-daemons">Have as few daemons as possible, but no fewer</span>
If in doubt, making things private initially, and making them public if it
later proves to be necessary
# Have as few daemons as possible, but no fewer
Sometimes it's necessary to have more than one process (app code talking
to a daemon/service/agent, typically a [D-Bus
service]( {{< ref "/guidelines/d-bus_services.md" >}} )). There are lots of good
service]( {{< ref "d-bus_services.md" >}} )). There are lots of good
reasons to do that:
- having a [privilege boundary](#privilege-boundary)
- mediating between multiple processes that all want to manipulate the
state of the same object (for instance,
[Barkway](https://git.apertis.org/cgit/barkway.git/) decides the
order of the popup stack, which is global shared state)
- having something persist in the background when switching between
apps or closing/reopening apps (for instance,
[Telepathy](http://telepathy.freedesktop.org/wiki/) puts telephony,
instant messaging and other real-time communications in the
background)
- having a [privilege boundary]({{< ref "#be-aware-of-where-the-privilege-boundaries-are" >}})
- mediating between multiple processes that all want to manipulate the
state of the same object (for instance,
[Barkway](https://git.apertis.org/cgit/barkway.git/) decides the
order of the popup stack, which is global shared state)
- having something persist in the background when switching between
apps or closing/reopening apps (for instance,
[Telepathy](http://telepathy.freedesktop.org/wiki/) puts telephony,
instant messaging and other real-time communications in the
background)
However, every time we introduce inter-process communication between two
components, we increase the complexity of the system, which increases
......@@ -98,7 +100,7 @@ Shell](https://wiki.gnome.org/Projects/GnomeShell) puts notifications
and the window manager (among other things) in the same process. If the
requirements allow it, we should do the same.
## <span id="privilege-boundary">Be aware of where the privilege boundaries are</span>
# Be aware of where the privilege boundaries are
Not all of the code in Apertis is equally-privileged: components that
run as root are more privileged than components that run as the user,
......@@ -138,13 +140,13 @@ dbus-daemon, and can be trusted. If we can derive the app name from the
AppArmor profile, then that cannot be faked either, so we can reliably
identify an app by its profile.
## <span id="high-level-api">Start from the API that the app developer will use</span>
# Start from the API that the app developer will use
This is related to the [surface area](#surface-area) and
[fewer daemons](#fewer-daemons) reasoning above. The
requirements for our SDK APIs take the form "third-party apps should be
able to do X, Y and Z". In many cases, it is tempting to address this by
providing a D-Bus API that the third-party app can use, with
This is related to the [surface area]( {{< ref "#minimize-surface-area" >}} )
and [fewer daemons]( {{< ref "#have-as-few-daemons-as-possible-but-no-fewer" >}} )
reasoning above. The requirements for our SDK APIs take the form "third-party
apps should be able to do X, Y and Z". In many cases, it is tempting to address
this by providing a D-Bus API that the third-party app can use, with
auto-generated GDBus C "bindings".
In Collabora's experience, auto-generated code has limits: it's often
......@@ -171,24 +173,23 @@ indication that the result is going to be something that apps can use.
Because we intend to use
[GObject-Introspection](https://wiki.gnome.org/Projects/GObjectIntrospection)
to provide JavaScript versions of the SDK APIs, the C API translates
directly into a JavaScript API, so making a high-quality C API that
follows GObject conventions translates directly into improving the
JavaScript API. If we do not [follow GNOME API
conventions](#gnome-conventions) then GObject-Introspection
will not give us a usable JavaScript API.
to provide JavaScript versions of the SDK APIs, the C API translates directly
into a JavaScript API, so making a high-quality C API that follows GObject
conventions translates directly into improving the JavaScript API. If we do not
[follow GNOME API conventions]( {{< ref "#follow-gnome-conventions" >}} ) then
GObject-Introspection will not give us a usable JavaScript API.
## <span id="restricted-features">Think about how much we want the app to be allowed to do</span>
# Think about how much we want the app to be allowed to do
This is not normally a concern, but in Apertis it is, because there's a
privilege boundary between apps (whether our apps or third-party apps).
We need to consider (and document) which of these categories each
feature is in:
- all apps (including third-party ones) should be able to do this
- apps should be able to do this if they have some special flag in
their manifest
- only trusted components within Apertis should be able to do this
- all apps (including third-party ones) should be able to do this
- apps should be able to do this if they have some special flag in
their manifest
- only trusted components within Apertis should be able to do this
For instance, in Android, all apps can write to /sdcard; only apps that
have asked for the "permission" can record audio; and only trusted
......@@ -206,7 +207,7 @@ volume, whereas the worst case for recording is sending private
conversations to the Internet. So recording and playback should have a
clear division between them.
## <span id="gnome-conventions">Follow GNOME conventions</span>
# Follow GNOME conventions
Our platform includes a lot of GNOME-related libraries such as GLib and
Clutter, and our API guidelines follow GObject/GNOME style quite
......@@ -229,29 +230,29 @@ See [the Coding
Conventions](https://designs.apertis.org/latest/coding_conventions.html)
for more on this topic, but here is a brief summary:
- namespace objects with the library's appropriate prefix
- use GLib naming conventions: CapitalizedWords for types;
CAPITALS_AND_UNDERSCORES for constants;
lowercase_and_underscores for parameters, struct members and
variables
- avoid Hungarian notation (pSomePointer, v_func_returning_void(),
etc.)
- use
[GError](https://developer.gnome.org/glib/stable/glib-Error-Reporting.html)
to report recoverable runtime errors
- use
[`g_return_[val_]if_fail()`](https://designs.apertis.org/latest/coding_conventions.html#pre-and-postcondition-assertions)
to report failed precondition checks and other library-user errors
- use GObject features where appropriate: signals, properties,
construct-time properties, virtual methods (vfuncs)
- use GIO features where appropriate: GAsyncResult, GCancellable,
GTask
- prefer to use GAsyncResult instead of your own function typedef for
asynchronous operations
- prefer to use signals instead of your own function typedef for
events
## <span id="reuse">Use existing software, or if we can't, learn from it</span>
- namespace objects with the library's appropriate prefix
- use GLib naming conventions: CapitalizedWords for types;
CAPITALS_AND_UNDERSCORES for constants;
lowercase_and_underscores for parameters, struct members and
variables
- avoid Hungarian notation (pSomePointer, v_func_returning_void(),
etc.)
- use
[GError](https://developer.gnome.org/glib/stable/glib-Error-Reporting.html)
to report recoverable runtime errors
- use
[`g_return_[val_]if_fail()`](https://designs.apertis.org/latest/coding_conventions.html#pre-and-postcondition-assertions)
to report failed precondition checks and other library-user errors
- use GObject features where appropriate: signals, properties,
construct-time properties, virtual methods (vfuncs)
- use GIO features where appropriate: GAsyncResult, GCancellable,
GTask
- prefer to use GAsyncResult instead of your own function typedef for
asynchronous operations
- prefer to use signals instead of your own function typedef for
events
# Use existing software, or if we can't, learn from it
Several components in Apertis overlap with existing open-source
projects, in which people have already spent a lot of time understanding
......@@ -288,7 +289,7 @@ perhaps the difference points to a design issue in our API, which would
mean we can improve it by correcting that design issue, and get a better
outcome.
## API review process
# API review process
Before a public API may be marked as stable and suitable for use by third-party
apps, it must be reviewed. The process for this is:
......
......@@ -23,7 +23,7 @@ with
[`g_settings_bind()`](https://developer.gnome.org/gio/stable/GSettings.html#g-settings-bind).
Due to its tight integration with other GLib utilities, it should be
used in preference to (e.g.) [SQLite]( {{< ref "/guidelines/sqlite.md" >}} ) for
used in preference to (e.g.) [SQLite]( {{< ref "sqlite.md" >}} ) for
configuration data (but not for user data or documents, or high volumes
of data).
......
......@@ -11,43 +11,42 @@ aliases = [
# JSON parsing
JSON is used for various formats within Apertis, and potentially also
for various web APIs. It is a well defined format, and several mature
libraries exist for parsing it. However, the JSON being parsed typically
comes from untrusted sources (user input or untrusted web APIs), so must
be validated extremely carefully to prevent exploits.
JSON is used for various formats within Apertis, and potentially also for
various web APIs. It is a well defined format, and several mature libraries
exist for parsing it. However, the JSON being parsed typically comes from
untrusted sources (user input or untrusted web APIs), so must be validated
extremely carefully to prevent exploits.
## Summary
- Use a standard library to parse JSON, such as json-glib. ([Parsing
JSON](#parsing-json))
- Be careful to pair up JSON reader functions on all code paths.
([Parsing JSON](#parsing-json))
- Write a JSON schema for each JSON format in use. ([Schema
validation](#schema-validation))
- Use Walbottle to validate JSON schemas and documents. ([Schema
validation](#schema-validation))
- Use Walbottle to generate test vectors for unit testing JSON reader
code. ([Unit testing](#unit-testing))
- Use a standard library to parse JSON, such as json-glib.
([Parsing JSON]( {{< ref "json_parsing.md#parsing-json" >}} ))
- Be careful to pair up JSON reader functions on all code paths.
([Parsing JSON]( {{< ref "json_parsing.md#parsing-json" >}} ))
- Write a JSON schema for each JSON format in use.
([Schema validation]( {{< ref "json_parsing.md#schema-validation" >}} ))
- Use Walbottle to validate JSON schemas and documents.
([Schema validation]( {{< ref "json_parsing.md#schema-validation" >}} ))
- Use Walbottle to generate test vectors for unit testing JSON reader
code. ([Unit testing]( {{< ref "json_parsing.md#unit-testing" >}} ))
## Parsing JSON
JSON should be parsed using a standard library, such as
[json-glib](https://developer.gnome.org/json-glib/stable/). That will
take care of checking the JSON for well-formedness and safely parsing
the values it contains. The output from json-glib is a hierarchy of
parsed JSON nodes which may be values, arrays or objects. The Apertis
code must then extract the data it requires from this hierarchy. This
navigation of the hierarchy is still security critical, as the parsed
JSON document may not conform to the expected format (the *schema* for
that document). See [Schema
validation](#schema-validation) for more information on
this.
[json-glib](https://developer.gnome.org/json-glib/stable/). That will take care
of checking the JSON for well-formedness and safely parsing the values it
contains. The output from json-glib is a hierarchy of parsed JSON nodes which
may be values, arrays or objects. The Apertis code must then extract the data
it requires from this hierarchy. This navigation of the hierarchy is still
security critical, as the parsed JSON document may not conform to the expected
format (the *schema* for that document). See
[Schema validation]( {{< ref "json_parsing.md#schema-validation" >}} ) for more
information on this.
When using json-glib, the
[`JsonReader`](https://developer.gnome.org/json-glib/stable/JsonReader.html)
object is typically used to navigate a parsed JSON document and extract
the required data. A common pitfall is to not pair calls to
object is typically used to navigate a parsed JSON document and extract the
required data. A common pitfall is to not pair calls to
[`json_reader_read_member()`](https://developer.gnome.org/json-glib/stable/JsonReader.html#json-reader-read-member)
and
[`json_reader_end_member()`](https://developer.gnome.org/json-glib/stable/JsonReader.html#json-reader-end-member).
......@@ -71,10 +70,10 @@ For example:
return retval;
}
This code is incorrect because `json_reader_end_member()` is not called
on the code path where the `member-name` member doesn’t exist. That
leaves the `JsonReader` in an error state, and any remaining read
operations will silently fail.
This code is incorrect because `json_reader_end_member()` is not called on the
code path where the `member-name` member doesn’t exist. That leaves the
`JsonReader` in an error state, and any remaining read operations will silently
fail.
Instead, the following should be done:
......@@ -95,26 +94,24 @@ Instead, the following should be done:
The same is true of other APIs, such as
[`json_reader_read_element()`](https://developer.gnome.org/json-glib/stable/JsonReader.html#json-reader-read-element).
Read the API documentation for json-glib functions carefully to check
whether the function will put the `JsonReader` into an error state on
failure and, if so, how to get it out of that error state.
Read the API documentation for json-glib functions carefully to check whether
the function will put the `JsonReader` into an error state on failure and, if
so, how to get it out of that error state.
## Schema validation
Ideally, all JSON formats will have an accompanying [JSON
schema](http://json-schema.org/) which describes the expected structure
of the JSON files. A JSON schema is analogous to an XML schema for XML
documents. If a schema exists for a JSON document which is stored in git
(such as a UI definition), that document can be validated at compile
time, which can help catch problems without the need for runtime
testing.
One tool for this is
[Walbottle](http://people.collabora.com/~pwith/walbottle/), which allows
validation of JSON documents against schemas. Given a schema called
schema](http://json-schema.org/) which describes the expected structure of the
JSON files. A JSON schema is analogous to an XML schema for XML documents. If a
schema exists for a JSON document which is stored in git (such as a UI
definition), that document can be validated at compile time, which can help
catch problems without the need for runtime testing.
One tool for this is [Walbottle](https://github.com/pwithnall/walbottle), which
allows validation of JSON documents against schemas. Given a schema called
`schema.json` and two JSON documents called `example1.json` and
`example2.json`, the following `Makefile.am` snippets will validate them
at compile time:
`example2.json`, the following `Makefile.am` snippets will validate them at
compile time:
json_schema_files = schema.json
json_files = example1.json example2.json
......@@ -130,32 +127,29 @@ at compile time:
## Unit testing
Due to the susceptibility of JSON handling code to break on invalid
input (as it assumes the input follows the correct schema, which it may
not, as it’s untrusted), it is important to unit test such code. See the
[Unit testing guidelines]( {{< ref "/guidelines/unit_testing.md" >}} ) for
suggestions on writing code for testing. The ideal is for the JSON
parsing code to be separated from whatever code calls it, so that it can
be linked into unit tests by itself, and passed JSON snippets to check
what it retrieves from them.
Thinking of JSON snippets which thoroughly test parsing and validation
code is hard, and is impossible to do without also using code coverage
metrics (see the [Tooling
guidelines]( {{< ref "/guidelines/tooling.md#code-coverage" >}} )). However,
given a JSON schema for the document, it is possible to automatically
Due to the susceptibility of JSON handling code to break on invalid input (as
it assumes the input follows the correct schema, which it may not, as it’s
untrusted), it is important to unit test such code. See the [Unit testing
guidelines]( {{< ref "/guidelines/unit_testing.md" >}} ) for suggestions on
writing code for testing. The ideal is for the JSON parsing code to be
separated from whatever code calls it, so that it can be linked into unit tests
by itself, and passed JSON snippets to check what it retrieves from them.
Thinking of JSON snippets which thoroughly test parsing and validation code is
hard, and is impossible to do without also using code coverage metrics (see the
[Tooling guidelines]( {{< ref "/guidelines/tooling.md#code-coverage" >}} )).
However, given a JSON schema for the document, it is possible to automatically
and exhaustively generate [unit test
vectors](http://en.wikipedia.org/wiki/Test_vector) which can be easily
copied into the unit tests to give good coverage.
vectors](http://en.wikipedia.org/wiki/Test_vector) which can be easily copied
into the unit tests to give good coverage.
This can be done using Walbottle:
json-schema-generate --valid-only schema.json
json-schema-generate --invalid-only schema.json
That command will generate sets of valid and invalid test vectors, each
of which is a JSON instance which may or may not conform to the given
schema.
That command will generate sets of valid and invalid test vectors, each of
which is a JSON instance which may or may not conform to the given schema.
## External links
......
......@@ -16,19 +16,19 @@ throughput databases. It is used for storing some user data in Apertis.
## Summary
- Use SQLite for appropriate use cases: not configuration data (use
[GSettings]( {{< ref "/guidelines/gsettings.md" >}} )). ([When to use
SQLite](#when-to-use-sqlite))
- Consider your vacuuming policy before committing to using SQLite.
([When to use SQLite](#when-to-use-sqlite))
- Avoid SQL injection vulnerabilities by using prepared statements.
([SQL injection](#sql-injection))
- Use SQLite for appropriate use cases: not configuration data (use
[GSettings]( {{< ref "gsettings.md" >}} )). ([When to use
SQLite]( {{< ref "gsettings.md#when-to-use-sqlite" >}} ))
- Consider your vacuuming policy before committing to using SQLite.
([When to use SQLite]( {{< ref "gsettings.md#when-to-use-sqlite" >}}))
- Avoid SQL injection vulnerabilities by using prepared statements.
([SQL injection]( {{< ref "gsettings.md#sql-injection" >}} ))
## When to use SQLite
Even though it is lightweight for a database, SQLite is a fairly
heavyweight solution to some problems. It should *not* be used for
configuration data: [GSettings]( {{< ref "/guidelines/gsettings.md" >}} ) should
configuration data: [GSettings]( {{< ref "gsettings.md" >}} ) should
be used for that. Similarly, it is not suitable for more fully featured
database systems which require support for concurrent access or advanced
SQL support. It fills the middle space, and is best suited to situations
......
......@@ -12,22 +12,22 @@ aliases = [
# XML parsing
XML is used for a few formats within Apertis, although not as many as
[JSON]( {{< ref "/guidelines/json_parsing.md" >}} ). It is more commonly used
[JSON]( {{< ref "json_parsing.md" >}} ). It is more commonly used
internally by various GLib systems and tools, such as
[GSettings]( {{< ref "/guidelines/gsettings.md" >}} ) and
[D-Bus]( {{< ref "/guidelines/d-bus_services.md" >}} ). In situations where it
[GSettings]( {{< ref "gsettings.md" >}} ) and
[D-Bus]( {{< ref "d-bus_services.md" >}} ). In situations where it
is parsed by Apertis code, the XML being parsed typically comes from
untrusted sources (untrusted web APIs or user input), so must be
validated extremely carefully to prevent exploits.
## Summary
- Use a standard library to parse XML, such as libxml2. ([Parsing
XML](#parsing-xml))
- Write an XML schema for each XML format in use. ([Schema
validation](#schema-validation))
- Use xmllint to validate XML documents. ([Schema
validation](#schema-validation))
- Use a standard library to parse XML, such as libxml2. ([Parsing
XML]( {{< ref "xml_parsing.md#parsing-xml" >}} ))
- Write an XML schema for each XML format in use. ([Schema
validation]( {{< ref "xml_parsing.md#schema-validation" >}} ))
- Use xmllint to validate XML documents. ([Schema
validation]( {{< ref "xml_parsing.md#schema-validation" >}} ))
## Parsing XML
......@@ -71,7 +71,7 @@ following `Makefile.am` snippet will validate them at compile time:
.PHONY: check-xml
Various existing autotools macros for systems which use XML, such as
[GSettings]( {{< ref "/guidelines/gsettings.md" >}} ), already automatically
[GSettings]( {{< ref "gsettings.md" >}} ), already automatically
validate the relevant XML files.
## External links
......
......@@ -215,7 +215,7 @@ of actuators (for example, seat adjustment) in a safe manner.
Improvements have been made to the core of
[Walbottle](https://github.com/pwithnall/walbottle), a unit test
generator for [JSON data formats]( {{< ref "/guidelines/json_parsing.md" >}} ),
generator for [JSON data formats]( {{< ref "json_parsing.md" >}} ),
to allow it to be used more for testing Apertis SDK libraries. Walbottle
has been integrated with libthornbury, a UI utility library. Further
improvements will be made to it after the 15.09 release, and it will be
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment