Skip to content
Snippets Groups Projects
Commit 1252b0f8 authored by Martyn Welch's avatar Martyn Welch Committed by Emanuele Aina
Browse files

Move documents to architecture


These documents are higher level than a guide, move to architecture.

Signed-off-by: default avatarMartyn Welch <martyn.welch@collabora.com>
parent 334c0498
Branches
No related tags found
1 merge request!45Reformat and move contents of guidelines
+++
date = "2016-09-05"
weight = 100
title = "Debug Symbol Packages"
aliases = [
"/old-wiki/Dbgsym"
]
+++
Packages automatically generate debug symbol packages at build time, the
packages have the package name extension `-dbgsym`.
# Infrastructure
To be able to benefit from `dbgsym` packages, the infrastructure must be ready
for them, the requirements are:
- `reprepro`: must support `dbgsym` since `4.17.0`
- `debhelper`: with `dbgsym` support since `9.20160114`
- `dpkg-dev`: support for `dbgsym` since `1.18.2~`
When building packages with those tools, automatic debug symbols will be
generated for all built packages.
`dbgsym` packages are supported since the Apertis `16.06` release, however not
all packages were rebuilt, therefore there are packages without `dbgsym`
packages. For the `16.09` release a full rebuild of the main archives was
performed to ensure all packages have a corresponding `dbgsym` package.
# How to use dbgsym packages
Usually, a package name with the `-dbgsym` extension, from Apertis
repositories, needs to be installed in the target runtime system, then your
system is able to resolve debug symbols for debugging purposes.
As reference you might read Debian documentation on [how to get a
backtrace](https://wiki.debian.org/HowToGetABacktrace), but please make sure to
use Apertis repositories to fetch the `dbgsym` packages.
# dbg packages are deprecated
Before `dbgsym` packages, package maintainers had to manually create their
`-dbg` packages listed in the `debian/control` file, these packages were
mirrored in every mirror and every suite, wasting lots of storage space. This
is not relevant for Apertis, but it is very relevant for Debian (Apertis parent
distribution). With `dbgsym` packages being produced, there is no need to add
any new stanzas to `debian/control`.
If you want to transition from a former `-dbg` package to a `-dbgsym` package
you might want to look into the
[dh_strip](http://manpages.ubuntu.com/manpages/xenial/en/man1/dh_strip.1.html)'s
`--dbgsym-migration=pkgname-dbg (<< currentversion~)` switch and drop `-dbg`
from `debian/control` file.
+++
date = "2015-02-20"
weight = 100
title = "Internationalization"
aliases = [
"/old-wiki/Guidelines/Internationalization"
]
+++
Internationalization (commonly abbreviated `i18n`) is a topic which covers many
areas: more than just translating UI strings, it involves changing settings and
defaults to match the customs and conventions of the locale a program is being
run in. For example, days of the week, human name formats, currencies, etc.
# Summary
- Design projects to be internationalized from the beginning.
([Basics](#basics))
- Use gettext (*not* intltool) for string translation. ([Basics](#basics))
- Remember that all strings are in UTF-8, and may contain multi-byte
characters. ([Unicode](#unicode))
- Programs cannot reasonably implement changing locales at runtime.
([Changing locale](#changing-locale))
# Basics
Documenting the whole process of preparing a project for internationalisation
is beyond the scope of this document, but some good guides exist:
- [GNOME developer translation guidelines](https://wiki.gnome.org/TranslationProject/DevGuidelines)
- [gtkmm translation guidelines](https://developer.gnome.org/gtkmm-tutorial/stable/chapter-internationalization.html.en)
(aimed at C++ programmers, but widely applicable to C programmers)
- [GLib internationalization API reference](https://developer.gnome.org/glib/stable/glib-I18N.html)
It is important to prepare a project for internationalization early in its
lifetime, otherwise non-internationalizable programming practices creep in, and
are hard to eliminate. For example,
[splitting strings into multiple translation units](https://wiki.gnome.org/TranslationProject/DevGuidelines/Never%20split%20sentences).
To add internationalization support to a project, follow the
[instructions here](https://wiki.gnome.org/TranslationProject/DevGuidelines/Localize%20using%20gettext%20and%20intltool),
which can be summarised as adding the following to `configure.ac`:
```
AM_GNU_GETTEXT_VERSION([0.19])
AM_GNU_GETTEXT([external])
GETTEXT_PACKAGE=AC_PACKAGE_TARNAME
AC_DEFINE_UNQUOTED(GETTEXT_PACKAGE, ["$GETTEXT_PACKAGE"], [Define to the Gettext package name])
AC_SUBST(GETTEXT_PACKAGE)
```
Note that intltool is outdated, and we only need to use gettext.
Add `po/Makefile.in` to `AC_CONFIG_FILES` and `po` to `SUBDIRS` in
`Makefile.am`. Then create an empty `po/POTFILES.in` file (which will be
modified when files are marked for translation), an empty `po/LINGUAS`
file (which will be modified when extra translation languages are
added), and create `po/Makevars` containing:
```
DOMAIN = $(PACKAGE)-$(VERSION)
COPYRIGHT_HOLDER =
MSGID_BUGS_ADDRESS =
EXTRA_LOCALE_CATEGORIES =
PO_DEPENDS_ON_POT = no
XGETTEXT_OPTIONS = \
--from-code=UTF-8 \
--keyword=_ --flag=_:1:pass-c-format \
--keyword=N_ --flag=N_:1:pass-c-format \
--flag=g_log:3:c-format --flag=g_logv:3:c-format \
--flag=g_error:1:c-format --flag=g_message:1:c-format \
--flag=g_critical:1:c-format --flag=g_warning:1:c-format \
--flag=g_print:1:c-format \
--flag=g_printerr:1:c-format \
--flag=g_strdup_printf:1:c-format --flag=g_strdup_vprintf:1:c-format \
--flag=g_printf_string_upper_bound:1:c-format \
--flag=g_snprintf:3:c-format --flag=g_vsnprintf:3:c-format \
--flag=g_string_sprintf:2:c-format \
--flag=g_string_sprintfa:2:c-format \
--flag=g_scanner_error:2:c-format \
--flag=g_scanner_warn:2:c-format
subdir = po
top_builddir = ..
```
These should be committed to git.
No other translation infrastructure files should be committed to git,
especially not the following. See the [module setup guidelines]( {{< ref
"module_setup.md#generated-files" >}} ) for more information.
- `po/ChangeLog`
- `po/Makefile.in.in`
- `po/POTFILES`
- `po/stamp-it`
- `po/*.mo`
# Unicode
All strings in GLib, unless otherwise specified, are in
[Unicode](http://en.wikipedia.org/wiki/Unicode), encoded as
[UTF-8](http://en.wikipedia.org/wiki/UTF-8). They must be handled as such,
which means all string manipulation must be done in terms of Unicode
characters, rather than bytes. In many cases, string manipulation functions do
not need to differentiate between the two; manual array indexing is a situation
where you should be careful.
GLib provides a
[set of UTF-8-safe versions of standard C string manipulation functions](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html),
which should always be used instead of the standard C ones.
## Sorting strings
When displaying sorted strings in the UI, care needs to be taken to ensure the
strings are sorted using Unicode algorithms, rather than plain ASCII
algorithms. This means using
[`g_utf8_collate()`](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-utf8-collate)
rather than [`strcmp()`](http://linux.die.net/man/3/strcmp) to establish an
order between two strings.
Furthermore, if section headings need to be used for splitting a list into
alphabetical sections, they need to be generated using the
[current locale’s alphabet](http://en.wikipedia.org/wiki/List_of_writing_systems),
rather than just the A–Z
[English alphabet](http://en.wikipedia.org/wiki/English_alphabet). One
approach to doing this would be to extract the first character of each item’s
name (using
[`g_utf8_get_char_validated()`](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-utf8-get-char-validated))
then using it as a section heading if it’s considered alphabetic for the
current locale (using
[`g_unichar_isalpha()`](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-unichar-isalpha)).
# Changing locale
Changing locale at runtime is not safe, as it requires calling
[`setenv()`](http://pubs.opengroup.org/onlinepubs/009695399/functions/setenv.html),
which is explicitly not thread safe. It also theoretically involves more than
just changing UI strings — it involves changing date formats, number formats,
and the output of any code which is predicated on those. The impacts of
changing locale can be far-reaching and subtle.
To change the locale of an application, the application has to be restarted.
# Language identifiers
When referring to languages (e.g. in configuration files or preferences),
always use the [ISO-639 language codes](http://en.wikipedia.org/wiki/ISO_639),
as used by gettext.
# External links
- [FAQ on Unicode and UTF-8](http://www.cl.cam.ac.uk/~mgk25/unicode.html)
- [Comprehensive introduction to Unicode](http://www.joelonsoftware.com/articles/Unicode.html)
- [Technical principles of Unicode](http://unicode.org/standard/principles.html)
- [GNOME developer translation guidelines](https://wiki.gnome.org/TranslationProject/DevGuidelines)
- [gtkmm translation guidelines](https://developer.gnome.org/gtkmm-tutorial/stable/chapter-internationalization.html.en)
(aimed at C++ programmers, but widely applicable to C programmers)
- [GLib internationalization API reference](https://developer.gnome.org/glib/stable/glib-I18N.html)
- [GLib Unicode API reference](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html)
+++
date = "2015-01-14"
weight = 100
title = "Tracker"
aliases = [
"/old-wiki/Guidelines/Tracker"
]
+++
[Tracker](https://wiki.gnome.org/Projects/Tracker) is a desktop search engine,
metadata indexing and storage service. It is the recommended way to search for
user files and access metadata about them. A
[full introduction to Tracker is here](https://wiki.gnome.org/Projects/Tracker/WhatIsTracker).
# Summary
- Avoid SPARQL injection vulnerabilities by using prepared statements.
([SPARQL injection](#sparql-injection))
# Using Tracker
Tracker is effectively a metadata store which applications can query using the
[SPARQL query language](http://en.wikipedia.org/wiki/SPARQL). SPARQL is
similar to SQL, so all the same considerations about
[SQL injection](#sparql-injection) apply when using it.
Describing how to use Tracker is beyond the scope of this document, however it
has good documentation in its
[getting started guide](https://wiki.gnome.org/Projects/Tracker/Documentation/GettingStarted)
and [API documentation](https://wiki.gnome.org/Projects/Tracker/Documentation).
# SPARQL injection
When using Tracker, queries **must** be constructed using prepared statements,
otherwise arbitrary SPARQL could be provided by the user which would affect the
query, potentially resulting in unauthorised user data disclosure. This would
be an
[SQL injection vulnerability](http://en.wikipedia.org/wiki/SQL_injection).
To build a SPARQL query, use
[`TrackerSparqlBuilder`](https://developer.gnome.org/libtracker-sparql/stable/TrackerSparqlBuilder.html),
which prevents SPARQL injection vulnerabilities as long as its ‘raw’ APIs
aren’t used. If its raw APIs are used, be very careful to escape all external
input to the query using
[`tracker_sparql_escape_string()`](https://developer.gnome.org/libtracker-sparql/stable/libtracker-sparql-Utilities.html#tracker-sparql-escape-string)
before including it in the query.
# External links
- [Tracker home page](https://wiki.gnome.org/Projects/Tracker)
- [Introduction to Tracker](https://wiki.gnome.org/Projects/Tracker/WhatIsTracker)
- [SPARQL query language](http://en.wikipedia.org/wiki/SPARQL)
- [Tracker getting started guide](https://wiki.gnome.org/Projects/Tracker/Documentation/GettingStarted)
- [Tracker API documentation](https://wiki.gnome.org/Projects/Tracker/Documentation)
+++
date = "2016-09-05"
weight = 100
title = "Dbgsym"
aliases = [
"/old-wiki/Dbgsym"
]
+++
# Dbgsym packages
Packages automatically generate debug symbol packages at build time, the
packages add the package name extension (<package_name>**-dbgsym**).
## Infrastructure
To be able to benefit from *dbgsym* packages, infrastructure must be
ready for them, the requirements are:
- *reprepro*: must support *dbgsym* since 4.17.0
- *debhelper*: with *dbgsym* support since 9.20160114
- *dpkg-dev*: support for *dbgsym* since 1.18.2\~
When building packages with those tools, automatic debug symbols will be
generated for all built packages.
## Since when can it be used in Apertis
*dbgsym* packages are supported since 16.06 release, however, not all
packages were rebuilt, therefore there are missing packages without
*dbgsym* packages. During 16.09 a full rebuild has been done with the
main archives to ensure all packages have a corresponding dbgsym
package.
## How to use dbgsym packages
Usually, a package name with **-dbgsym** extension, from Apertis
repositories, needs to be installed in the target runtime system, then
your system is able to resolve debug symbols for debugging purposes.
As reference you might read Debian documentation on [how to get a
backtrace](https://wiki.debian.org/HowToGetABacktrace), but please make
sure to use Apertis repositories to fetch *dbgsym* packages.
## dbg packages are deprecated
Before *dbgsym* packages, package maintainers had to manually create
their -dbg packages listed in the *debian/control* file, these packages
were mirrored in every mirror and every suite, wasting lots of storage
space. This is not relevant for Apertis, but it is very relevant for
Debian (Apertis parent distribution).
Therefore with *dbgsym* packages, there is no need to add any new
stanzas to *debian/control*.
If you want to transition from a former **-dbg** package to a
**-dbgsym** package you might want to look into the
[dh_strip](http://manpages.ubuntu.com/manpages/xenial/en/man1/dh_strip.1.html)'s
switch --dbgsym-migration=pkgname-dbg (\<\< currentversion\~)' switch
and drop **-dbg** from *debian/control* file.
\ No newline at end of file
+++
date = "2015-02-20"
weight = 100
title = "Internationalization"
aliases = [
"/old-wiki/Guidelines/Internationalization"
]
+++
# Internationalization
Internationalization (commonly abbreviated ‘i18n’) is a topic which
covers many areas: more than just translating UI strings, it involves
changing settings and defaults to match the customs and conventions of
the locale a program is being run in. For example, days of the week,
human name formats, currencies, etc.
## Summary
- Design projects to be internationalized from the beginning.
([Basics](#basics))
- Use gettext (*not* intltool) for string translation.
([Basics](#basics))
- Remember that all strings are in UTF-8, and may contain multi-byte
characters. ([Unicode](#unicode))
- Programs cannot reasonably implement changing locales at runtime.
([Changing locale](#changing-locale))
## Basics
Documenting the whole process of preparing a project for
internationalisation is beyond the scope of this document, but some good
guides exist:
- [GNOME developer translation
guidelines](https://wiki.gnome.org/TranslationProject/DevGuidelines)
- [gtkmm translation
guidelines](https://developer.gnome.org/gtkmm-tutorial/stable/chapter-internationalization.html.en)
(aimed at C++ programmers, but widely applicable to C programmers)
- [GLib internationalization API
reference](https://developer.gnome.org/glib/stable/glib-I18N.html)
It is important to prepare a project for internationalization early in
its lifetime, otherwise non-internationalizable programming practices
creep in, and are hard to eliminate. For example, [splitting strings
into multiple translation
units](https://wiki.gnome.org/TranslationProject/DevGuidelines/Never%20split%20sentences).
To add internationalization support to a project, follow the
[instructions
here](https://wiki.gnome.org/TranslationProject/DevGuidelines/Localize%20using%20gettext%20and%20intltool),
which can be summarised as adding the following to `configure.ac`:
AM_GNU_GETTEXT_VERSION([0.19])
AM_GNU_GETTEXT([external])
GETTEXT_PACKAGE=AC_PACKAGE_TARNAME
AC_DEFINE_UNQUOTED(GETTEXT_PACKAGE, ["$GETTEXT_PACKAGE"], [Define to the Gettext package name])
AC_SUBST(GETTEXT_PACKAGE)
Note that intltool is outdated, and we only need to use gettext.
Add `po/Makefile.in` to `AC_CONFIG_FILES` and `po` to `SUBDIRS` in
`Makefile.am`. Then create an empty `po/POTFILES.in` file (which will be
modified when files are marked for translation), an empty `po/LINGUAS`
file (which will be modified when extra translation languages are
added), and create `po/Makevars` containing:
DOMAIN = $(PACKAGE)-$(VERSION)
COPYRIGHT_HOLDER =
MSGID_BUGS_ADDRESS =
EXTRA_LOCALE_CATEGORIES =
PO_DEPENDS_ON_POT = no
XGETTEXT_OPTIONS = \
--from-code=UTF-8 \
--keyword=_ --flag=_:1:pass-c-format \
--keyword=N_ --flag=N_:1:pass-c-format \
--flag=g_log:3:c-format --flag=g_logv:3:c-format \
--flag=g_error:1:c-format --flag=g_message:1:c-format \
--flag=g_critical:1:c-format --flag=g_warning:1:c-format \
--flag=g_print:1:c-format \
--flag=g_printerr:1:c-format \
--flag=g_strdup_printf:1:c-format --flag=g_strdup_vprintf:1:c-format \
--flag=g_printf_string_upper_bound:1:c-format \
--flag=g_snprintf:3:c-format --flag=g_vsnprintf:3:c-format \
--flag=g_string_sprintf:2:c-format \
--flag=g_string_sprintfa:2:c-format \
--flag=g_scanner_error:2:c-format \
--flag=g_scanner_warn:2:c-format
subdir = po
top_builddir = ..
These should be committed to git.
No other translation infrastructure files should be committed to git,
especially not the following. See the [module setup
guidelines]( {{< ref "module_setup.md#generated-files" >}} ) for
more information.
- `po/ChangeLog`
- `po/Makefile.in.in`
- `po/POTFILES`
- `po/stamp-it`
- `po/*.mo`
## Unicode
All strings in GLib, unless otherwise specified, are in
[Unicode](http://en.wikipedia.org/wiki/Unicode), encoded as
[UTF-8](http://en.wikipedia.org/wiki/UTF-8). They must be handled as
such, which means all string manipulation must be done in terms of
Unicode characters, rather than bytes. In many cases, string
manipulation functions do not need to differentiate between the two;
manual array indexing is a situation where you should be careful.
GLib provides a [set of UTF-8-safe versions of standard C string
manipulation
functions](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html),
which should always be used instead of the standard C ones.
### Sorting strings
When displaying sorted strings in the UI, care needs to be taken to
ensure the strings are sorted using Unicode algorithms, rather than
plain ASCII algorithms. This means using
[`g_utf8_collate()`](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-utf8-collate)
rather than [`strcmp()`](http://linux.die.net/man/3/strcmp) to establish
an order between two strings.
Furthermore, if section headings need to be used for splitting a list
into alphabetical sections, they need to be generated using the [current
locale’s
alphabet](http://en.wikipedia.org/wiki/List_of_writing_systems), rather
than just the A–Z [English
alphabet](http://en.wikipedia.org/wiki/English_alphabet). One approach
to doing this would be to extract the first character of each item’s
name (using
[`g_utf8_get_char_validated()`](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-utf8-get-char-validated))
then using it as a section heading if it’s considered alphabetic for the
current locale (using
[`g_unichar_isalpha()`](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-unichar-isalpha)).
## Changing locale
Changing locale at runtime is not safe, as it requires calling
[`setenv()`](http://pubs.opengroup.org/onlinepubs/009695399/functions/setenv.html),
which is explicitly not thread safe. It also theoretically involves more
than just changing UI strings — it involves changing date formats,
number formats, and the output of any code which is predicated on those.
The impacts of changing locale can be far-reaching and subtle.
To change the locale of an application, the application has to be
restarted.
## Language identifiers
When referring to languages (e.g. in configuration files or
preferences), always use the [ISO-639 language
codes](http://en.wikipedia.org/wiki/ISO_639), as used by gettext.
## External links
- [FAQ on Unicode and
UTF-8](http://www.cl.cam.ac.uk/~mgk25/unicode.html)
- [Comprehensive introduction to
Unicode](http://www.joelonsoftware.com/articles/Unicode.html)
- [Technical principles of
Unicode](http://unicode.org/standard/principles.html)
- [GNOME developer translation
guidelines](https://wiki.gnome.org/TranslationProject/DevGuidelines)
- [gtkmm translation
guidelines](https://developer.gnome.org/gtkmm-tutorial/stable/chapter-internationalization.html.en)
(aimed at C++ programmers, but widely applicable to C programmers)
- [GLib internationalization API
reference](https://developer.gnome.org/glib/stable/glib-I18N.html)
- [GLib Unicode API
reference](https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html)
+++
date = "2015-01-14"
weight = 100
title = "Tracker"
aliases = [
"/old-wiki/Guidelines/Tracker"
]
+++
# Tracker
[Tracker](https://wiki.gnome.org/Projects/Tracker) is a desktop search
engine, metadata indexing and storage service. It is the recommended way
to search for user files and access metadata about them. A [full
introduction to Tracker is
here](https://wiki.gnome.org/Projects/Tracker/WhatIsTracker).
## Summary
- Avoid SPARQL injection vulnerabilities by using prepared statements.
([SPARQL injection](#sparql-injection))
## Using Tracker
Tracker is effectively a metadata store which applications can query
using the [SPARQL query language](http://en.wikipedia.org/wiki/SPARQL).
SPARQL is similar to SQL, so all the same considerations about [SQL
injection](#sparql-injection) apply when using it.
Describing how to use Tracker is beyond the scope of this document,
however it has good documentation in its [getting started
guide](https://wiki.gnome.org/Projects/Tracker/Documentation/GettingStarted)
and [API
documentation](https://wiki.gnome.org/Projects/Tracker/Documentation).
## SPARQL injection
When using Tracker, queries **must** be constructed using its prepared
statements, otherwise arbitrary SPARQL could be provided by the user
which would affect the query, potentially resulting in unauthorised user
data disclosure. This would be an [SQL injection
vulnerability](http://en.wikipedia.org/wiki/SQL_injection).
To build a SPARQL query, use
[`TrackerSparqlBuilder`](https://developer.gnome.org/libtracker-sparql/stable/TrackerSparqlBuilder.html),
which prevents SPARQL injection vulnerabilities as long as its ‘raw’
APIs aren’t used. If its raw APIs are used, be very careful to escape
all external input to the query using
[`tracker_sparql_escape_string()`](https://developer.gnome.org/libtracker-sparql/stable/libtracker-sparql-Utilities.html#tracker-sparql-escape-string)
before including it in the query.
## External links
- [Tracker home page](https://wiki.gnome.org/Projects/Tracker)
- [Introduction to
Tracker](https://wiki.gnome.org/Projects/Tracker/WhatIsTracker)
- [SPARQL query language](http://en.wikipedia.org/wiki/SPARQL)
- [Tracker getting started
guide](https://wiki.gnome.org/Projects/Tracker/Documentation/GettingStarted)
- [Tracker API
documentation](https://wiki.gnome.org/Projects/Tracker/Documentation)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment