webkit2.md

+++
date = "2018-06-08"
weight = 100

title = "WebKit2"

aliases = [
    "/old-wiki/WebKit2"
]
+++

Apertis is moving to a WebView widget and browser based on WebKit2GTK+.
The purpose of this migration is to take advantage of the new WebKit2
multi-process architecture, adopt a port that is upstream and that is
based on a more future-proof toolkit.

# Packages for testing and using the API

The engine is available in Apertis on the webkit2gtk source package. To
test it on a development or SDK image, install the
**webkit2gtk-testing** package and run **/usr/bin/GtkClutterLauncher**.
To develop programs that use the API, you'll need the
**libwebkit2gtk-4.0-dev** package.

# Where to build the code

WebKit is a very large code base and it can be very tricky to build and
debug. To give you an idea, a full debug build on a 64 bits system will
use over 23GB of disk space. If you need to build it yourself, here are
a few recommendations.

You can use any architecture, but amd64 is preferred, especially for
debug builds. We are not going to support building on any other
distribution such as Ubuntu given the eventual need for patched basic
libraries such as glib or clutter to get it to build or work.

By bind-mounting `/run` from the container to the host you can launch
\`GtkClutterLauncher\` directly from within the container, accessing
your current X session. Relying on your `$HOME` avoid incurring in the
limited disk space available on the SDK image.

Using a shared folder to export the build to a
[VirtualBox]( {{< ref "/virtualbox.md" >}} ) instance is quite likely to be more
efficient than building in the VM (understatement of the year).

When building on ARM you will probably need a lot of swap space, since
the board is likely to not have more than 1 or 2 gigabytes of RAM. Make
it at least 4GB large, 8GB if doing a debug build. Be sure to use an SSD
to back it or your builds will take longer than forever.

If you need to debug you'll need a debug build. Those are very hard to
do on a 32 bits system, so we advise against trying unless you need to
debug something that is specific to those architectures.

# Getting and building the code

Grab the code from here:

<https://gitlab.apertis.org/hmi/webkit-gtk-clutter>

You can install the build dependencies using APT on a development image:

`$ sudo apt-get build-dep webkit2gtk`

Remember that the port we are targetting is the GTK+ port, not the
Clutter one. In order to build using the build-webkit script run:

`$ ./Tools/Script/build-webkit --gtk --no-media-stream`

Or if you want to use cmake directly, something like this:

`$ mkdir build`
`$ cd build`
`$ cmake -DPORT=GTK ..`
`$ make`

# Passing arguments to the build

Along witht he switches available in build-webkit, it is possible to use
a lot of CMake parameters to customize the build process. To install to
a specific prefix, for example, you can provide CMake with the
CMAKE_INSTALL_PREFIX option:

`$ cmake -DPORT=GTK -DCMAKE_INSTALL_PREFIX=/my/install/path/ ..`

To pass cmake arguments to the build when using build-webkit, use the
--cmakeargs parameters:

`$ ./Tools/Script/build-webkit --gtk --no-accelerated-2d-canvas --no-media-stream --cmakeargs='-DCMAKE_INSTALL_PREFIX=/my/install/path/'`

# Developing and debugging

The simplest way of testing the engine is through the test launcher that
lives in the same repository as the rest of the codebase:

<https://gitlab.apertis.org/hmi/webkit-gtk-clutter/blob/master/Tools/GtkClutterLauncher/main.cpp>

After the build is done it will be in the build directory at
**bin/GtkClutterLauncher**. For instance, if you build using
build-webkit it will be at
**WebKitBuild/Release/bin/GtkClutterLauncher** or
**WebKitBuild/Debug/bin/GtkClutterLauncher** in case you ran a debug
build.

# How to use the port on a Clutter application

The port provides a ClutterActor wrapper that controls the GTK+ widget
behind the curtains. To do that it provides a new object in addition to
the usual WebKit2GTK+ API: **WebKitWebClutterView**. It can be created
and used like any other actor.

# How to customize gtk widgets

You can style gtk widget by using GtkCssProvider. Read
[WebKit2_CustomizableWidget]( {{< ref "/webkit2_customizablewidget.md" >}} )

# Updating the package

WebKit2 supports `make dist` to build the tarball when invoking `cmake`
with `-DDEVELOPER_MODE=ON`. That will generate a tarball named like
this: ''webkitgtk-2.14.1.tar.xz ''. When updating the package you need
to update the version.

`# Checkout the master branch`
`git checkout master`

`# Compute the package version`
`HASH=$(git show-ref --head HEAD --abbrev=7 --hash) # eg. cc1ecf7`
`DATE=$(date '+%Y%m%d') # eg. 20170429`
`RELEASE=$(awk '/set.PROJECT_VERSION_/ { gsub(")","", $2); print $2 }' Source/cmake/OptionsGTK.cmake | paste -sd '.') # eg. 2.14.1`
`VERSION="${RELEASE}+${DATE}+${HASH}" # eg. 2.14.1+20170429+cc1ecf7`
`DEBVERSION="${VERSION}-0co1" # eg. 2.14.1+20170429+cc1ecf7-0co1`

`# From the master branch, generate the dist tarball`
`cmake . -DPORT=GTK -DENABLE_SPELLCHECK=OFF -DENABLE_ACCELERATED_2D_CANVAS=OFF -DENABLE_GLES2=ON -DENABLE_PLUGIN_PROCESS_GTK2=OFF -DENABLE_TOOLS=ON -DENABLE_MINIBROWSER=ON -DDEVELOPER_MODE=ON`
`make -j $(getconf _NPROCESSORS_ONLN)`
`make dist`
`mv webkitgtk-${RELEASE}.tar.xz ../`

`# Tag the snapshot - the format is v`<upstream version>`+`<iso-date>`+abbrev-commit`
`git tag v${VERSION}`

`# Inside the git repository, switch to the apertis/master branch and import the tarball into the upstream branch:`
`git checkout apertis/master`
`gbp import-orig --upstream-branch=upstream --debian-branch=apertis/master --upstream-vcs-tag=v${VERSION} -u ${VERSION} ../webkitgtk-${RELEASE}.tar.xz `

`# Run dch -i to create the new changelog entry, commit`
`dch -v ${DEBVERSION} -D 17.06`
`git commit -a -m "Release new snapshot: ${DEBVERSION}"`

`# Build the package, exporting it to a build area and tagging it at the end`
`gbp buildpackage --git-debian-branch=apertis/master --git-export-dir=../build-area/ --git-tag -S -uc -us`

`# Push branches and tags`
`git push origin --tags`
`git push origin upstream`
`git push origin apertis/master`

# Cross-building during development

Building WebKit with full debug symbols requires a huge amount of CPU
and memory and it's simply not possible to do so in a native ARM build.
Unless you can assign at least 8GB of RAM to a VM, it's also unlikely to
work there.

The simplest solution is to use a
[bubblewrap](https://github.com/projectatomic/bubblewrap) container and
cross build from it:

<code>

    CHROOT=apertis-17.03-amd64
    mkdir "$CHROOT"
    wget https://images.apertis.org/release/17.03/ospacks/sdk/amd64/17.03.0/ospack_17.03-amd64-sdk_17.03.0.tar.gz
    tar -f ospack_17.03-amd64-sdk_17.03.0.tar.gz -C "$CHROOT" -x --exclude 'dev/*'
    bwrap \
      --bind "$CHROOT" / \
      --proc /proc \
      --dev-bind /dev /dev \
      --ro-bind /sys /sys \
      --bind /tmp /tmp \
      --tmpfs /run \
      --dir /run/user/1000 \
      --bind $HOME /home/user \
      --setenv debian_chroot "$(basename "$CHROOT")" \
      --setenv GSETTINGS_BACKEND memory \
      --setenv XDG_RUNTIME_DIR /run/user/1000  \
      --setenv USER user \
      --setenv HOME /home/user \
      /bin/bash
    # now inside the container
    ade sysroot install

    # to install packages you don't have to be root, but you need to pretend to be
    apt update
    function sudo () { PATH=/sbin:/usr/sbin:/usr/local/sbin:$PATH fakeroot "$@"; }
    sudo apt install eatmydata

</code>

Using `-DCMAKE_BUILD_TYPE=Debug` will automatically produce full debug
symbols and enable assertions in the code.

To simply have working backtraces it can be enough to use
`-DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_C_FLAGS_RELWITHDEBINFO=-g1
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-g1 -DCMAKE_CXX_FLAGS=-g1
-DCMAKE_C_FLAGS=-g1` which reduces the amount of debug symbols and does
not enable runtime assertions.

<code>

    cat > /opt/sysroot/apertis-17.03-armhf-cross-toolchain.cmake <<EOF
    set(CMAKE_SYSTEM_NAME Linux)
    set(CMAKE_SYSTEM_PROCESSOR armhf)

    set(CMAKE_SYSROOT "/opt/sysroot/apertis/17.03/armhf")
    set(CMAKE_FIND_ROOT_PATH "/opt/sysroot/apertis/17.03/armhf")

    find_program(CCACHE_FOUND ccache)
    if(CCACHE_FOUND)
      set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)
      set_property(GLOBAL PROPERTY RULE_LAUNCH_LINK ccache)
    endif(CCACHE_FOUND)

    set(CMAKE_C_COMPILER arm-linux-gnueabihf-gcc)
    set(CMAKE_CXX_COMPILER arm-linux-gnueabihf-g++)

    set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
    set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
    set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
    set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
    EOF

    mkdir -p WebKitBuild/Debug
    cd WebKitBuild/Debug
    cmake ../.. \
      -DPORT=GTK \
      -DENABLE_SPELLCHECK=OFF \
      -DENABLE_ACCELERATED_2D_CANVAS=OFF \
      -DENABLE_GLES2=ON \
      -DENABLE_PLUGIN_PROCESS_GTK2=OFF \
      -DENABLE_TOOLS=ON \
      -DENABLE_MINIBROWSER=ON \
      -DDEVELOPER_MODE=ON \
      -DCMAKE_BUILD_TYPE=RelWithDebInfo \
      -DCMAKE_C_FLAGS_RELWITHDEBINFO=-g1 \
      -DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-g1 \
      -DCMAKE_CXX_FLAGS=-g1 \
      -DCMAKE_C_FLAGS=-g1 \
      -DCMAKE_INSTALL_PREFIX=/usr \
      -DCMAKE_INSTALL_SYSCONFDIR=/etc \
      -DCMAKE_INSTALL_LOCALSTATEDIR=/var \
      -DCMAKE_INSTALL_LIBEXECDIR=lib/arm-linux-gnueabihf \
      -DCMAKE_TOOLCHAIN_FILE=/opt/sysroot/apertis-17.03-armhf-cross-toolchain.cmake \
      -DPKG_CONFIG_EXECUTABLE=/usr/bin/arm-linux-gnueabihf-pkg-config
    make -j12

</code>

Note that you may also need to manually edit
`/usr/bin/arm-linux-gnueabihf-pkg-config` to set the `SYSROOT` to
`/opt/sysroot/apertis/17.03/armhf`. See  for more info.

# Fetch the WebKitGTK+ release branches

The WebKitGTK+ release branches are not available on the GIT mirror,
they are only available from SVN.

To fetch them in your git repository use
[`git-svn`](https://git-scm.com/docs/git-svn):

<code>

    git clone git@gitlab.apertis.org:/hmi/webkit-gtk-clutter webkit-gtk-clutter
    cd webkit-gtk-clutter
    git remote add webkit git://git.webkit.org/WebKit.git
    git fetch webkit

    git svn init --prefix=webkit/ -T trunk http://svn.webkit.org/repository/webkit
    git config --replace svn-remote.svn.fetch trunk:refs/remotes/webkit/master

    # list release branches from https://svn.webkit.org/repository/webkit/releases/WebKitGTK/
    git config svn-remote.svn.branches 'releases/WebKitGTK/{webkit-2.14,webkit-2.16}:refs/remotes/webkit/*'

    # take a walk, it will take a while while it converts from SVN to something sensible (it took ~2h here)
    git svn fetch

    # check out the WebKitGTK+ 2.14.x release branch (or do whatever with it)
    git checkout -b webkit-2.14 webkit/webkit-2.14

</code>