Once upon a time, when Apertis was still using live-build, scripts
had to indicate in their extension if they had to be run in the chroot
environment or outside of it.

With Debos this is no longer needed, so drop the weird extensions that
were cargo-culted in this repository. :)

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The current SDK theme is a mix of clashing colours (highlighting in green,
deep purple background). Apply "xfce-4.6" theme to chrome, add custom
Apertis wallpaper.

Signed-off-by: Martyn Welch <martyn.welch@collabora.co.uk>

This pass the release version during test execution because it is needed
by lava-submit.

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

The release string in the profile name for lava jobs is unneeded.

Signed-off-by: Ritesh Raj Sarraf <ritesh.sarraf@collabora.com>

The mildenhall themes are located in /var/lib/MILDENHALL_extensions.
Move this directory to /usr/share when building OSTree images.
Add a symbolic link so that unmodified binaries can still run.

Apertis: https://phabricator.apertis.org/T5362


Signed-off-by: Frédéric Dalleau <frederic.dalleau@collabora.com>

Signed-off-by: Ritesh Raj Sarraf <rrs@debian.org>

Since e1f04718, recipes add .tar.gz themselves, accepting a generic basename
instead. Sysroot generation initially didn’t take that into account, with
337970b9 fixing that issue. However, the sysroot conf file was still broken
as the .tar.gz suffix wasn’t explicitly included.

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Commit e1f04718 broke sysroot tarball name generation, resulting
in .tar.gz.tar.gz files.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The default demopack installs media files in `/home/shared`: by
including them only in the ospack they are unavailble on the ostree
images as the whole `/home` gets filtered out during `ostree commit`.

Moving the extraction of the demopack when the full filesystem gets
deployed to the images makes the media files available on ostree images
as well.

See https://phabricator.apertis.org/T5313



Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

APT-based images use the `general_storage` partition to back the storage
for `/home`: do the same in the OSTree recipes.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 6 years ago and Frédéric Dalleau committed 6 years ago

Export the list of packages installed in the final artifact just before
removing the dpkg database.

This will allow to quickly check the artifacts contents without
unpacking and processing them.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 6 years ago and Frédéric Dalleau committed 6 years ago

Dump and store the list of files available on the images just before
finalizing them, so developers do not need to run the images just to see
what they ship.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 6 years ago and Frédéric Dalleau committed 6 years ago

Split the filelist extraction from the pkglist as it may be useful to
insert a cleanup phase between the two.

Also rework the pkglist extraction to capture the whole dpkg status file
rather than only a subset of it, in order to preserve as much
information as possible.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Emanuele Aina authored 6 years ago and Frédéric Dalleau committed 6 years ago

Make ospack and sysroot recipes take a generic basename for the output
artifacts, to which the `.tar.gz` file extension is appended by the
recipe itself. The recipes for the images already works this way as it
works better when generating multiple artifacts.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Deploying the images.apertis.org website only requires the copy of a
couple of static files, which the sshPublisher can do easily.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Andrew Lee (李健秋) authored 6 years ago and Emanuele Aina committed 6 years ago

According to the Jenkins documentation[1] the `when {branch 'master'}`
conditional only works on multibranch jobs.

[1] https://jenkins.io/doc/book/pipeline/syntax/#when



Since this is not meant to be a multibranch job the upload step is
always skipped:

   Stage "Upload" skipped due to when conditional

Replace the conditional with a check on the `GIT_BRANCH` environment
variable so it works properly.

Signed-off-by: Andrew Lee (李健秋) <andrew.lee@collabora.co.uk>

Ensure that ostree repositories are published just like the other
artifacts. Push the whole repository if one doesn't exist already.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Testing the upload logic and having a place to fetch the artifacts
without having to hack the job definition is definitely helpful.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Make sure the destination folders exist before uploading there.
This should make running the job from an empty slate work properly.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

To handle the case where we're creating a new OSTree branch the current
code ignores the `ostree pull` exit code.

Somewhat unsurprisingly, it's not a great idea. Issues like network
timeouts or any other random incident may cause it to fail, producing an
inconsistent state until `ostree-push` fails when trying to push the
newly created commit to the remote refs because they don't have any
parent in common (the new commit will have the empty parent since we
failed to pull the actual ref head).

  Traceback (most recent call last):
    File "/usr/bin/ostree-push", line 588, in <module>
      exit(push_main())
    File "/usr/bin/ostree-push", line 561, in push_main
      return pusher.run()
    File "/usr/bin/ostree-push", line 426, in run
      needed = self.needed_commits(revs[0], revs[1], commits)
    File "/usr/bin/ostree-push", line 361, in needed_commits
      'commit %s' %(remote, local))
  __main__.PushException: Remote commit d00fbf17cafe4028458ffda0795512bfa21fee083596ed6fbdd624a2ff539f66 not descendent of commit bd9a0f33ec50062810c0f5e829c49cb707034e0c861938874149438da0d41999

To avoid that, manually do a pre-flight HTTP request to check if the
ref exists, and yield an error for any HTTP code other than 200 and 404.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Generating a local OSTree summary made sense when we used rsync to push
the repository, but now that we moved to ostree-push it's no longer
useful.

Currently ostree-push does not support updating the summary at all, so
let's drop it altogether.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

SSH complains loudly if the current user is not in the NSS user
database, so we build a fake database to make it happy.
Since it is used in couple of places, refactor that in a function so it
lives in a single place.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Rename the `ostreelocal` variable to `ostree` to be consistent with
other recipes.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

The `ostree-convert-from-debian` script is splitting the vmlinuz name to
obtain the version and is doing this after concatenating it with the path.
This works OK when run with in a docker environment as the path doesn't
contain a `-`, which the script is splitting on. However, if run in the
SDK (where we can't use docker) there's a possibility that the path will
also contain that character (a strong chance given our recipe repository
convention), resulting in the version number being detected completely
wrong, enough so to cause images to fail to boot.

Use the vmlinuz name without the path prepended to work out the version.

Signed-off-by: Martyn Welch <martyn.welch@collabora.co.uk>

Newlines are perfectly fine to end commands in the triple-quoted shell
snippets. Jenkins will also run the commands with `sh -xe`, so the
script will automatically fail as soon as a command fails and we don't
need to do anything special like sprinkling `&&` all over.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Today uploadDirectory() gets only called in places already guarded by
`if production`, so checking for that inside the function itself is
redundant and misleading.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Tweak formatting to consistently indent command line parameters when
they are split over multiple lines.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Get rid of some unneeded newline escaping in triple-quoted strings.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

VDI images are meant for VirtualBox and having a bmap file along with
them is not useful at all as bmaptool does not understand the VDI file
format and cannot identify unused blocks.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

This commit removes the line that changed the amd64 tests jobs
to run in the qemu devices, and now they will run in the device
specified by the lqa profile (minnowboard for uefi jobs and qemu
for the sdk jobs).

Signed-off-by: Luis Araujo <luis.araujo@collabora.co.uk>

Dash is great as a system shell, but it's pretty horrible as a user
shell as there is no tab completion or history. There are things that
the busybox ash applet does provide making it a lot more pleasant to
use, while only less then 500 kilobytes to the ospack.

As an added benefit this gives easy acces to the wget/nc/ip busybox
applets.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

Commit 921c72f1 decoupled ospack building from image buiding, but a
typo caused the pipeline to only build the ospack for the SDK and not
the image.

Flip the flag so we build it again.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>

Developers would benefit from pregenerated listings to avoid downloading
and unpacking all the ospacks just to check the version of a shipped
package or the availability of a file.

Being able to rely on pregenerated package and file listings would help
speed up the checks needed by T5042 to identify which images are
affected.

Signed-off-by: Andrew Lee (李健秋) <andrew.lee@collabora.co.uk>

Mark the pipeline as failed if the build of any artifact failed.
try/catch pairs for build stages preventing from correct status
set for the whole pipeline.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Add "--scratchsize 10G" option for debos to avoid the oom-killer.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

New versions of fakemachine drop the bios, kernel and system boot
messages to give a more pleasant user experience. However for running in
CI especially when there are sometimes kernel issues in fakemachine this
is not the right thing.  So turn back on the boot messages.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>