diff --git a/README b/README index dd461116a60d49b78ba87846fac0f4aa90456bc0..b6da68f43115b3a08386dd911777dc85e0f07174 100644 --- a/README +++ b/README @@ -1,4 +1,318 @@ -# SDK PERSISTENT WORKSPACE +# Introduction -This package allow to use an external disk to store the workspace(/home and -/opt) and a convenient mechanism to overwrite configuration files +The Apertis Persistent SDK Tool is a package that contains a command line tool +named psdk. This tool helps the developer to decouple user files from the +Apertis SDK. The idea is to use a second disk that can be attached to more than +one SDK to store: + - /home + - /opt + - Configuration files selected by the developer + +The background information about the design of psdk can be found at: + https://designs.apertis.org/private/latest/maintaining-workspace-across-sdk-updates.html + +psdk has the following options: + + user@apertis:~$ psdk -h + usage: psdk [-h] [-e] [-s] [-c] [-i] [path] + + positional arguments: + path Path to disk device node, e.g. /dev/vdb or to a + configuration file, e.g. /etc/apt/sources.list + + optional arguments: + -h, --help show this help message and exit + + Configuration files management: + -e, --etc Move a configuration file to the persistent storage. Use + this option once for each file you want to move to the + persistent storage. Usage: psdk -e /etc/configuration.file. + A backup of the original file will be at + /etc/configuration.file.PSKD. Use the original path (e.g. + /etc/configuration.file) to make changes to a file after + moving it to the persistent storage. + -s, --scan Scan persistent storage for configuration files and + configures the SDK to use each configuration file found. + + Persistent disk management: + -c, --configure Configure this SDK to use use an already initialized disk + for persistent workspace. Usage: psdk --configure /dev/sdb + (replace sdb by the correct path to the disk containing the + persistent workspace) + -i, --init Initialize a new disk for persistent workspace. Files from + /home and from /opt will be copied from this SDK. After + initializing the disk psdk will configure this SDK to use + the persistent disk. Usage: psdk --init /dev/sdb (replace + sdb by the correct path to the disk you want to use) + +# Using psdk to upgrade to a new version of the SDK + +The SDK is distributed as a VirtualBox image, and developers make changes to +adjust the SDK to their needs. These changes include installing tools, +libraries, changing system configuration files, and adding content to their +workspace. There is one VirtualBox image for each version of the SDK, and +before psdk a version upgrade required each developer to manually migrate their +SDK customization to the new version. + +psdk automates the tasks of moving the developer customization to a second +disk and configuring the SDK to use the second disk. + +## Upgrading to a new SDK using psdk (short version): +- On the old SDK: + - Add a new second disk to be used as persistent storage + - Call `psdk -i /dev/sdb` assuming sdb points to the new disk + - Reboot + - Call `psdk -e /etc/file` for each configuration file you want + on persistent storage + - Shutdown the SDK +- On the new SDK: + - Add the persistent disk you configured on the old SDK as second disk + - Call `psdk -c /dev/sdb` assuming sdb points to the new disk + - Reboot + - Call `psdk -s` to use the configuration files that are on the + persistent storage + - Reboot + +## Upgrading to a new SDK using psdk: + +ATTENTION: Do not use VirtualBox *Shareable hard disks* expert feature. btrfs +file system is not prepared to handle this feature, and using it will lead to +file system corruption and data loss. VirtualBox documentation on [Special +Write Modes](https://www.virtualbox.org/manual/ch05.html#hdimagewrites) has +more information. + +First step is to prepare the persistent disk on the old SDK. Start by adding a +second disk to the old SKD. This disk should be big enough to host the contents +of /home and /etc: We recommend using more than 40GiB and no less than 20GiB. As +we will use dynamically allocated VDI images the unused space does not consume +disk space on the host, so if in doubt, create a larger disk. + +1 - Add a second disk to the old SDK using VirtualBox 6.0.8: +- Go to Settings of the old SDK Virtual Machine +- Go to Storage +- Select Controller SATA +- Click on "Adds a new storage attachment" icon and then "Add Hard Disk" +- Choose Create new disk +- Choose VDI (VirtualBox Disk Image) and click Next +- Choose Dynamically allocated and click Next +- Choose the folder where to save the VDI file and name the disk as + apertis-persistent-sdk.vdi. For the disk size we recommend no less + than 20GiB and preferably more than 40GiB. +- Click on Create + +2 - Start the old SDK virtual machine and locate the new disk using +`sudo fdisk -l`: + + user@oldSDK:~$ sudo fdisk -l + Disk /dev/sdb: 40 GiB, 42949672960 bytes, 83886080 sectors + Disk model: VBOX HARDDISK + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + + + Disk /dev/sda: 18.6 GiB, 20000000000 bytes, 39062500 sectors + Disk model: VBOX HARDDISK + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + Disklabel type: gpt + Disk identifier: 1112CA3E-EC00-4BC6-9A47-48DF7AD20425 + + Device Start End Sectors Size Type + /dev/sda1 6176 500000 493825 241.1M EFI System + /dev/sda2 500001 19531250 19031250 9.1G Linux filesystem + /dev/sda3 19531251 39062466 19531216 9.3G Linux filesystem + +The new disk is /dev/sdb because it has no partitions and it has the same size +of the disk created on VirtualBox. + +3 - Initialize the persistent disk and configure the SDK to use the persistent +disk: + +Calling `psdk -i /dev/sdb` will prepare the persistent disk, copy the contents +of /home and /opt to the persistent disk and finally configure the SDK to use +the persistent disk. + + user@oldSDK:~$ psdk -i /dev/sdb + Initializing the persistent disk... + Will partition the persistent storage disk... + Will format the persistent storage partition... + btrfs-progs v4.20.1 + See http://btrfs.wiki.kernel.org for more information. + + Label: sdk-persistent + UUID: ce589a3a-06b6-4b04-bbf2-cb6238c61ab9 + Node size: 16384 + Sector size: 4096 + Filesystem size: 40.00GiB + Block group profiles: + Data: single 8.00MiB + Metadata: DUP 1.00GiB + System: DUP 8.00MiB + SSD detected: no + Incompat features: extref, skinny-metadata + Number of devices: 1 + Devices: + ID SIZE PATH + 1 40.00GiB /dev/sdb1 + + + Will copy contents of /home to persistent storage... + '/home/./sysroot' -> '/run/media/persistent/./sysroot' + '/home/./user' -> '/run/media/persistent/./user' + '/home/./user/Pictures' -> '/run/media/persistent/./user/Pictures' + '/home/./user/.config' -> '/run/media/persistent/./user/.config' + '/home/./user/.config/xfce4' -> '/run/media/persistent/./user/.config/xfce4' + ... + + Will copy contents of /opt to persistent storage... + '/opt/devroot' -> '/run/media/persistent/opt/devroot' + '/opt/devroot/opt' -> '/run/media/persistent/opt/devroot/opt' + '/opt/devroot/proc' -> '/run/media/persistent/opt/devroot/proc' + '/opt/devroot/tmp' -> '/run/media/persistent/opt/devroot/tmp' + '/opt/devroot/etc' -> '/run/media/persistent/opt/devroot/etc' + ... + + Configuring the SDK to use the persistent storage... + + + Please reboot the SDK for the changes to take effect. + +Reboot the SDK + user@oldSDK:~$ sudo reboot + +4 - Moving configuration files to the persistent storage + +After rebooting the SDK you will be able to move configuration files to the +persistent storage using `psdk -e /etc/file`. Let's move `/etc/cntlm.conf` to +the persistent storage as an example: + + user@oldSDK:~$ psdk -e /etc/cntlm.conf + Configuration file moved to persistent storage. From now on, changes made to + /etc/cntlm.conf will be saved on the persistent storage. + +Here is what psdk -e does: +- Copy the contents of the configuration file to the persistent storage + dereferencing symbolic links. +- Rename the configuration file to add the .PSKD extension. So in our + example the backup will be at /etc/cntlm.conf.PSKD. +- Create a symbolic link from the persistent storage to the + configuration file. + +After call `psdk -e /etc/cntlm.conf`, you can continue to use the configuration +file /etc/cntlm.conf normally. Any changes made will be saved on the persistent +storage. + +Repeat this step for each configuration file you want on the persistent +storage. Notice that `psdk -e` only accept files as argument. No directories +are allowed. + +After you are done, power off the old SDK. + +5 - On the new SDK add the persistent disk you configured on the old SDK: +- Go to Settings of the new SDK Virtual Machine +- Go to Storage +- Select Controller SATA +- Click on "Adds a new storage attachment" icon and then "Add Hard Disk" +- Click on Choose existing disk +- You will find apertis-persistent-sdk.vdi under Attached. Click on it + and then on Choose. +- Click on OK + +6 - Start the new SDK virtual machine and locate the new disk using +`sudo fdisk -l`: + + user@newSDK:~$ sudo fdisk -l + Disk /dev/sda: 18.6 GiB, 20000000000 bytes, 39062500 sectors + Disk model: VBOX HARDDISK + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + Disklabel type: gpt + Disk identifier: 1112CA3E-EC00-4BC6-9A47-48DF7AD20425 + + Device Start End Sectors Size Type + /dev/sda1 6176 500000 493825 241.1M EFI System + /dev/sda2 500001 19531250 19031250 9.1G Linux filesystem + /dev/sda3 19531251 39062466 19531216 9.3G Linux filesystem + + Disk /dev/sdb: 40 GiB, 42949672960 bytes, 83886080 sectors + Disk model: VBOX HARDDISK + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + Disklabel type: gpt + Disk identifier: 25C02522-9601-3447-9DCF-D426C27B7C75 + + Device Start End Sectors Size Type + /dev/sdb1 2048 41943006 41940959 40G Linux filesystem + +Now both disks are initialized, and /dev/sda seems to be the SDK disk. To find +out if the partition /dev/sdb1 contains the persistent storage we can call +`sudo e2label /dev/sdb1`: + + user@newSDK:~$ sudo e2label /dev/sdb1 + e2label: Bad magic number in super-block while trying to open /dev/sdb1 + /dev/sdb1 contains a btrfs file system labelled 'sdk-persistent' + +Ignore the bad magic number warning. The important part is that the partion has +the label 'sdk-persistent'. Our persistent disk is /dev/sdb.` + +7 - Configure the SDK to use the persistent disk: + +Calling `psdk -c /dev/sdb` will configure the SDK to use the persistent disk. + + user@newSDK:~$ psdk -c /dev/sdb + /sbin/e2label: Bad magic number in super-block while trying to open /dev/sdb1 + Configuring the SDK to use the persistent storage... + Created symlink /etc/systemd/system/default.target.wants/run-media-root-opt.mount → /etc/systemd/system/run-media-root-opt.mount. + Created symlink /etc/systemd/system/default.target.wants/run-media-root-home.mount → /etc/systemd/system/run-media-root-home.mount. + Created symlink /etc/systemd/system/default.target.wants/opt.mount → /etc/systemd/system/opt.mount. + + + Please reboot the SDK for the changes to take effect. + +Reboot the SDK + user@newSDK:~$ sudo reboot + +8 - Using configuration files from the persistent storage + +After rebooting the SDK you will be able to use configuration files from the +persistent storage. You need to call `psdk -s` once to scan the persistent +storage for configuration files and use them: + + user@newSDK:~$ psdk -s + Using configuration file found on persistent storage. From now on, + changes made to /etc/cntlm.conf will be saved on the persistent storage. + +Depending on which files were altered on /etc, you will need to reboot: + user@newSDK:~$ sudo reboot + +# Where are the old files and folders? + +The old /home and /opt directories are accessible from /run/media/root/. A +backup copy of configuration files that were moved to the persistent are kept +on /etc with the extension .PSDK. + +# Recovering the SDK + +If you accidentally delete the persistent disk of an SDK, the SDK will not +longer boot. To fix an SDK that does not book due missig persistent disk: + - Stop the SDK bootloader from automatically booting. To do so you need to + press space a few times. When you succeed the bootloader will show you a menu + with two options: Apertis and Reboot to Firmware Interface + - Press `e` to edit the Apertis entry. + - The initrd and root variables are important. Do not change the content of + these, but delete everything else. + - Add init=/bin/sh at the end of the line + - Press enter to boot + - Remount the filesystem with write permissions: mount -o remount,rw / + - Fixing the SDK involves editing /etc/fstab, removing .mount files from + /etc/systemd/system, and fixing files on /etc that are links to the persistent + storage. + - Call: sync + - Force-off the SDK + +Please note that even if the SDK will boot after these changes, no content from +the persistent SDK will be available as the disk is no longer available.