adding testcases for ota api network failures

test-cases/apertis-update-manager-ota-api-check-network-failure.yaml

+metadata:
+  name: apertis-update-manager-ota-check-network-failure
+  format: "Apertis Test Definition 1.0"
+  image-types:
+    minimal: [ armhf, arm64 ]
+  image-deployment:
+    - OSTree
+  type: functional
+  exec-type: manual
+  priority: medium
+  maintainer: "Apertis Project"
+  description: "Test the apertis-update-manager API for updates over the air when the network is diconnected and reconnected."
+
+  pre-conditions:
+    - >
+      This test requires a properly configured time source: when testing
+      devices not carrying a battery-backed real time clock in a network which
+      prevents connections to NTP servers (and only in that case) manually
+      ensure that the time on the device is set appropriately and that it is
+      propagated to the hardware clock so it is stay set on the next reboot if
+      the power is not plugged off.
+    - $ sudo timedatectl --adjust-system-clock set-time "2019-08-21 18:49:03" # use the appropriate date
+    - >
+      If direct access to repository with updates for DUT is restricted and proxy server should be used,
+      then need to add the address of this proxy for OSTree on DUT by command:
+    - $ sudo ostree config set 'remote "origin"'.proxy "http://10.168.128.45:3128"
+
+  resources:
+    - "The DUT u-boot environment must be clean: in u-boot, run: `env default -a` followed by `saveenv`"
+    - "A PC must be connected to DUT serial port"
+    - "The DUT must be connected to network"
+
+  expected:
+    - "CLI utility `updatectl` is able to interact with Apertis update manager even if there's a network failure in between the updates"
+    - "The OTA update was properly applied"
+
+run:
+  steps:
+    - "Check the initial deployment"
+    - $ sudo ostree admin status
+    - "Prepare the copy of commit and deploy to allow the upgrade to the same version"
+    - "Command below shows you an initial commit ID, for instance"
+    - |
+        $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
+    - "Get the Collection ID and ref"
+    -  $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
+    -  $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
+    - "Create the commit with changed timestamp to allow upgrade with recent update file"
+    - |
+        $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago"); echo "New commit: $NEWID"
+    - "Deploy the prepared commit"
+    - $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
+    - "Wait until the system is booted again and check the deployment"
+    - $ sudo ostree admin status
+    - "The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'"
+    - "Remove the initial deployment"
+    - $ sudo ostree admin undeploy 1
+    - "Reboot the system"
+    - "Check the current deployment"
+    - $ sudo ostree admin status
+    - "Start the user interface agent with mode preventing automatic system reboot after update"
+    - $ sudo updatectl --register-upgrade-handler &
+    - "Enable network updates with CLI tool"
+    - $ sudo updatectl --check-network-updates
+    - "Check that the user interface agent reports the pending update"
+    - |
+        >** Message: 21:31:26.117: Network connected: Yes
+        ** Message: 21:31:26.123: Upgrade status: Checking
+    - "Once the above message is seen disconnect the network connection to the target"
+    -  |
+        >** Message: 21:31:26:129: Network connected: No
+    - "Re-estabish the network connection to the target"
+    -  | 
+        >** Message: 21:31:28.057: Upgrade status: Downloading
+        ** Message: 21:31:51.170: Upgrade status: Deploying
+        ** Message: 21:32:11.879: An upgrade is pending
+    - "After the update, the device does *not* reboot automatically"
+    - "Check if there is pending deployment and reboot the DUT"
+    - $ sudo ostree admin status
+    - $ sudo reboot
+    - "Check the current deployment has been updated and that the rollback entry points to the prepared deployment"
+    - $ sudo ostree admin status
+    - "Start the user interface agent"
+    - $ sudo updatectl &
+    - "Enable network updates with CLI tool"
+    - $ sudo updatectl --check-network-updates
+    - "Check that the user interface agent reports the system is up to update"
+    - |
+        >** Message: 21:47:11.501: Network connected: Yes
+        ** Message: 21:47:11.508: Upgrade status: Checking
+        ** Message: 21:47:13.079: Upgrade status: Downloading
+        ** Message: 21:47:13.080: System is up to date

test-cases/apertis-update-manager-ota-api-switch-network.yaml

+metadata:
+  name: apertis-update-manager-ota-api-switch-network
+  format: "Apertis Test Definition 1.0"
+  image-types:
+    minimal: [ armhf, arm64 ]
+  image-deployment:
+    - OSTree
+  type: functional
+  exec-type: manual
+  priority: medium
+  maintainer: "Apertis Project"
+  description: "Test the apertis-update-manager API for updates over the air when the network is changed from a phone hotspot to wifi."
+
+  pre-conditions:
+    - >
+      This test requires a properly configured time source: when testing
+      devices not carrying a battery-backed real time clock in a network which
+      prevents connections to NTP servers (and only in that case) manually
+      ensure that the time on the device is set appropriately and that it is
+      propagated to the hardware clock so it is stay set on the next reboot if
+      the power is not plugged off.
+    - $ sudo timedatectl --adjust-system-clock set-time "2019-08-21 18:49:03" # use the appropriate date
+    - >
+      If direct access to repository with updates for DUT is restricted and proxy server should be used,
+      then need to add the address of this proxy for OSTree on DUT by command:
+    - $ sudo ostree config set 'remote "origin"'.proxy "http://10.168.128.45:3128"
+
+  resources:
+    - "The DUT u-boot environment must be clean: in u-boot, run: `env default -a` followed by `saveenv`"
+    - "A PC must be connected to DUT serial port"
+    - "The DUT must be connected to network"
+
+  expected:
+    - "CLI utility `updatectl` is able to interact with Apertis update manager even if there's a change in the network source"
+    - "The OTA update was properly applied"
+
+run:
+  steps:
+    - "Check the initial deployment"
+    - $ sudo ostree admin status
+    - "Prepare the copy of commit and deploy to allow the upgrade to the same version"
+    - "Command below shows you an initial commit ID, for instance"
+    - |
+        $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
+    - "Get the Collection ID and ref"
+    -  $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
+    -  $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
+    - "Create the commit with changed timestamp to allow upgrade with recent update file"
+    - |
+        $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago"); echo "New commit: $NEWID"
+    - "Deploy the prepared commit"
+    - $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
+    - "Wait until the system is booted again and check the deployment"
+    - $ sudo ostree admin status
+    - "The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'"
+    - "Remove the initial deployment"
+    - $ sudo ostree admin undeploy 1
+    - "Reboot the system"
+    - "Check the current deployment"
+    - $ sudo ostree admin status
+    - "Start the user interface agent with mode preventing automatic system reboot after update"
+    - $ sudo updatectl --register-upgrade-handler &
+    - "Enable network updates with CLI tool"
+    - $ sudo updatectl --check-network-updates
+    - "Check that the user interface agent reports the pending update"
+    - |
+        >** Message: 21:31:26.117: Network connected: Yes
+        ** Message: 21:31:26.123: Upgrade status: Checking
+    - "Once the above message is seen switch off the phone hotspot"
+    - |
+        >** Message: 21:31:26:129: Network connected: No
+    - "Re-estabish the network connection to the target via a wi-fi network or an ethernet connection"
+    - |
+        >** Message: 21:31:28.057: Upgrade status: Downloading
+        ** Message: 21:31:51.170: Upgrade status: Deploying
+        ** Message: 21:32:11.879: An upgrade is pending
+    - "After the update, the device does *not* reboot automatically"
+    - "Check if there is pending deployment and reboot the DUT"
+    - $ sudo ostree admin status
+    - $ sudo reboot
+    - "Check the current deployment has been updated and that the rollback entry points to the prepared deployment"
+    - $ sudo ostree admin status
+    - "Start the user interface agent"
+    - $ sudo updatectl &
+    - "Enable network updates with CLI tool"
+    - $ sudo updatectl --check-network-updates
+    - "Check that the user interface agent reports the system is up to update"
+    - |
+        >** Message: 21:47:11.501: Network connected: Yes
+        ** Message: 21:47:11.508: Upgrade status: Checking
+        ** Message: 21:47:13.079: Upgrade status: Downloading
+        ** Message: 21:47:13.080: System is up to date