Apertis Tests Cases and Renderer

This repository contains all the Apertis test cases written using the YAML format defined at the format/ directory.

The repository also contains the renderer-test-case program, which is used to generate the test case pages in HTML format from the YAML files.

Format Rules/Notes

• The renderer heavily uses lists of strings to parse those into the different HTML components. Each string in a list can be started with a special character to get a specific format if it is supported by the YAML directive containing such a string.

  For the pre-conditions, notes, expected, run.steps (only manual tests) and post-conditions directives, start a line with the following characters to give the described format:

  • $: Command line.
  • >: Command output.
  • @: Attach image.
  • ~: Add a web link.

  For the run.steps field in the automated tests, only the following formatting rules apply:

  • #: Add a comment.
  • Everything else is a command line.

• Test cases with an install.deps directive will automatically add a pre-conditions section to the test case page with the list of packages to install specified in such a directive.

Macro helpers

Some test cases share the same set of instructions, for those, some macros are available: macro_ostree_preconditions , macro_install_packages_preconditions, and macro_modules_preconditions. Please check the tests cases for example about how to use them.

Images

Test cases can contain images. These are attached to a test case page starting a line in one of the special directives with the character @ followed by the image name, then this image should be placed under the renderer/images directory.

Renderer

The main program is named atc in the parent directory and can be directly invoked like:

$ ./atc --help

The above command will show the help.

To render a single test case, use something like:

$ ./atc test-cases/<test-case-name>.yaml

That will create a <test-case-name>.html page.

To render a complete set of test cases inside a directory, just pass a directory instead of a file to the command, and optionally pass a new directory path with the -d flag so all the newly generated test cases are saved there instead of the current directory:

$ ./atc test-cases/ -d apertis-test-cases-v0.1/

Now all the html tests cases should be available inside the apertis-test-cases-v0.1/ directory.

Renderer (parser) behaviour

• The renderer parser will always abort with a failure if a mandatory field is missing.

• The renderer parser will abort with a failure for those fields set to values with incorrect types.

• The renderer parser will show warning messages for unrecognized values of fields with multiple choices (for example, image-type, execution-type).

Tests

The module renderer/tests.py contains tests for the parser, renderer and different formatting methods. There are also a set of tests files located inside the renderer/tests_files/ directory that are used by some of these tests.

The unittest can be executed from the parent directory with:

$ python3 -m unittest discover -v

It is highly recommended to execute these tests if changes are applied to any of the renderer components since merge requests will only be accepted if all the tests pass.