Newer
Older
# Apertis Tests Cases and Renderer
This repository contains all the Apertis test cases written using the YAML format
defined at the `format/` directory.
The repository also contains the `renderer-test-case` program, which is used to
generate the test case pages in HTML format from the YAML files.
- The renderer heavily uses `lists` of strings to parse those into the different
HTML components. Each string in a list can be started with a special character
to get a specific format if it is supported by the YAML directive containing
such a string.
For the `pre-conditions`, `notes`, `expected`, `run.steps` (only manual
tests) and `post-conditions` directives, start a line with the following
characters to give the described format:
- `$`: Command line.
- `>`: Command output.
- `@`: Attach image.
- `~`: Add a web link.
For the `run.steps` field in the automated tests, only the following formatting
rules apply:
- `#`: Add a comment.
- Everything else is a command line.
- Test cases with an `install.deps` directive will automatically add a
`pre-conditions` section to the test case page with the list of packages to
install specified in such a directive.
## Macro helpers
Some test cases share the same set of instructions, for those, some `macros` are
available: `macro_ostree_preconditions` , `macro_install_packages_preconditions`,
and `macro_modules_preconditions`. Please check the tests cases for example about
how to use them.
## Images
Test cases can contain images. These are attached to a test case page starting a
line in one of the special directives with the character `@` followed by the image
name, then this image should be placed under the `renderer/images` directory.
The main program is named `atc` in the parent directory and can be directly invoked
like:
```
The above command will show the help.
To render a single test case, use something like:
```
$ ./atc test-cases/<test-case-name>.yaml
```
That will create a `<test-case-name>.html` page.
To render a complete set of test cases inside a directory, just pass a directory
instead of a file to the command, and optionally pass a new directory path with
the `-d` flag so all the newly generated test cases are saved there instead of the
$ ./atc test-cases/ -d apertis-test-cases-v0.1/
Now all the html tests cases should be available inside the
### Renderer (parser) behaviour
- The renderer parser will always abort with a failure if a mandatory field is
missing.
- The renderer parser will abort with a failure for those fields set to values
with incorrect types.
- The renderer parser will show warning messages for unrecognized values of
fields with multiple choices (for example, image-type, execution-type).
## Tests
The module `renderer/tests.py` contains tests for the parser, renderer and
different formatting methods. There are also a set of tests files located inside
the `renderer/tests_files/` directory that are used by some of these tests.
The unittest can be executed from the parent directory with:
$ python3 -m unittest discover -v
```
It is highly recommended to execute these tests if changes are applied to any
of the `renderer` components since merge requests will only be accepted if all the
tests pass.