README.md

# Apertis Tests Cases and Renderer

This repository contains all the Apertis test cases written using the YAML format
defined at the `format/` directory.

The repository also contains the `renderer-test-case` program, which is used to
generate the test case pages in HTML format from the YAML files.

## Format Rules/Notes

- The renderer heavily uses `lists` of strings to parse those into the different
  HTML components. Each string in a list can be started with a special character
  to get a specific format if it is supported by the YAML directive containing
  such a string.

  For the `pre-conditions`, `notes`, `expected`, `run.steps` (only manual
  tests) and `post-conditions` directives, start a line with the following
  characters to give the described format:

    - `$`: Command line.
    - `>`: Command output.
    - `@`: Attach image.
    - `~`: Add a web link.

  For the `run.steps` field in the automated tests, only the following formatting
  rules apply:

    - `#`: Add a comment.
    - Everything else is a command line.

- Test cases with an `install.deps` directive will automatically add a
  `pre-conditions` section to the test case page with the list of packages to
  install specified in such a directive.

## Macro helpers

Some test cases share the same set of instructions, for those, some `macros` are
available: `macro_ostree_preconditions` , `macro_install_packages_preconditions`,
and `macro_modules_preconditions`. Please check the tests cases for example about
how to use them.

## Images

Test cases can contain images. These are attached to a test case page starting a
line in one of the special directives with the character `@` followed by the image
name, then this image should be placed under the `renderer/images` directory.

## Renderer

The main program is named `atc` in the parent directory and can be directly invoked
like:

```
$ ./atc --help
```

The above command will show the help.

To render a single test case, use something like:

```
$ ./atc test-cases/<test-case-name>.yaml
```

That will create a `<test-case-name>.html` page.

To render a complete set of test cases inside a directory, just pass a directory
instead of a file to the command, and optionally pass a new directory path with
the `-d` flag so all the newly generated test cases are saved there instead of the
current directory:

```
$ ./atc test-cases/ -d apertis-test-cases-v0.1/
```

Now all the html tests cases should be available inside the
`apertis-test-cases-v0.1/` directory.

### Renderer (parser) behaviour

- The renderer parser will always abort with a failure if a mandatory field is
  missing.

- The renderer parser will abort with a failure for those fields set to values
  with incorrect types.

- The renderer parser will show warning messages for unrecognized values of
  fields with multiple choices (for example, image-type, execution-type).

## Tests

The module `renderer/tests.py` contains tests for the parser, renderer and
different formatting methods. There are also a set of tests files located inside
the `renderer/tests_files/` directory that are used by some of these tests.

The unittest can be executed from the parent directory with:

```
$ python3 -m unittest discover -v
```

It is highly recommended to execute these tests if changes are applied to any
of the `renderer` components since merge requests will only be accepted if all the
tests pass.