Do not add the `component` field on branches that do not have it

This saves a few bytes for each `debian/*` and `upstream/*` branch.

Signed-off-by: Emanuele Aina <emanuele.aina@collabora.com>