Set different default storage classes for dev/prod
The default storage class for Kubernetes clusters usually has a Delete
reclaim policy, and it is indeed the case for the do-block-storage
class used on DigitalOcean clusters. This is largely a reflection of
the Kubernetes' original focus on stateless workloads.
That's a scary default for production cluster with stateful workloads
such as the ones we employ, for which we definitely want to opt-in to
the Retain reclaim policy as it prevents accidental data deletion when
a PVC (or even the PV) is deleted.
On DigitalOcean that need is best served by the
do-block-storage-retain storage class.
However, on the development cluster there should never be any valuable
data and we are more concerned about not leaving costly leftovers behind
after creating and destroying services, so the Delete reclaim policy
is definitely a more suitable option.
This commit adds a variable to each inventory to configure the most stuitable storage class. Each playbook should opt-in to it for their persistent workloads.
This avoids changing the default storage class as explicitly opting-in is preferable to relying on implicit global state.