Skip to content

Looking back

What I'd keep

  • The framework owns the run. Python decides when to provision, test, pause and destroy; Terraform is a tool it calls.
  • Triggers as the unit of work. Each run tests only what changed.
  • Fan-out within a run. Ten OS images become ten small hosts, built and validated in parallel (diagram).
  • A ticket as the system of record. Every run explains itself.
  • Failed environments kept for investigation. We could log in, read the agent's logs, and compare them with what reached the data backend.
  • Secrets from a secret store. Tokens came from the CI provider or a cloud secrets manager, never from the repository.

What I'd change

  • Readiness checks, not sleeps. Fixed pauses in boot scripts were the main source of flaky cluster builds.
  • Releases as events. Subscribe to vendor feeds and "latest image" pointers instead of scheduled checks.
  • The matrix as data. Generate the documentation from a queryable store, so publishing is a review of a generated change.

The lesson

The hard part wasn't Terraform or the tests. It was defining what "supported" means and making that definition executable.