Skip to content

A grid of passing checks with a few failures

Support Matrix Design

A practical blueprint for automating a product's support matrix: the published list of operating systems, Kubernetes versions, container runtimes and other third-party components a product is proven to work with.

The approach suits any product that ships an agent, sensor, operator or installer into infrastructure it doesn't control. When a new third-party version appears, or someone asks for a specific combination, the framework builds a representative environment, installs the product, runs end-to-end validation, records the outcome and tears everything down.

Architecture philosophy

A support matrix is a promise to customers. Every cell should be backed by a repeatable run that anyone can trace: what was built, what was tested, what passed, and who decided.

Why this matters

A support matrix is easy to publish and hard to keep honest. Third-party components release constantly: a new OS image, a new Kubernetes minor version, a different container runtime or network plugin. Each one is a question customers will ask before they upgrade: is this supported yet?

Answering it manually is slow and inconsistent:

  • The combinations multiply. Each new OS image, cluster version or runtime adds rows faster than anyone can test them by hand.
  • Customers don't run one shape. Single hosts, clusters they run themselves and managed clusters all behave differently.
  • Nobody controls the release calendar. Upstream projects ship when they ship, and the support page quietly goes stale.
  • Standing test labs aren't free. They cost money every day and slowly stop looking like what customers run.

The core idea

Treat each new third-party version as a trigger. A Python framework owns the whole run, and Terraform builds and destroys the infrastructure underneath it.

Support matrix harnessA detected new version or a requested combination triggers a Python framework that provisions with Terraform, installs the product, validates it, reports, and tears down, producing a support verdict.New version or requestDetected, or a requested combinationPython frameworkOwns the full lifecycle of one runProvisionTerraform applyInstallProduct under testValidateEnd-to-end suitesTeardownTerraform destroyReporting runs alongside every stageSupport verdictMatrix cell: supported or notPythonTerraformInputs and steps
  1. A trigger arrives: a new version on one axis of the matrix, such as an OS image or the product's agent or CLI, is detected automatically, or a specific combination is requested by hand.
  2. The framework generates configuration: triggers become Terraform variables and test parameters. Nothing is hand-edited per run.
  3. Terraform provisions the topology: the smallest environment that represents how customers run the product.
  4. The product is installed and validated: end-to-end suites prove it installs, runs, reports in and behaves correctly.
  5. Results are published and the environment is destroyed: the outcome is recorded against a tracking ticket, and a pass flags the documentation for an update.

Key benefits

  • Repeatable: every matrix cell is backed by the same automated run, not a one-off manual check.
  • Scales with triggers: any number of triggers fan out into infrastructure and tests without code changes.
  • Cost-aware: infrastructure exists only for the life of a run and defaults to the smallest viable sizes.
  • Traceable: every run leaves an evidence trail, from what was built and which checks ran to what should change in the docs.
  • Reusable: infrastructure modules and test suites are shared across topologies.
  • Portable: the framework runs the same way in CI and on a laptop, because it owns its own artefacts.

Technology foundations

  • Python: the orchestrator, configuration generator, validation framework and reporting layer.
  • Terraform: infrastructure as code for every topology, driven by the framework rather than by hand.
  • pytest: end-to-end validation suites, parametrised from the triggers of each run.
  • Object storage: run artefacts, so any run can be inspected or replayed outside CI.
  • An issue tracker: the system of record for each run's outcome and any bugs it finds.
  • Docs and chat: Confluence or a docs repository for listings, and Slack or Teams for alerts, all through pluggable publishers (designed; the original published to the tracker and object storage).

None of these is mandatory. The patterns hold with other languages, IaC tools or trackers.

Documentation navigation

  • Design principles: the layered design and the dependency rule behind it.
  • Framework layers: how triggers become infrastructure, tests and reports.
  • Trigger sources: scheduled image checks, GitHub release watchers and other detectors that find new versions.
  • Topologies: VM hosts, self-managed Kubernetes and managed Kubernetes, and what each one proves.
  • Run lifecycle: the tracking ticket, the pass and fail paths, and support states.
  • Publishing: how results reach issue trackers, documentation and chat (Confluence, Slack, Teams).
  • Images and registries: the second family of matrix dimensions.
  • Provisioning patterns: module composition, the JSON contract, image resolution and access.
  • Cost and safety: keeping ephemeral infrastructure cheap and contained.
  • Testing the framework: static checks up to the support matrix run as the end-to-end test.
  • Looking back: what I'd keep and what I'd change if I built this again today.

Documentation scope

This repository contains documentation only. It's based on a framework I designed and built for a previous employer, rewritten here as a vendor-neutral blueprint. All examples are illustrative and use AWS syntax, where the topologies were built. No production code, product names or internal details are included.