Skip to content

Topologies

A topology is the smallest environment that represents how customers run the product. Each has its own Terraform root module, and all of them are built from the same shared modules.

Three topology shapesSide by side: a VM host topology with hosts in a public subnet, a self-managed Kubernetes topology in a private subnet behind a bastion, and a managed Kubernetes topology with a managed control plane and node groups.VM hostOne host per triggerSelf-managed K8sPrivate, via bastionManaged K8sProvider control planeNetworkPublic subnetHosts ×NImage per triggerAccessRunner IP onlyBastionOnly way inControl plane ×1Private subnetWorkers ×NOS, runtime, CNIControl planeManaged, versionedNode groupsNode image typeAdd-onsInstall methodShared network modulesShared network modulesShared network modulesTerraform root per topologyComposed modules

Which triggers each topology exercises

Trigger VM host Self-managed K8s Managed K8s
OS image and architecture ✓ ✓ (workers) ✓ (node image type)
Product version ✓ ✓ ✓
Kubernetes version ✓ ✓
Container runtime ✓
Network plugin (CNI) ✓
Install method (e.g. Helm chart or DaemonSet manifest) ✓ ✓

Choosing the topology is part of the trigger. A new OS image might run through the VM host topology and the self-managed cluster as worker nodes. A new Kubernetes version only makes sense on the cluster topologies.

VM host

The simplest shape: one host per trigger, in a small network, with the product installed at boot.

  • Network: a dedicated VPC with a public subnet. NAT is available but off by default, because it costs money and hosts don't need it.
  • Hosts: one instance per trigger, each with its own resolved image and a deliberately small size.
  • Access: SSH is allowed only from the public IP of the machine running the framework, which is looked up at plan time. Nothing else gets in.
  • Install: the product is installed by boot-time user data, so a host is under test as soon as it's ready.

This topology answers the most common question: does the product install and work on this OS version and architecture?

Self-managed Kubernetes

A minimal cluster the framework builds itself: a single control-plane node and as many workers as the triggers need, laid out the way a careful operator would run it.

  • Private by default: the control plane and workers sit in a private subnet with no public addresses. NAT is enabled for this topology, because nodes still need outbound access to pull images and reach the product's backend.
  • Bastion host: the only way in. The framework's cluster client tunnels through it to run kubectl commands and test steps.
  • Workers carry the triggers: worker OS, Kubernetes version, container runtime and network plugin all vary per run.
  • Install methods: the product is deployed each supported way, for example with a Helm chart or a raw DaemonSet manifest, because each method can fail differently.

This topology exists because self-managed clusters are where customers combine components most freely, and where support questions are hardest to answer without building one.

How the cluster is built

The cluster is bootstrapped with kubeadm, driven entirely by templated boot scripts (cloud-init user data). The triggers become template variables, so one module can build any combination:

Trigger Becomes Example values
Worker OS The image each node boots from Ubuntu, Debian, RHEL-family
Kubernetes version Pinned kubeadm, kubelet and kubectl packages A specific minor version
Container runtime Which runtime is installed and configured containerd, Docker, CRI-O
Network plugin Which CNI manifest is applied after kubeadm init Calico, Weave, Flannel, Cilium

The boot sequence:

  1. Every node installs its runtime and the pinned Kubernetes packages.
  2. The control plane runs kubeadm init, then applies the chosen network plugin.
  3. Each worker keeps retrying kubeadm join until the control plane is ready to accept it.
  4. The framework reaches the cluster by jumping through the bastion over SSH, then waits for every node to report Ready before installing the product.

A network plugin is only offered as a trigger once its install step is scripted. A request for one that isn't fails fast with a clear message, instead of building a cluster that can't schedule pods.

Gotchas worth knowing

  • Some containerd packages ship a default config with the CRI plugin disabled, so kubeadm can't talk to the runtime until that line is changed.
  • Since Kubernetes 1.24, Docker as a runtime needs the cri-dockerd shim; containerd or CRI-O work directly.
  • kubelet must never be newer than the API server, so pin both to the trigger's version rather than installing "latest".
  • Fixed sleeps between steps are fragile. Wait on real signals, such as the API server answering or nodes reporting Ready.

Managed Kubernetes

A cloud provider's managed control plane with node groups the framework defines.

  • Control plane: managed and versioned by the provider, so the Kubernetes version is a provisioning parameter.
  • Node groups: the node image type is a trigger, alongside the product version. The provider publishes these as named types, for example standard and Arm builds of its own Linux, or a container-optimised OS.
  • Install method: designed to be deployed the same ways as on the self-managed cluster.

Status

Its infrastructure modules were built. Validation was designed to reuse the suites from the other two shapes, but wasn't finished, so the managed column in the table above shows intended coverage.

What stays the same

Every topology:

  • uses the same shared modules for networking, access and key pairs;
  • receives its inputs from the same JSON contract;
  • returns its outputs (host IDs, addresses, cluster endpoint) to the framework the same way;
  • is torn down by the same orchestration step.

Adding a topology still means real work: its own root module, sometimes a new infrastructure module (the self-managed cluster needed one), and new checks for whatever is unique to it. What it doesn't need is a new framework. Networking, access, reporting and most suites carry over.