> For the complete documentation index, see [llms.txt](https://experience-digital-transform.gitbook.io/experience-digital-transformation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://experience-digital-transform.gitbook.io/experience-digital-transformation/cloud-for-dpi/chapter-4/practical-path-on-openstack.md). # Practical path on OpenStack The worked path discovers OpenStack, installs CAPO and supplies the values accepted by its ClusterClass. Creation, scaling and deletion then follow CAPI's generic lifecycle. The complete runnable package is under [`chapter-4/lab`](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/README.md); the sections below retain the commands and observations needed to understand it. The helpers use Bash on Linux `amd64`. Windows users must run the complete toolchain inside **WSL 2** rather than mix PowerShell and WSL paths. Native Windows and macOS are unvalidated. The guarded path assumes Unix permissions and the pinned Linux `clusterctl` binary. [\[33\]](https://learn.microsoft.com/windows/wsl/install) The practical blocks load paths and context names once from an ignored session file. Confirmations remain invocation-local. ```bash cd "$(git rev-parse --show-toplevel)/chapter-4/lab" cp lab-session.env.example lab-session.env # Replace OS_CLOUD and repository values, review the context names, then load once. . ./lab-session.env printf '%s\n' "$OS_CLOUD" "$MANAGEMENT_CONTEXT" "$SERVICE_A_CONTEXT" "$LOADBALANCER_CONTEXT" ``` Representative success output (the profile name is environment-specific): ``` kubernetes-admin@kubernetes service-a-admin@service-a loadbalancer-admin@loadbalancer ``` ## 1. Discover the IaaS environment Automation repeats its inputs, including incorrect ones. The first task is therefore to identify the OpenStack project, quota and current use, approved images, control-plane and worker flavors, compute availability zone, networks, subnets, routes and security rules. Node, Pod, Service and transit ranges also have to be non-overlapping. The initial queries expose those choices without creating resources: ```bash # Inspect the target selected once in lab-session.env. openstack --os-cloud "$OS_CLOUD" quota show -f yaml openstack --os-cloud "$OS_CLOUD" image list --status active openstack --os-cloud "$OS_CLOUD" flavor list openstack --os-cloud "$OS_CLOUD" availability zone list --compute openstack --os-cloud "$OS_CLOUD" network list openstack --os-cloud "$OS_CLOUD" subnet list ``` The commands above should leave us with unique identifiers and state, not only familiar names. This shortened output shape shows the fields to retain; angle-bracket values are environment-specific values, not reference-lab evidence: Representative success output: ``` $ openstack --os-cloud "$OS_CLOUD" image list --status active +--------------+------------------------------+--------+ | ID | Name | Status | +--------------+------------------------------+--------+ | | | active | +--------------+------------------------------+--------+ $ openstack --os-cloud "$OS_CLOUD" availability zone list --compute +-----------+-------------+ | Zone Name | Zone Status | +-----------+-------------+ | | available | +-----------+-------------+ $ openstack --os-cloud "$OS_CLOUD" network list +----------------+---------------------------+--------+ | ID | Name | Status | +----------------+---------------------------+--------+ | | | ACTIVE | +----------------+---------------------------+--------+ ``` The selected UUIDs and ranges go into a local **environment contract**, stored as `environment-contract.yaml`. A shortened entry connects those read-backs to one child cluster: ```yaml openstack: image_id: control_plane_flavor: worker_flavor: availability_zone: clusters: - name: service-a network_id: subnet_id: node_cidr: 10.41.0.0/24 pod_cidr: 10.241.0.0/16 service_cidr: 10.97.0.0/16 api_vip: 10.41.0.10 ```
The management control path crosses private API addresses, while Pod, Service, Node and transit ranges remain distinct and routable by design.

Figure 7. Network ranges and routes are part of the cluster contract and must be chosen before rollout.

**OpenStack application credentials** should be scoped to the project and automation task; their secret value is neither printed nor committed. The contract validator rejects missing IDs and overlapping CIDRs before the read-only preflight compares those values with OpenStack: ```bash # Create a local contract, replace every REPLACE_ value, and validate it. cp environment-contract.example.yaml environment-contract.yaml python3 scripts/preflight.py environment-contract.yaml ./scripts/preflight-readonly.sh environment-contract.yaml "$OS_CLOUD" ``` When all live values are present and the CIDRs do not overlap, the local validator ends with this deterministic line: Representative success output: ``` environment contract valid (live mode): loadbalancer, service-a; CIDRs do not overlap ``` Creation begins only when every required value is explicit and the validator succeeds. [\[10\]](https://docs.openstack.org/keystone/latest/user/application_credentials.html)[\[11\]](https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/common/index.html#quota-show)[\[22\]](https://www.rfc-editor.org/rfc/rfc1918) The complete discovery and evidence path is in [Walkthrough 1](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/01-discovery.md); [C4-P1](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p1-discovery-turn-cloud-facts-into-an-environment-contract) shows the same decision gate. ## 2. Create the seed VM foundation The seed cluster starts with a virtual machine, and **OpenTofu** creates the IaaS resources around it. Chapter 3 introduces OpenTofu, its plan/apply workflow and its state boundary in [From manual work to automation](/experience-digital-transformation/cloud-for-dpi/chapter-3/from-manual-work-to-automation.md#from-manual-work-to-automation). The Chapter 4 module applies that workflow to a private management network, router, fixed ports, security groups, one key pair, the seed VM and a small **jump host**. [\[21\]](https://opentofu.org/docs/intro/core-workflow/) The reference lab requires a jump host because it assumes the operator has no direct private access to the management network. The jump host receives the only **floating IP**; the seed VM keeps a fixed private address and is reached through SSH `ProxyJump`. This is a property of the worked environment, not a Kubernetes or CAPI requirement. An environment with an approved VPN, private operator network or another direct administration path can omit the jump host and adapt the OpenTofu module and Ansible inventory accordingly. OpenTofu stores infrastructure intent as code and separates planning from application. This lets us inspect public exposure, replacement and deletion before OpenStack changes. The same sequence creates the seed foundation here: ```bash # Validate the declaration, save the proposed change, review it, then apply it. tofu init tofu fmt -check tofu validate tofu plan -out=seed.tfplan tofu show seed.tfplan tofu apply seed.tfplan # Save the declared output values for inventory and later identity checks. mkdir -p ../../../artifacts tofu output -json > ../../../artifacts/seed-outputs.json ``` Representative success output: ``` OpenTofu has been successfully initialized! Success! The configuration is valid. Plan: to add, 0 to change, 0 to destroy. Apply complete! Resources: added, 0 changed, 0 destroyed. ``` The JSON records each output's type, sensitivity flag and value. Environment-specific IDs and addresses differ, but the saved file has this shape: ```json { "seed_private_ip": { "sensitive": false, "type": "string", "value": "" }, "seed_server_id": { "sensitive": false, "type": "string", "value": "" }, "gateway_security_group_id": { "sensitive": false, "type": "string", "value": "" } } ``` Those values support two separate read-back checks. First, `seed_server_id` must identify an `ACTIVE` Nova server whose attached Neutron port carries `seed_private_ip`. This proves that OpenTofu created the seed on the intended private network. Second, `gateway_security_group_id` identifies the rules on the reference lab's public jump host; its SSH ingress rule must name the reviewed administration-source CIDR rather than the whole Internet. This proves the jump-host rule boundary, not that every network path is reachable. ```bash # Compare the saved identities with the live Nova and Neutron resources. SEED_ID=$(tofu output -raw seed_server_id) SEED_PRIVATE_IP=$(tofu output -raw seed_private_ip) GATEWAY_SECURITY_GROUP_ID=$(tofu output -raw gateway_security_group_id) openstack --os-cloud "$OS_CLOUD" server show "$SEED_ID" -f json openstack --os-cloud "$OS_CLOUD" port list --server "$SEED_ID" \ --fixed-ip "ip-address=$SEED_PRIVATE_IP" -f json openstack --os-cloud "$OS_CLOUD" security group rule list \ "$GATEWAY_SECURITY_GROUP_ID" -f json ``` Representative success output: ``` {"id":"","status":"ACTIVE"} [{"id":"","fixed_ips":["10.42.0.20"]}] [{"protocol":"tcp","port_range_min":22,"remote_ip_prefix":""}] ``` [Walkthrough 2](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/02-seed-infrastructure.md) contains the input-file preparation and the same identity checks; [C4-P2](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p2-seed-infrastructure-review-and-apply-the-opentofu-plan) follows the plan review and apply. ## 3. Initialize the seed cluster The new VM still needs to become a Kubernetes machine. **Ansible** connects to the selected host and applies repeatable configuration without requiring us to execute every command by hand. An **Ansible playbook** is a versioned YAML document containing one or more ordered plays. A play selects hosts from the inventory, chooses whether privilege escalation is needed, and runs tasks. Each task calls a module such as `ansible.builtin.apt` for packages or `ansible.builtin.template` for a rendered file. Modules normally describe the required end state, so a repeated run can report that a correct setting needs no change. [\[34\]](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) This shortened excerpt comes from the seed playbook used by the lab. Intervening host-preparation tasks are omitted, but the shown lines are unchanged: ```yaml - name: Configure and initialise the Chapter 4 seed cluster hosts: management-seed become: true tasks: - name: Install container and repository prerequisites ansible.builtin.apt: name: - ca-certificates - curl - gpg - containerd state: present update_cache: true - name: Render the complete seed-cluster definition ansible.builtin.template: src: kubeadm-config.yaml.j2 dest: /etc/kubernetes/kubeadm-config.yaml mode: '0600' ``` `hosts` connects the play to `management-seed` in the inventory. `become: true` permits the system changes. The package task declares what must be installed, while the template task combines reviewed variables with the kubeadm template. The complete playbook also waits for cloud-init, checks DNS and time synchronisation, configures the container runtime and kernel, disables swap, installs the pinned Kubernetes tools, initializes the API and installs Calico. Ansible then uses kubeadm with the rendered configuration to initialize the first Kubernetes API. The API address and the Pod and Service ranges come from the environment contract. kubeadm creates the control plane but intentionally leaves Pod networking to a CNI implementation. [\[8\]](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)[\[17\]](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)[\[20\]](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/) An **Ansible inventory** maps a logical host name to its connection data and groups. The reference inventory names `management-seed`, sets its private `ansible_host`, selects the remote `ansible_user`, and adds `ansible_ssh_common_args` for the lab's `ProxyJump`. With direct private access, we omit that last field. [\[32\]](https://docs.ansible.com/ansible/latest/inventory_guide/intro_inventory.html) ```yaml all: hosts: management-seed: ansible_host: 10.42.0.20 ansible_user: ubuntu ansible_ssh_common_args: -o ProxyJump=ubuntu@ ``` We inspect the parsed inventory before running the playbook. The first command below produced the shown host graph from the checked-in example inventory: ```bash # Review the parsed inventory and proposed tasks before configuring the seed. ansible-inventory -i inventory.yaml --graph ansible-playbook -i inventory.yaml seed.yaml --syntax-check ansible-playbook -i inventory.yaml seed.yaml --check --diff ansible-playbook -i inventory.yaml seed.yaml ``` Representative success output: ``` @all: |--@ungrouped: | |--management-seed ``` For Pod networking, we use **Calico**, a widely deployed CNI implementation that combines routing with network-policy enforcement. The **Tigera operator** installs and manages Calico through Kubernetes resources and therefore has to be installed before the Calico `Installation` declaration can be applied. The playbook downloads the pinned operator manifest, verifies its checksum, waits for its API, and applies the reviewed Pod range. [\[17\]](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)[\[23\]](https://docs.tigera.io/calico/latest/getting-started/kubernetes/quickstart) The resulting **kubeconfig** contains the API endpoint, cluster trust data and client identity. In the reference lab it is copied through the jump host because the seed has no public address. An operator with direct private access copies it directly through the approved administration path. In either case, the local file is created with restrictive permissions and is never printed. The following checks use that file to establish whether the cluster is actually usable: ```bash # Inspect the copied kubeconfig before using its reviewed session context. test -r "$KUBECONFIG" kubectl --kubeconfig "$KUBECONFIG" config get-contexts kubectl --kubeconfig "$KUBECONFIG" config view --minify \ -o jsonpath='{.clusters[0].cluster.server}{"\n"}' kubectl --kubeconfig "$KUBECONFIG" config current-context printf 'Reviewed management context: %s\n' "$MANAGEMENT_CONTEXT" ./scripts/require-context.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" kubectl --kubeconfig "$KUBECONFIG" cluster-info kubectl --kubeconfig "$KUBECONFIG" get nodes -o wide kubectl --kubeconfig "$KUBECONFIG" get pods -A kubectl --kubeconfig "$KUBECONFIG" get tigerastatus ``` The checked-in kubeadm configuration produces `kubernetes-admin@kubernetes`; verify that `current-context` reports this name and that the selected server is the seed API. If cluster or user names differ, update `MANAGEMENT_CONTEXT` in the ignored session file with the reviewed value. The guard fails on a mismatch. [\[20\]](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/) A successful result has a responding API, a seed Node in `Ready`, running CoreDNS Pods, and healthy Calico components. Volatile hashes and ages vary, but the successful output has this shape: Representative success output: ``` $ kubectl --kubeconfig "$KUBECONFIG" get nodes NAME STATUS ROLES AGE VERSION management-seed Ready control-plane v1.36.3 $ kubectl --kubeconfig "$KUBECONFIG" -n kube-system get pods -l k8s-app=kube-dns NAME READY STATUS RESTARTS AGE coredns- 1/1 Running 0 coredns- 1/1 Running 0 $ kubectl --kubeconfig "$KUBECONFIG" get tigerastatus NAME AVAILABLE PROGRESSING DEGRADED SINCE apiserver True False False calico True False False ippools True False False ``` `Pending`, repeated restarts, `AVAILABLE=False`, `DEGRADED=True` or a Node in `NotReady` require diagnosis before lifecycle controllers are installed. The exact reference-lab `ProxyJump` copy, its direct-access alternative, the context guard and checks are in [Walkthrough 3](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/03-seed-cluster.md), with the visible sequence in [C4-P3](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p3-seed-cluster-configure-the-vm-and-initialize-kubernetes). ## 4. Configure repeatable child clusters With the Management cluster up and running, the CAPI, kubeadm, CAPO and ORC controllers can now be installed. Their jobs were introduced before the architecture, so this step focuses on connecting them to the prepared image and local OpenStack environment. Versions and checksums are pinned in `VERSIONS.yaml`. ```bash # Verify the Management target loaded from lab-session.env. test -r "$KUBECONFIG" kubectl --kubeconfig "$KUBECONFIG" config get-contexts kubectl --kubeconfig "$KUBECONFIG" config view --minify \ -o jsonpath='{.clusters[0].cluster.server}{"\n"}' kubectl --kubeconfig "$KUBECONFIG" config current-context ./scripts/require-context.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" # Install the pinned providers into that confirmed Management context. CONFIRM_MANAGEMENT_CONTEXT="$MANAGEMENT_CONTEXT" \ ./scripts/install-providers.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" CONFIRM_MANAGEMENT_CONTEXT="$MANAGEMENT_CONTEXT" \ ./scripts/install-management-ingress.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" kubectl --kubeconfig "$KUBECONFIG" get providers -A -o wide kubectl --kubeconfig "$KUBECONFIG" get deployments -A ``` Representative success output: ``` credential secret service-a/ applied without printing its content NAME TYPE VERSION provider/cluster-api CoreProvider provider/infrastructure-openstack InfrastructureProvider deployment "contour" successfully rolled out ``` The provider list should contain the CAPI core provider, the kubeadm bootstrap and control-plane providers, and the OpenStack infrastructure provider. Their controller Deployments must be Available before they can reconcile a Cluster request. **cert-manager** is a Kubernetes controller that issues and renews X.509 certificates from configured issuers. `clusterctl` installs and owns its Management-cluster instance because CAPI and provider admission webhooks need TLS certificates. We reuse that instance instead of installing a duplicate. Each child cluster receives its own cert-manager controllers through a ClusterResourceSet, so certificate resources and resulting Secrets remain inside that cluster. Installing the controllers does not select a trust source: Chapter 5 configures the approved Issuers, DNS validation, public certificates and renewal policy. [\[35\]](https://cert-manager.io/docs/)[\[37\]](https://cluster-api.sigs.k8s.io/clusterctl/commands/init.html#cert-manager) CAPO reads OpenStack access from namespaced **Kubernetes Secrets** created from a protected `clouds.yaml`; the secret name is part of the environment contract. Those credentials allow CAPO to create infrastructure but never belong in a Cluster manifest or Git repository. A **ClusterClass** is a reusable blueprint. It names the templates for the infrastructure cluster, control plane and worker machines, then defines which values each Cluster may supply. A shortened view of the checked-in class shows those relationships: ```yaml # Shortened teaching excerpt, not a complete manifest. kind: ClusterClass metadata: name: openstack-runtime spec: infrastructure: templateRef: {kind: OpenStackClusterTemplate, name: openstack-runtime-cluster} controlPlane: templateRef: {kind: KubeadmControlPlaneTemplate, name: openstack-runtime-control-plane} workers: machineDeployments: - class: general-worker infrastructure: templateRef: {kind: OpenStackMachineTemplate, name: openstack-runtime-worker-machine} variables: - {name: networkID, required: true} - {name: imageID, required: true} - {name: workerFlavor, required: true} ``` A **Cluster** request selects that class and supplies the values for one concrete cluster. The class contains the repeatable structure; the request contains the environment and role-specific choices: ```yaml # Shortened teaching excerpt from service-a.yaml. kind: Cluster metadata: name: service-a spec: topology: classRef: {name: openstack-runtime, namespace: capi-lab-blueprints} version: v1.36.3 workers: machineDeployments: - {class: general-worker, name: workers, replicas: 1} variables: - {name: networkID, value: 00000000-0000-0000-0000-000000000020} - {name: imageID, value: 00000000-0000-0000-0000-000000000001} ``` The all-zero UUIDs above are documentation placeholders. The renderer replaces them with the approved local contract. ### Cluster-wide add-ons, ingress and storage A new Kubernetes cluster can run Pods after its CNI is ready, but it still needs certificate, ingress and storage controllers before it exposes and persists typical applications. A **StorageClass** names a provisioner and its binding policy. A **PersistentVolumeClaim (PVC)** asks that provisioner for storage without requiring the workload to know the underlying disk API. [\[30\]](https://kubernetes.io/docs/concepts/storage/storage-classes/) A **ClusterResourceSet** is a CAPI add-on object stored in the Management cluster. It selects child Clusters by label and applies Kubernetes resources held in a referenced ConfigMap or Secret when their APIs become reachable. With `strategy: Reconcile`, CAPI also reapplies an updated resource payload. The ClusterResourceSet distributes the add-on declaration; it does not store application data or provide storage by itself. [\[29\]](https://cluster-api.sigs.k8s.io/tasks/cluster-resource-set) Before child creation, the renderer writes three ClusterResourceSets into its Management namespace. CAPI applies them after the child API and CNI become reachable. Management uses another path: `clusterctl` owns cert-manager, while the guarded helper installs private Contour/Envoy. | Add-on | Chapter 4 validation | Configuration retained for Chapter 5 | | -------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | | cert-manager `v1.20.3` | CRDs and the controller, cainjector and webhook Deployments are ready | Issuers, trust roots, DNS validation, Certificate resources and renewal policy | | Contour/Envoy `v1.33.7` | Contour and Envoy are ready, Envoy remains private, and a request crosses the workload ingress | Public edge exposure, production routes, TLS policy, DNS and capacity | | Local Path Provisioner `v0.0.37` | A PVC binds and one Pod writes and reads its marker | Cinder CSI, Ceph CSI or another durable integration, plus backup and recovery | The cert-manager manifest is split across several ConfigMaps because one Kubernetes API object must remain below its size limit. One ClusterResourceSet references all parts and selects only Clusters carrying the matching certificate-baseline label. Contour uses a separate ingress-baseline label. This keeps add-on choice explicit even though the two example child clusters select both baselines. [\[29\]](https://cluster-api.sigs.k8s.io/tasks/cluster-resource-set)[\[35\]](https://cert-manager.io/docs/)[\[38\]](https://projectcontour.io/getting-started/) The Contour payload exposes Envoy through a private `ClusterIP` Service. This gives the LoadBalancer cluster a routed target inside each workload cluster without creating a public endpoint for every spoke. Chapter 5 can change only the LoadBalancer cluster's edge exposure while retaining private workload ingress endpoints. The storage payload contains the pinned Local Path Provisioner manifest. It creates the `local-path` StorageClass and stores each volume on the Node where its Pod runs. It provides a working PVC path for the exercise, but no cross-node replication, volume failover or OpenStack block-volume lifecycle. [\[31\]](https://github.com/rancher/local-path-provisioner/releases/tag/v0.0.37) Before any object is created, the installed Kubernetes API performs a **server-side dry run** of the class, Cluster and add-on resources. This checks the actual CAPI and CAPO schemas and admission rules: ```bash # Render local values, then validate the class, Cluster and add-on declarations. python3 scripts/render-manifests.py --contract environment-contract.yaml --output rendered kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f manifests/base/clusterclass.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/service-a.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/addons/service-a-cert-manager.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/addons/service-a-contour.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/addons/service-a-storage.yaml ``` Representative success output: ``` rendered/service-a.yaml rendered/addons/service-a-cert-manager.yaml clusterclass.cluster.x-k8s.io/openstack-standard serverside-applied (server dry run) cluster.cluster.x-k8s.io/service-a serverside-applied (server dry run) ``` [Walkthrough 4](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/04-workload-configuration.md) includes credential-Secret creation, controller checks and complete declarations. [C4-P4](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p4-workload-cluster-configuration-image-clusterclass-and-variables) connects the image, class and local variables. ## 5. Roll out a workload cluster The `service-a` request now activates the lifecycle chain from Figure 3. The shared ClusterClass is applied first, followed by the rendered Cluster. A diff shows the intended API change before either command writes it: ```bash # Inspect and apply the reusable class and one concrete cluster request. kubectl --kubeconfig "$KUBECONFIG" diff -f manifests/base/clusterclass.yaml || test "$?" -eq 1 kubectl --kubeconfig "$KUBECONFIG" diff -f rendered/addons/service-a-cert-manager.yaml || test "$?" -eq 1 kubectl --kubeconfig "$KUBECONFIG" diff -f rendered/addons/service-a-contour.yaml || test "$?" -eq 1 kubectl --kubeconfig "$KUBECONFIG" diff -f rendered/addons/service-a-storage.yaml || test "$?" -eq 1 kubectl --kubeconfig "$KUBECONFIG" diff -f rendered/service-a.yaml || test "$?" -eq 1 kubectl --kubeconfig "$KUBECONFIG" apply -f manifests/base/clusterclass.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/addons/service-a-cert-manager.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/addons/service-a-contour.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/addons/service-a-storage.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/service-a.yaml # Follow reconciliation in the Management cluster. clusterctl --kubeconfig "$KUBECONFIG" describe cluster service-a \ --namespace service-a --show-conditions all kubectl --kubeconfig "$KUBECONFIG" -n service-a \ get cluster,machine,machinedeployment,kubeadmcontrolplane -o wide ``` Representative success output: ``` clusterclass.cluster.x-k8s.io/openstack-standard configured clusterresourceset.addons.cluster.x-k8s.io/service-a-cert-manager created NAME CLUSTERCLASS AVAILABLE service-a openstack-standard True ``` `clusterctl describe cluster` prints an ownership and condition tree rather than one fixed success message. For a completed rollout, the root Cluster reports `Available=True`; the KubeadmControlPlane is initialized and available; the MachineDeployment has its requested available replicas; and its Machines and OpenStackMachines are ready. A false condition points to the layer that is still blocked, such as infrastructure allocation, bootstrap or control-plane initialization. [\[5\]](https://cluster-api.sigs.k8s.io/clusterctl/commands/describe-cluster.html) Once the control plane exists, its generated kubeconfig is saved without printing it. Calico is then installed with the Pod CIDR declared for `service-a`, and the child is checked directly: ```bash # Retrieve protected child access, install networking, and inspect the result. umask 077 clusterctl --kubeconfig "$KUBECONFIG" get kubeconfig service-a \ --namespace service-a > service-a.kubeconfig ./scripts/install-cni.sh "$SERVICE_A_KUBECONFIG" 10.241.0.0/16 kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" get nodes -o wide kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" get pods -A ./scripts/verify-resource-chain.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ service-a service-a "$SERVICE_A_KUBECONFIG" "$OS_CLOUD" ``` Representative success output: ``` NAME STATUS ROLES VERSION service-a-control-plane- Ready control-plane service-a-md-0- Ready verified Machine -> OpenStack server -> Ready Node: ``` The final helper verifies a **resource identity chain** by matching each CAPI Machine to its OpenStack instance ID and the corresponding Kubernetes Node provider ID. This proves that the Ready Node is the machine CAPI owns, rather than an unrelated server with a similar name. [\[4\]](https://cluster-api-openstack.sigs.k8s.io/topics/external-cloud-provider.html)[\[6\]](https://cluster-api.sigs.k8s.io/clusterctl/commands/get-kubeconfig.html) The ClusterResourceSets introduced above can apply their payloads only after the Management cluster can reach the child API. We therefore check the child API and CNI first, then verify the certificate, ingress and storage controllers. The PVC test remains the separate proof that the storage controller does useful work: ```bash # Inspect CAPI add-on reconciliation and all resulting child controllers. kubectl --kubeconfig "$KUBECONFIG" -n service-a \ get clusterresourceset,clusterresourcesetbinding kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n cert-manager \ rollout status deployment/cert-manager --timeout=5m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n cert-manager \ rollout status deployment/cert-manager-webhook --timeout=5m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n projectcontour \ rollout status deployment/contour --timeout=5m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n projectcontour \ rollout status daemonset/envoy --timeout=5m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n local-path-storage \ rollout status deployment/local-path-provisioner --timeout=5m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" get storageclass local-path # Request storage, run one Pod that writes and reads it, then print its marker. kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" apply \ -f manifests/probe/storage-smoke.yaml kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n storage-smoke \ wait --for=jsonpath='{.status.phase}'=Bound pvc/local-path-smoke --timeout=3m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n storage-smoke \ wait --for=jsonpath='{.status.phase}'=Succeeded pod/local-path-smoke --timeout=3m kubectl --kubeconfig "$SERVICE_A_KUBECONFIG" -n storage-smoke \ logs pod/local-path-smoke ``` Representative success output: ``` NAME MATCHING CLUSTER RESOURCES APPLIED service-a-cert-manager service-a True service-a-contour service-a True service-a-storage service-a True deployment "cert-manager" successfully rolled out deployment "cert-manager-webhook" successfully rolled out deployment "contour" successfully rolled out daemon set "envoy" successfully rolled out deployment "local-path-provisioner" successfully rolled out NAME PROVISIONER DEFAULT local-path rancher.io/local-path false persistentvolumeclaim/local-path-smoke condition met pod/local-path-smoke condition met chapter-4-storage-ready ``` The final log line must be `chapter-4-storage-ready`. Together with a Bound PVC and a Succeeded Pod, it proves that dynamic provisioning, scheduling, mounting, writing and reading worked on this cluster. It does not prove node-failure recovery or durable production storage. The complete timeout and diagnostic path is in [Walkthrough 5](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/05-workload-rollout.md); [C4-P5](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p5-workload-cluster-rollout-follow-reconciliation-to-ready-nodes) follows reconciliation to Ready Nodes. ## 6. Change capacity and prepare upgrades **Scaling** changes the Cluster declaration, not an individual VM. The lab helper copies the environment contract, changes the `service-a` worker count, renders the result, runs the server-side validation, displays `kubectl diff`, and applies only after the confirmed context is supplied: ```bash # Scale service-a from one to two workers through its owning declaration. CONFIRM_MANAGEMENT_CONTEXT="$MANAGEMENT_CONTEXT" \ CONFIRM_APPLY="$MANAGEMENT_CONTEXT" \ ./scripts/scale-workers.sh environment-contract.yaml 2 \ "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ "$SERVICE_A_KUBECONFIG" "$OS_CLOUD" ``` Representative success output: ``` scale-out server present: verified Machine -> OpenStack server -> Ready Node: ``` This is the same CAPI-to-CAPO-to-kubeadm chain introduced earlier. The helper now waits for both Machines, verifies both identity chains and confirms that the added server exists. Running it again with replica count `1` performs the reverse operation and confirms that the removed server ID disappears. The exercise changes worker capacity only; its small values are not production sizing guidance.
A replica change becomes a Machine, an OpenStack server and a Kubernetes Node; validation reads the same chain in reverse.

Figure 8. Scaling changes the declaration and verifies every resulting ownership boundary.

An **upgrade** changes both the Kubernetes version and its prepared machine image. The pair must be compatible with CAPI, CAPO, kubeadm and the CNI. The helper rejects a namespace mismatch or incomplete current rollout, binds the child kubeconfig through the Machine-to-server-to-Node chain, and writes a proposal to `evidence/upgrade/service-a.yaml` for validation and diff before apply: ```bash # Prepare an exact version and image pair without changing the live Cluster. TARGET_KUBERNETES_VERSION="REPLACE_WITH_REVIEWED_TARGET_VERSION" TARGET_IMAGE_ID="REPLACE_WITH_PREPARED_GLANCE_IMAGE_UUID" CONFIRM_MANAGEMENT_CONTEXT="$MANAGEMENT_CONTEXT" PREPARE_ONLY=yes \ ./scripts/upgrade-cluster.sh environment-contract.yaml service-a service-a \ "$TARGET_KUBERNETES_VERSION" "$TARGET_IMAGE_ID" \ "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ "$SERVICE_A_KUBECONFIG" "$OS_CLOUD" kube-system coredns # Bind approval to the reviewed pair, then repeat validation, diff and apply. CONFIRM_MANAGEMENT_CONTEXT="$MANAGEMENT_CONTEXT" \ CONFIRM_UPGRADE="service-a/service-a:${TARGET_KUBERNETES_VERSION}:${TARGET_IMAGE_ID}" \ ./scripts/upgrade-cluster.sh environment-contract.yaml service-a service-a \ "$TARGET_KUBERNETES_VERSION" "$TARGET_IMAGE_ID" \ "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ "$SERVICE_A_KUBECONFIG" "$OS_CLOUD" kube-system coredns ``` Representative success output: ``` prepare-only complete; no Cluster object or contract was changed upgrade verified: service-a/service-a version= image= workload=kube-system/coredns ``` The approved run checks CoreDNS before and after the change, waits for control-plane and worker rollout conditions, and verifies the requested version, image and identity chain. Production adds an application-level service test. This release keeps the path `not-tested` because no approved OpenStack target version and image were supplied. [\[13\]](https://cluster-api.sigs.k8s.io/reference/versions.html)[\[28\]](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/operate-cluster.html#upgrade-a-cluster) [Walkthrough 6](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/06-adjustments.md) contains the complete scale and upgrade safeguards. [C4-P6](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p6-adjustments-scale-workers-and-plan-upgrades) shows the replica change and keeps the unexecuted upgrade distinct from validated evidence. ## 7. Add the LoadBalancer cluster and prove the service path The `loadbalancer` request selects the same ClusterClass but supplies a different network, role and machine size. This demonstrates why the class and Cluster request are separate: the lifecycle machinery is reused while the local values describe a different cluster. ```bash # Render, validate and create the edge-role cluster. python3 scripts/render-manifests.py --contract environment-contract.yaml --output rendered kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/loadbalancer.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/addons/loadbalancer-cert-manager.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/addons/loadbalancer-contour.yaml kubectl --kubeconfig "$KUBECONFIG" apply --server-side --dry-run=server \ -f rendered/addons/loadbalancer-storage.yaml kubectl --kubeconfig "$KUBECONFIG" diff -f rendered/loadbalancer.yaml || test "$?" -eq 1 kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/addons/loadbalancer-cert-manager.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/addons/loadbalancer-contour.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/addons/loadbalancer-storage.yaml kubectl --kubeconfig "$KUBECONFIG" apply -f rendered/loadbalancer.yaml ./scripts/wait-for-cluster.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ loadbalancer loadbalancer 30m ``` Representative success output: ``` Cluster loadbalancer/loadbalancer reached Available; OpenStack state and child Node readiness were not checked. ``` After retrieving its kubeconfig and installing its CNI, we wait for cert-manager, Contour and Envoy in the LoadBalancer cluster. Two paths are then tested independently. The first sends an authenticated `/readyz` request from a disposable Pod in the Management cluster to each private child API. A passing helper prints `ManagementPathReady` for `service-a` and `loadbalancer`. This establishes the management path used by CAPI. The **data path** test creates a disposable HTTP destination, Service and Contour `Ingress` in `service-a`. Both child kubeconfigs are bound to explicit contexts and API servers. The verifier derives the destination from `service-a`'s private Envoy Service and confirms the Ingress class, host and backend before a curl Pod in the LoadBalancer cluster calls it: ```bash # Prove management access to both child APIs. CONFIRM_MANAGEMENT_PATH_TEST=yes \ ./scripts/verify-management-api-path.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ "$SERVICE_A_KUBECONFIG" service-a CONFIRM_MANAGEMENT_PATH_TEST=yes \ ./scripts/verify-management-api-path.sh "$KUBECONFIG" "$MANAGEMENT_CONTEXT" \ "$LOADBALANCER_KUBECONFIG" loadbalancer # Bind the local contract to both reviewed contexts and their actual API servers. python3 scripts/render-path-contract.py \ --source-kubeconfig "$LOADBALANCER_KUBECONFIG" --source-context "$LOADBALANCER_CONTEXT" \ --destination-kubeconfig "$SERVICE_A_KUBECONFIG" --destination-context "$SERVICE_A_CONTEXT" \ --output hub-spoke-contract.yaml # Review the generated identities, add approved network evidence and set both approval fields. CONFIRM_PATH_DESTINATION="service-a/${SERVICE_A_CONTEXT}" \ ./scripts/prepare-path-probe.sh hub-spoke-contract.yaml CONFIRM_PATH_TEST="loadbalancer/${LOADBALANCER_CONTEXT}->service-a/${SERVICE_A_CONTEXT}" \ ./scripts/verify-hub-spoke.sh hub-spoke-contract.yaml ``` Representative success output: ``` ManagementPathReady: service-a /readyz returned ok from a Management-cluster Pod ManagementPathReady: loadbalancer /readyz returned ok from a Management-cluster Pod Path contract written: hub-spoke-contract.yaml WorkloadIngressReady: service-a/service-a-admin@service-a via private Envoy , HTTP 200 PathReady: loadbalancer/loadbalancer-admin@loadbalancer -> service-a/service-a-admin@service-a via private Envoy , HTTP 200 ``` `WorkloadIngressReady` proves the local Envoy route. `PathReady` adds the edge-to-workload hop with both contexts, the derived private address and HTTP status. Public exposure, DNS, TLS and production routes continue in Chapter 5. An unapproved or unbound path remains `not-tested`. [Walkthrough 7](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/lab/walkthroughs/07-loadbalancer-edge.md) provides the route contract, child bootstrap and cleanup commands. [C4-P7](https://git.vanillacore.net/giga/content/src/branch/main/chapter-4/VIDEO_SCOPES.md#c4-p7-load-balancer-and-edge-cluster-prove-a-real-hub-and-spoke-path) shows the control and data paths separately. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://experience-digital-transform.gitbook.io/experience-digital-transformation/cloud-for-dpi/chapter-4/practical-path-on-openstack.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.