> For the complete documentation index, see [llms.txt](https://experience-digital-transform.gitbook.io/experience-digital-transformation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://experience-digital-transform.gitbook.io/experience-digital-transformation/sovereign-cloud-stack/1.-virtualization-layer-operating-scs-iaas/1.1-virtualization-overview-and-openstack-architecture.md). # 1.1 Virtualization Overview and OpenStack Architecture ## Virtualizing Hardware * Applications are written to run on servers. * Virtualization: in software, create an execution environment for the application that looks like a server (landscape). ### Types of virtual hardware * **Compute virtualization:** virtual CPUs, virtual memory (RAM). * **Storage virtualization:** virtual hard disks (block storage, "volumes"); shared storage (NFS/CIFS) and object storage options. * **Network virtualization:** virtual network cards (ports) connected to virtual networks, routers, etc.; virtual firewalls and load-balancers. ### Advantages * **Assignment granularity** — can assign arbitrary amounts (e.g. 4 CPUs, 16 GiB RAM, 155GB disk), independent of actual server sizes. Real hardware must of course have sufficient capacity. Fractional amounts (shared resources) are possible. * **Changing assignment on the fly** — aka vertical scaling. Hot-plug vs. cold-plug (reboots). * **Additional servers, disks, … for scalable software** — horizontal scaling. * **Procurement delay** (weeks) vs. SW provisioning (seconds/minutes at worst). ### Common technologies * **Compute virtualization:** VMware ESXi (proprietary), Xen, **KVM**. * **Storage virtualization:** SAN (FibreChannel), vSAN, NAS (NetApp and friends), **Ceph (RBD)**. * **Network virtualization (SDN):** VLANs, VxLANs, VMware NSX-T, Layer3 networks (routed), Cloud: VPC, Linux: Bridge, **OpenvSwitch (OvS)** and **OVN** (Open Virtual Network). ### Infrastructure-as-Code * Automate the set up of virtual hardware completely — not just single servers, but complete landscapes for complicated application workloads. Tooling: scripts with CLI tools, SDK (e.g. python, go, …), terraform/opentofu, ansible, pulumi, … * **Automated tests** — now include the complete roll-out of the test infrastructure. Can be torn down automatically after the test (to limit cost). Load tests. * **Keep state at confined places** — database, data volume, configuration repository. * **Keep everything else stateless** — can always be redeployed (in case of failure, in case of a security breach, for an upgrade). No backups needed. Typically scales trivially. ## OpenStack Architecture overview

Figure 1.1: OpenStack Architecture Overview

**What is OpenStack?** A set of services that manages virtualized resources and gives users interfaces to control these. **How is it built?** * **API services** — independent processes for Compute, Block Storage, Images, Networking, LoadBalancers, Identity, etc. All APIs are RESTful APIs (GET, POST, PUT, PATCH, DELETE verbs for list, create, set, change and delete operations). Many APIs are paginated, i.e. GET returns only a subset of resources. * Some APIs have background services — conductors, schedulers, etc. * Services talk to each other via REST API or via RabbitMQ. * Database keeps tables for the state of resources — tables are each owned by one service. * Identity service has a central role. ### Keystone: Identity service has a central role * User authenticates to keystone (typically with some secret) and gets a token. * Token can be validated by individual services. * Tokens have a scope that determines what you can do with them: unscoped, domain scope, project scope. * Identity service hosts the service catalogue. ### Keystone: Raw REST API example (against CiaB) Discovery (`--cacert ...` needed b/c of self-signed certificate): ```bash dragon@cumulus(//):~ [4]$ curl -sS -g --cacert "/etc/ssl/certs/ca-certificates.crt" \ -X GET https://api.in-a-box.cloud:5000/v3 \ -H "Accept: application/json" | jq . ``` ```json { "version": { "id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [ { "rel": "self", "href": "https://api.in-a-box.cloud:5000/v3/" } ], "media-types": [ { "base": "application/json", "type": "application/vnd.openstack.identity-v3+json" } ] } } ``` The services are versioned and have versioned (and microversioned) APIs. Password authentication: ```bash curl -sS --cacert "/etc/ssl/certs/ca-certificates.crt" \ -X POST https://api.in-a-box.cloud:5000/v3/auth/tokens \ -H "Accept: application/json" -H "Content-Type: application/json" -d ' { "auth": { "identity": { "methods": ["password"], "password": { "user": { "name": "test", "password": "test", "domain": { "name": "test" } } } }, "scope": { "project": { "name": "test" } } } } ' | jq ``` This returns a token response containing the user, roles (`reader`, `member`, `load-balancer_member`, `creator`), and the service **catalog** — a list of service endpoints (e.g. `skyline` panel, `placement`), each with public/internal interfaces and a region. * All services are identified by unique 16-byte IDs; resources also carry 16-byte UUIDs (typically in 8-4-4-4-12 notation). Tools typically support addressing resources by name, but names are *not* enforced to be unique everywhere. * Much easier: store the keystone endpoint and credentials in `~/.config/openstack/clouds.yaml` and `secure.yaml`, and issue `openstack --os-cloud test catalog list`. ```yaml # ~/.config/openstack/clouds.yaml --- clouds: test: auth: username: test project_name: test auth_url: https://api.in-a-box.cloud:5000/v3 project_domain_name: test user_domain_name: test cacert: /etc/ssl/certs/ca-certificates.crt identity_api_version: 3 ``` ```yaml # ~/.config/openstack/secure.yaml --- clouds: test: auth: password: test ``` ### OpenStack Core services | Type | Name | Function | | ------------ | -------- | ------------------------------------------- | | identity | keystone | Identity and Access management | | network | neutron | Networking (L2+L3) | | volumev3 | cinder | Block Storage (virtual hard disks) | | image | glance | Image handling | | compute | nova | VM management | | object-store | swift | Object storage (S3-like) – optional for SCS | All core services (except Swift) need to be present for SCS-compatible IaaS compliance. Note on swift: an S3-compatible service is required (ideally, both S3 and Swift are offered). Note on terminology: OpenStack calls hosts (hardware nodes) **hypervisors** and VMs **instances** or **servers**. ### Other standard OpenStack services | Type | Name | Function and Notes | | ------------- | --------- | -------------------------------------------------------- | | orchestration | heat | Deploy sets of resources (like cloudformation, heat-cfn) | | dns | designate | DNS service | | load-balancer | octavia | L4 and L7 load balancing | | key-manager | barbican | Secrets management (e.g. for L7 LB, like vault) | ### Internal OpenStack services | Type | Name | Function and Notes | | --------- | ---------- | --------------------------------------------------------------------------- | | placement | placement | Determine where VMs are started (scheduler), typically only used internally | | telemetry | ceilometer | Collect usage/metering data (typically not exposed) | ### Optional OpenStack services | Type | Name | Function and Notes | | --------------- | ------- | ---------------------------------- | | metric | gnocchi | Aggregation of metering data | | alarming | aodh | Trigger notifications | | clustering | senlin | Manage sets of resources | | sharev2 | manila | Shared file-systems (NFS, CIFS, …) | | baremetal | ironic | Bare Metal instance management | | container-infra | magnum | Manage container clusters | Many more "big tent" services exist (trove – database, zaqar – queuing, mistral – workflow, sahara – big data, cloudkitty – metering, watcher – optimization), see [openstack.org/software](https://www.openstack.org/software/). They have varying degrees of maturity and are not supported by default in the SCS reference implementation. ### Dashboards: Horizon and Skyline **Horizon** is the traditional dashboard that almost every OpenStack cloud offers. It takes a bit of time to get used to.

Figure 1.2: Horizon Screenshot

**Skyline** is a relatively new project and only offered on some clouds. It has a more modern look and can easily be extended.

Figure 1.3: Skyline Dashboard

Both dashboards allow downloading `clouds.yaml` or old-style `openrc` files. ### Internal infrastructure OpenStack requires queuing and a database to work. * mariadb (free version of MySQL) — clustered with galera for high availability. * memcached and/or proxysql for caching (performance). * Queuing connects publishers and consumers of messages, which should be processed in order (FIFO). OpenStack traditionally uses **rabbitmq** as queuing (AMQP) service. Quorum queues are preferred where possible. * OSISM comes with significantly more infrastructure: monitoring, life-cycle management, log aggregation and search, k3s cluster (for extensibility). ## OpenStack user management ### Domains and Projects * Resources belong to **projects** (formerly called tenants) — think of a project as a workspace container for virtualized resources. * Projects are structured into **domains** (optional, mandatory in SCS). Domains are thus containers (realms) for projects, users and groups. * `Domain_Name:Project_Name`, `Domain_Name:User_Name`, `Domain_Name:Group_Name` tuples are unique identifiers (the corresponding IDs are unique as well). * The `domain_manager` role can be handed to customers to manage projects, users and groups on their own (self-service). This is secure — they cannot grant themselves rights to projects of other domains. This was introduced by the SCS community and only merged upstream in OpenStack 2024.2 (SCS R8). The admin can, however, assign rights across domain boundaries if so wanted (not typically a good idea). ### Role assignments * Roles are rights (privileges) that users have towards projects — e.g. "User XYZ\@domA has the right `reader` in project ABC\@domA." * Standard roles with increasing level of privilege: * Keystone: `reader`, `member`, `manager`, `admin` * Most services: `reader`, `member`, `admin` * Octavia: `load-balancer_observer`, `load-balancer_member`, `load-balancer_admin` * Barbican: `observer`, `creator`, `admin` * The special cases for Octavia and Barbican are being worked on in the secure RBAC cleanup that is currently happening. * The `service` role is internal for communication between services. * These privileges have a scope, e.g. project or domain or system. Try `openstack --os-cloud=admin role assignment list --names`. * Normal users require `member` (and `creator` for Barbican and `load-balancer_member` for Octavia) roles for their own project(s). Resellers or IT managers would have `manager` privilege for their domain(s). * Role assignment can be indirected via group membership. ## Security Architecture **Compute separation** * Hardware virtualization technology separates VMs from each other. SCS mandates microcode and hypervisor/kernel mitigation to be active against known CPU vulnerabilities. * SCS mandates Hyperthreading to be switched off if it's not secure (scs-0100). * Highest security environments might want to use dedicated hosts or assign dedicated host groups (host aggregates) to avoid sharing hardware with untrusted users. **Network separation** * User-controlled networks are separated using encapsulation or VxLAN technology. * Internal control traffic is encrypted (https). * User-plane traffic can be encrypted by the application operator, optionally using container side-cars or done at the virtualization layer (at the cost of performance). **Storage separation** * Ceph enforces storage isolation. * Users can use LUKS for sensitive data. * Optional disk encryption, exposed via storage class. **Architecture** * Secrets are stored in ansible vault or external vaults (keepass, hashicorp vault/openbao). * Internal control traffic encrypted. * Secure delegation of administrative powers with the domain-manager role limited to its own domain. * CI jobs with security scans. * Constant validation of code (CI) allows quick reaction to vulnerabilities. ## Assignments: Virtualization Architecture **Service catalogue (CiaB)** * Retrieve it using openstack CLI tooling (alternatively: python SDK). What services do you see? Why are there several endpoints per service? Anything unexpected? * Get a token from keystone. * List networks (and external networks) for a project. **Domains, projects, roles** * What are domains good for? * Create a new domain (admin privileges required). Create a domain-manager for it. Create a project in it (domain-manager privileges required). Create a user in it. Grant her access to the project. * Review all roles (admin privileges) — explain line by line. ### Digression: Setting up or using a CiaB for testing See [docs.scs.community — testbed deployment](https://docs.scs.community/docs/iaas/deployment-examples/testbed/). * A Cloud-in-a-Box (CiaB) is an SCS setup for a single node. Can be deployed on bare metal (**wipes your disk!**) or in a VM. * You can collect a lot of experience with it, but it's not secure nor highly available, so **never** even consider exposing it to the internet or using it for any production purpose. * Moderate resource needs: minimum 32GiB RAM, 4 cores, 400GiB of disk space (bare metal or VM), more is better. For small setups, consider disabling k3s and OpenSearch. * Recommended: 96GiB+ RAM, 8+ cores, 2x2+TB NVMe. Demo system: Aoostar Gem12 w/ Ryzen7 8845HS (8 core / 16 HT, Zen4), 96GiB RAM, 2x4TB Lexar NM790 NVMe, 2x2.5Gbps network -> \~$1500. * Switch it on three times, first time with boot image connected (USB, PXE, BMC). Installation process downloads tens of GB of data — ensure a good internet connection. * Create a wireguard tunnel to it — this even works from Windows (download and install Wireguard client software). Look at the dashboards listed in the docs page. * Note that the CiaB uses a self-signed certificate; you need to have your browser (and openstack client tooling) trust it. Blink-based browsers (Chrome, chromium, edge, …) don't allow this any longer. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://experience-digital-transform.gitbook.io/experience-digital-transformation/sovereign-cloud-stack/1.-virtualization-layer-operating-scs-iaas/1.1-virtualization-overview-and-openstack-architecture.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.