Site Roles & Inventory
Per-site roles and complete node inventory: compute nodes, networking devices, and planned service VMs at Site A and Site B.
Each site carries a distinct role. Site A (10.10.0.0/16) runs management, ZFS storage, Proxmox Backup Server (PBS), DNS, and monitoring. Site B (10.20.0.0/16) runs Ceph block storage, Kubernetes/OpenShift, and distributed compute. This page details the physical nodes, networking devices, and planned service VMs at each site.
Site A — Management & Storage
Site A is the management hub, hosting primary PBS, ZFS-backed storage, DNS, and monitoring infrastructure. Proxmox cluster name: sa-pve.
Compute Nodes
| Host | Hardware | Primary Workloads | Proxmox Mgmt (VLAN 20) |
|---|---|---|---|
sa-edge-01 | Supermicro SYS-E200-8D | Proxmox + OPNsense VM (sa-fw-01) | 10.10.20.10 |
sa-cmp-01 | ThinkPad P51 | Proxmox worker, CI (Intel X550-T2) | 10.10.20.11 |
sa-cmp-02 | ThinkPad P52 | Proxmox worker, GPU / AI (Intel X550-T2) | 10.10.20.12 |
sa-stor-01 | Supermicro 5049A-T | ZFS mirror, PBS, DNS, monitoring, databases | 10.10.20.20 |
Networking Devices
| Device | Model | Role | VLAN 10 IP |
|---|---|---|---|
sa-sw-01 | Netgear XS716T | 10 Gb L2 core switch | 10.10.10.2 |
sa-sw-02 | — | Access / IPMI switch | 10.10.10.3 |
sa-sw-03 | — | Access / IPMI / AP switch | 10.10.10.4 |
sa-ap-01 | UniFi U7 Pro XGS | Wi-Fi 7 AP (PoE++ on sa-sw-03 port 3) | 10.10.10.6 |
| UniFi Gateway Max | — | Bootstrap / fallback only | 10.10.10.5 |
sa-sw-01 carries Proxmox, Corosync, VM, Kubernetes, storage, and backup VLANs. Proxmox Management (VLAN 20) and IPMI (VLAN 10) for sa-edge-01 and sa-stor-01 are offloaded to sa-sw-02 and sa-sw-03 to free core-switch ports for dedicated Corosync links. See Site A Port Map for the full assignment.
Planned Service VMs
| VM | Role | VLAN 30 Mgmt IP | VLAN 90 Backup IP |
|---|---|---|---|
sa-dns-01 | Primary Technitium DNS (authoritative) | 10.10.30.10 | — |
sa-dns-02 | DNS replica (AXFR from sa-dns-01) | 10.10.30.11 | — |
| PBS-A | Proxmox Backup Server (primary site) | 10.10.30.20 | 10.10.90.40 |
Site B — Ceph, Kubernetes & Compute
Site B is the distributed compute hub, running Ceph across five nodes and hosting Kubernetes/OpenShift workloads. Proxmox cluster name: sb-pve.
Compute Nodes
| Host | Hardware | Primary Workloads | Proxmox Mgmt (VLAN 20) |
|---|---|---|---|
sb-edge-01 | Supermicro SYS-E200-8D | Proxmox + OPNsense VM (sb-fw-01) | 10.20.20.10 |
sb-cmp-01 | Supermicro SYS-5019D-4C-FN8TP | Ceph MON/MGR, Kubernetes control-plane | 10.20.20.20 |
sb-cmp-02 | Supermicro SYS-5019D-4C-FN8TP | Ceph MON/MGR, Kubernetes control-plane | 10.20.20.21 |
sb-cmp-03 | Supermicro SYS-5018D-FN4T | Ceph OSD, Kubernetes worker | 10.20.20.30 |
sb-cmp-04 | Supermicro SYS-5018D-FN4T | Ceph OSD, Kubernetes worker | 10.20.20.31 |
sb-cmp-05 | Supermicro SYS-5018D-FN4T | Ceph OSD, Kubernetes worker | 10.20.20.32 |
Networking Devices
| Device | Model | Role | VLAN 10 IP |
|---|---|---|---|
sb-sw-01 | Netgear XS748T | 10 Gb L2 core switch | 10.20.10.2 |
sb-sw-02 | UniFi USW 24 PoE | Access / IPMI / AP switch | 10.20.10.3 |
| USG Pro | — | Bootstrap / fallback only | 10.20.10.4 |
Planned Service VMs
| VM | Role | VLAN 30 Mgmt IP | VLAN 90 Backup IP |
|---|---|---|---|
sb-dns-01 | DNS replica (AXFR from sa-dns-01) | 10.20.30.10 | — |
sb-dns-02 | DNS replica (AXFR from sa-dns-01) | 10.20.30.11 | — |
| PBS-B | Proxmox Backup Server (DR replica) | 10.20.30.20 | 10.20.90.40 |
E200 Workload Guidance
Both edge nodes (sa-edge-01, sb-edge-01) use the Supermicro SYS-E200-8D, a compact platform whose primary responsibility is running the site OPNsense VM. The OPNsense VM must remain pinned to its local E200 and must not be migrated by the Proxmox HA subsystem.
OPNsense VM is pinned — no HA migration
sa-fw-01 stays on sa-edge-01; sb-fw-01 stays on sb-edge-01. Never enable HA migration for the OPNsense VM. Migrating the firewall VM takes down the site.
Suitable E200 workloads
OPNsense VM, DNS helper, UniFi controller, WireGuard helper, small reverse proxy, monitoring agent.
Avoid on E200
Heavy databases, Ceph OSDs, storage-heavy VMs, heavy Kubernetes workers, or any workload that competes with OPNsense for CPU or memory.
Related Pages
- Architecture Overview — topology diagram and hard architecture rules
- Two-Site Model — design rationale and site separation
- Per-Site Inventory — NIC, RAM, and storage detail per node
- Site A Port Map — switch port assignments and VLAN tagging
- Site B Port Map — switch port assignments and VLAN tagging
- Build Phases — ordered bring-up sequence
The Two-Site Model
Two independent Proxmox clusters connected by WireGuard: the rationale for splitting across two physical sites, the per-site topology pattern, and the hard rules governing inter-site connectivity.
Network Topology
Physical and logical topology of both sites: the ISP-to-host data path, Netgear 10 Gb core and access switch hierarchy, WireGuard inter-site link, and UniFi WAN-transit branch.