AORXI Homelab
Compute & Storage

Proxmox Clusters

Proxmox clustering plan for both sites: cluster names, node membership, the IP and /etc/hosts prerequisites, dedicated Corosync on VLAN 25, and the phased build order.

Two independent Proxmox clusters — sa-pve at Site A and sb-pve at Site B — form the compute foundation of this homelab. This page covers cluster membership, the prerequisites that must be met before forming each cluster, the Corosync heartbeat network, and the steps to bring both clusters up.

Cluster Overview

One Proxmox cluster per site. The clusters are completely independent and must never share nodes or run Corosync across the WireGuard tunnel or WAN link.

One cluster per site — no cross-WAN clustering

One Proxmox cluster per site: sa-pve at Site A, sb-pve at Site B. Never stretch a Proxmox cluster across WAN or WireGuard. Corosync requires LAN-like latency; a cross-site cluster risks split-brain and quorum instability.

SiteClusterNodesCreate First On
Site Asa-pvesa-edge-01, sa-cmp-01, sa-cmp-02, sa-stor-01sa-stor-01
Site Bsb-pvesb-edge-01, sb-cmp-01, sb-cmp-02, sb-cmp-03, sb-cmp-04, sb-cmp-05sb-cmp-01

Prerequisites Before Clustering

Clustering must not begin until every node in the site meets all conditions below. Clustering with bootstrap or mismatched IPs requires destructive remediation.

Cluster only after every node has its final IP and correct /etc/hosts

Only create or join a cluster after every node in the site has its final 10.x.20.x Proxmox Management (VLAN 20) address and /etc/hosts lists all cluster peers by name and final IP. Never cluster from a bootstrap 192.168.x.x address.

Never join nodes across sites

sa-pve nodes and sb-pve nodes must never be joined to each other. Never run Corosync over WireGuard or WAN.

Node IP Reference

NodeSiteProxmox Mgmt IP (VLAN 20)Bootstrap IP
sa-stor-01Site A10.10.20.20192.168.1.20
sa-edge-01Site A10.10.20.10192.168.1.10
sa-cmp-01Site A10.10.20.11192.168.1.11
sa-cmp-02Site A10.10.20.12192.168.1.12
sb-cmp-01Site B10.20.20.20192.168.16.20
sb-edge-01Site B10.20.20.10192.168.16.10
sb-cmp-02Site B10.20.20.21192.168.16.21
sb-cmp-03Site B10.20.20.30192.168.16.30
sb-cmp-04Site B10.20.20.31192.168.16.31
sb-cmp-05Site B10.20.20.32192.168.16.32

Bootstrap 192.168.1.0/24 is the live Site A UniFi LAN. Confirm bootstrap IP assignments do not collide with the UniFi DHCP pool or existing reservations before assigning them.

Corosync Network (VLAN 25)

Corosync heartbeat traffic runs on dedicated VLAN 25 (Corosync heartbeat) interfaces, one per site. VLAN 25 carries no default gateway and uses standard 1500-byte MTU.

VLANNameSite A subnetSite B subnetGateway
25Corosync heartbeat10.10.25.0/2410.20.25.0/24no GW

Corosync stays on VLAN 25 — no WAN, no WireGuard, no jumbo frames

Run Corosync only on the local VLAN 25 interface. Never route Corosync over WireGuard or WAN. Never enable jumbo frames (MTU 9000) on VLAN 25; standard 1500 MTU only.

sa-stor-01 Corosync wiring

sa-stor-01 (5049A-T / X11SPA-T) has one onboard 1GbE (Intel i210, the red RJ45) used for Proxmox management. Its onboard 10G port (Aquantia AQC107) is deliberately avoided for critical traffic due to atlantic driver instability under Linux/Proxmox. Corosync on sa-stor-01 stays tagged on the XL710 add-in card port 1 alongside VLAN 30 rather than on a dedicated link.

E200 Cluster Membership

Both sa-edge-01 and sb-edge-01 (Supermicro SYS-E200-8D) join their local cluster. The OPNsense VM on each E200 must remain pinned and must not be HA-migrated.

OPNsense VM stays pinned to its local E200 — no HA migration

sa-fw-01 stays on sa-edge-01; sb-fw-01 stays on sb-edge-01. Never enable automatic HA migration for these VMs. Migrating the firewall VM disrupts all cluster network traffic that routes through it.

Suitable E200 workloads

In addition to OPNsense, the E200 can host lightweight VMs: DNS helper, UniFi controller, WireGuard helper, small reverse proxy, monitoring agent. Avoid heavy databases, Ceph OSDs, or anything that competes with OPNsense for CPU or memory.

Build Order

The steps below correspond to Phase 6 of the full Build Phases sequence. Phases 1–5 (hardware prep, Proxmox install on bootstrap IPs, OPNsense/VLAN configuration, switch configuration, and IP migration to final 10.x.20.x addresses) must be complete before starting here.

Phase 6A — Site A (sa-pve)

  1. Confirm all four sa-pve nodes are reachable on their final VLAN 20 addresses and that /etc/hosts lists all peers.
  2. Create the cluster on sa-stor-01 using cluster name sa-pve.
  3. Join sa-edge-01 to the cluster.
  4. Join sa-cmp-01 to the cluster.
  5. Join sa-cmp-02 to the cluster.
  6. Verify quorum — all four nodes must be online with quorum before proceeding.

Phase 6B — Site B (sb-pve)

  1. Confirm all six sb-pve nodes are reachable on their final VLAN 20 addresses and that /etc/hosts lists all peers.
  2. Create the cluster on sb-cmp-01 using cluster name sb-pve.
  3. Join sb-edge-01 to the cluster.
  4. Join sb-cmp-02 to the cluster.
  5. Join sb-cmp-03, sb-cmp-04, sb-cmp-05 to the cluster.
  6. Verify quorum — all six nodes must be online with quorum before proceeding.

Confirm quorum before storage bring-up

Cluster quorum must be stable at each site before provisioning ZFS on Site A or Ceph on Site B. Storage bring-up is Phase 9; quorum verification is the gate between Phase 6 and later phases.

Compute & Storage

Proxmox cluster layout, Site A ZFS storage, Site B Ceph cluster, and PBS cross-site backup replication.

Site A ZFS Storage

ZFS mirror storage plan for sa-stor-01 (Supermicro 5049A-T): drive layout, vdev structure, network configuration, and the service VMs the pool serves.

On this page

Cluster OverviewPrerequisites Before ClusteringNode IP ReferenceCorosync Network (VLAN 25)E200 Cluster MembershipBuild OrderPhase 6A — Site A (sa-pve)Phase 6B — Site B (sb-pve)Related Pages