Current Build State
Dated snapshot of the homelab build: which nodes are up, what phase each site is in, and the immediate next steps as of 2026-07-03.
Four Site A nodes are running Proxmox on the temporary bootstrap LAN. Two of the three edge VMs are up: UniFi OS Server sa-uos-01 and OpenBao sa-bao-01 (initialized; repo secrets run bao-first). The OPNsense VM sa-fw-01 is not currently running — the 2026-07-01 bring-up captured in the bootstrap runbook was destroyed / never finished (owner-confirmed 2026-07-03), and its provision stack is parked at enabled: false. Site B has not started. This page is a dated snapshot — use it to re-orient after a break and to track what is confirmed versus what is still pending.
Snapshot — as of 2026-07-03 (corrects the 2026-07-02 claim)
Site A is at Phase 0 minus step 3.1: Proxmox on all four nodes, UOS and bao live, but sa-fw-01 must be (re)provisioned per the bootstrap runbook step 3.1 before any Phase-1 cutover work. The wizard/GUI values below were captured from the destroyed bring-up and remain the reference for the re-provision. Update after each completed build step.
Status Overview
| Component | State |
|---|---|
| Site A — Proxmox installed on all 4 nodes | Complete |
Site A — sa-fw-01 OPNsense VM installed, GUI reachable | Superseded — VM not running (2026-07-03) |
| Site A — OPNsense setup wizard complete (Phase 1, WAN DHCP) | Superseded — values kept as reference |
Site A — sa-fw-01 rebuildable from repo (Pulumi aorxi-opnsense + Ansible opnsense/config) | Demonstrated once — stack now parked enabled: false |
Site A — re-provision sa-fw-01 (bootstrap runbook step 3.1: provision-set → preview → up → config-apply) | Pending |
Site A — sa-uos-01 UniFi OS Server provisioned + first-run config (Pulumi aorxi-unifi + Ansible unifi/config) | Complete |
Site A — sa-bao-01 OpenBao provisioned + initialized; KV homelab mount seeded, bao-first secrets live | Complete |
Site A — VLAN interfaces on OPNsense LAN trunk (vtnet1) | Pending |
Site A — Netgear sa-sw-01 VLAN / trunk configured | Pending |
| Site A — nodes migrated to final VLAN 20 IPs | Pending |
| Site A — Corosync VLAN 25 IPs assigned | Pending |
Site A — sa-pve cluster formed | Pending |
| Site A — ZFS storage, Proxmox Backup Server (PBS-A), DNS VMs deployed | Pending |
| Site B — any build activity | Not started |
Site A
Bootstrap Network
All four Site A nodes are running Proxmox on the temporary bootstrap LAN 192.168.1.0/24, served by the UniFi Gateway Max. The host octet of each bootstrap IP mirrors the final VLAN 20 management octet, which simplifies the cutover — no renumbering logic, just a subnet swap.
| Host | Role | Bootstrap IP | Final VLAN 20 IP |
|---|---|---|---|
sa-edge-01 | Supermicro E200-8D — Proxmox + OPNsense VM host | 192.168.1.10 | 10.10.20.10 |
sa-cmp-01 | ThinkPad P51 — Proxmox worker / CI | 192.168.1.11 | 10.10.20.11 |
sa-cmp-02 | ThinkPad P52 — Proxmox worker / GPU / AI | 192.168.1.12 | 10.10.20.12 |
sa-stor-01 | Supermicro 5049A-T — ZFS / PBS-A / DNS / monitoring | 192.168.1.20 | 10.10.20.20 |
OPNsense VM — sa-fw-01
sa-fw-01 was installed on sa-edge-01 on 2026-07-01 with the setup wizard completed and the web GUI reachable — but that VM was destroyed / never finished (owner-confirmed 2026-07-03) and is not currently running. Its provision stack is parked at enabled: false with credentials unset; re-provision via the bootstrap runbook step 3.1 before any Phase-1 work. The values below were captured from that bring-up and are the reference for the re-provision.
Setup wizard values (captured 2026-07-01, to re-apply):
| Setting | Value |
|---|---|
| Hostname | sa-fw-01 |
| Domain | core.aorxi.io |
| WAN type | DHCP (behind UniFi — Phase 1 only) |
| DNS servers | 1.1.1.1 / 9.9.9.9; WAN DNS override unchecked |
| Block private networks on WAN | OFF — re-enable at Fios handoff |
| Block bogon networks on WAN | OFF — re-enable at Fios handoff |
Re-enable WAN security rules at Fios handoff
"Block private networks" and "Block bogon networks" are deliberately OFF while WAN sits behind UniFi on a private 192.168.x.x subnet. Both must be turned ON before or immediately after cutting WAN over to the real Fios ONT.
sa-edge-01 Bridge Layout
Proxmox bridges on sa-edge-01 as configured during the 2026-06-28 session. A physical NIC can belong to only one bridge.
| Bridge | Physical NIC | Purpose | IP / Notes |
|---|---|---|---|
vmbr0 | Onboard 1G #1 (igb, i350) | Proxmox host management | 192.168.1.10 — bootstrap only |
vmbr1 | Onboard 10G #1 (ixgbe, X552/X557) | OPNsense WAN / Fios | No host IP; MTU 1500 |
vmbr2 | Onboard 10G #2 (ixgbe) | OPNsense LAN trunk to sa-sw-01 p1 | VLAN-aware, IDs 2-4094; MTU 1500 |
vmbr3 | Onboard 1G #2 (igb) | Corosync (VLAN 25) | No IP yet — assigned at clustering step |
Inside OPNsense, vtnet0 maps to vmbr1 (WAN) and vtnet1 maps to vmbr2 (LAN trunk).
Temporary bootstrap NIC — remove after VLAN migration
A temporary third vNIC (vtnet2 on vmbr0) was added to sa-fw-01 during initial bring-up to reach the GUI from a WiFi-only admin laptop. Remove it once management is reachable on tagged VLAN 20 via the LAN trunk.
Edge Stack (IaC posture)
All three Site A edge VMs live on sa-edge-01 and are rebuildable from the repo. sa-uos-01 and sa-bao-01 are live on the temp LAN (192.168.1.0/24) and re-IP to their final addresses when VLANs go live; sa-fw-01 is not running (as of 2026-07-03) and re-lands on its seed IP when step 3.1 is re-run:
| VM | Now (temp LAN) | Final | Provisioned by |
|---|---|---|---|
sa-fw-01 (OPNsense) | not running — seed mgmt 192.168.1.41 after re-provision | VLAN gateways .1 | Pulumi aorxi-opnsense + Ansible opnsense/config |
sa-uos-01 (UniFi OS Server 5.1.19) | 192.168.1.40 | 10.10.10.40 (VLAN 10) | Pulumi aorxi-unifi + Ansible unifi/config |
sa-bao-01 (OpenBao 2.5.4) | DHCP lease (recorded in BAO_ADDR) | 10.10.30.40 (VLAN 30) | Pulumi aorxi-openbao + Ansible openbao/config |
sa-bao-01 runs standalone on Shamir manual unseal until Site B exists; cross-site transit auto-unseal activates when sb-bao-01 comes online. Boot order on the E200: OPNsense first, then UOS and bao.
Site B
Site B build has not started. All Site B nodes (sb-edge-01, sb-cmp-01 through sb-cmp-05) are specified in the design but none have been brought up. Site B begins only after Site A networking, clustering, and storage are stable. See Architecture Overview for the full Site B hardware list and planned roles.
Next Steps
Immediate build sequence from this state, in order:
- Re-provision
sa-fw-01— bootstrap runbook step 3.1:make opnsense-provision-set(mgmt IP + root hash) →opnsense-provision-preview→opnsense-provision-up, thenmake opnsense-config-check→opnsense-config-apply. Re-apply the wizard reference values above. Everything below assumes a running OPNsense VM. - Build VLAN interfaces on
vtnet1— create sub-interfaces on the OPNsense LAN trunk for VLANs 10, 20, 25, 30, 40, 50, 60, 70, 80, 90, 100, 110, and 120. Assign gateway IPs and DHCP scopes per the IP Addressing plan. - Configure
sa-sw-01VLAN trunk — set VLANs and port membership on the Netgear XS716T per the Site A Port Map. Port 1 carries the tagged OPNsense LAN trunk for all internal VLANs. - Migrate Proxmox management to VLAN 20 — move each node's management interface from
192.168.1.0/24to its final10.10.20.x/24IP. SSH drops during the network restart; keep IPMI or a physical console open. - Remove the temporary bootstrap NIC — delete
vtnet2fromsa-fw-01once the GUI is reachable on VLAN 20. - Assign Corosync IPs (VLAN 25) — configure
10.10.25.x/24on each node's Corosync interface and verify node-to-node reachability before clustering. - Update
/etc/hosts— all four nodes must resolve each other by final VLAN 20 IP before the cluster is created. - Form
sa-pvecluster — create onsa-stor-01, then joinsa-edge-01,sa-cmp-01,sa-cmp-02. - ZFS storage on
sa-stor-01— configure mirror vdevs from the Samsung SM863 1.92 TB drives. - Deploy PBS-A and DNS VMs — Proxmox Backup Server (PBS-A) at
10.10.30.20/10.10.90.40; Technitium DNS atsa-dns-01(10.10.30.10) andsa-dns-02(10.10.30.11). - WAN cutover and WireGuard — move WAN from UniFi to the Fios ONT directly; enable WAN security rules; build WireGuard site-to-site VPN to Site B.
See Build Phases for the full phased build sequence with verification checkpoints.
Build Phases
End-to-end build sequence for the two-site homelab: seven phases from flat-network bootstrap to a full Kubernetes stack on Proxmox, OPNsense, Ceph, and WireGuard.
IPMI / KVM Remote Console
Runbook for Supermicro IPMI access and remote ISO installation via Java iKVM, HTML5 KVM, IPMIView, and SMB virtual media.