Firewall / OPNsense
OPNsense's role as the infrastructure router and firewall at each site, VM placement constraints, the intentional double-NAT UniFi topology, and the six-phase migration approach.
OPNsense (sa-fw-01 at Site A, sb-fw-01 at Site B) is the infrastructure router and firewall at each site, handling WAN, VLAN gateways, DHCP, firewall policy, and the WireGuard site-to-site VPN. Each OPNsense VM runs on its local Supermicro SYS-E200-8D and is never HA-migrated; UniFi routers remain in place as bootstrap/fallback only, behind OPNsense, with user traffic intentionally double-NATed.
OPNsense at Each Site
One OPNsense VM runs at each site, pinned to the local SYS-E200-8D node.
| VM | Host | Site | Proxmox Mgmt IP |
|---|---|---|---|
sa-fw-01 | sa-edge-01 | Site A | 10.10.20.10 |
sb-fw-01 | sb-edge-01 | Site B | 10.20.20.10 |
OPNsense is the infrastructure router — Proxmox and OpenShift must not sit behind UniFi
OPNsense owns WAN, VLAN gateways, DHCP, firewall policy, and WireGuard at each site. All Proxmox and OpenShift nodes connect to the OPNsense-side network. UniFi routing devices are bootstrap/fallback and user-side only — never the primary infrastructure gateway.
OPNsense VM stays pinned to its local E200 — no HA migration
sa-fw-01 must remain on sa-edge-01; sb-fw-01 must remain on sb-edge-01. Proxmox HA must not be enabled for either firewall VM. Migrating the firewall VM off its local E200 would black-hole all site traffic.
The SYS-E200-8D nodes join their local Proxmox cluster but carry only lightweight workloads alongside OPNsense. Suitable co-residents include a DNS helper, UniFi controller, WireGuard helper, small reverse proxy, or monitoring agent. Heavy databases, Ceph OSDs, and Kubernetes workers must not run on the E200.
UniFi as Bootstrap / Fallback
Existing UniFi routers remain in place at both sites so current Wi-Fi and user devices continue to work without disruption. OPNsense is inserted upstream of UniFi.
ISP/ONT
|
OPNsense (sa-fw-01 / sb-fw-01) <-- WAN terminates here
|
10 Gb core switch
|-- Proxmox / storage / K8s (OPNsense-side, no UniFi)
|
UniFi Router WAN
|
Existing UniFi LAN / Wi-Fi / user devicesInfrastructure traffic (Proxmox, OpenShift, Proxmox Backup Server (PBS)) reaches the internet directly through OPNsense. User traffic traverses two NAT hops: UniFi NAT then OPNsense NAT.
Double NAT for user devices is intentional
UniFi sits behind OPNsense, producing double NAT for user devices. This is a deliberate trade-off to preserve existing Wi-Fi and user connectivity without reconfiguring every client. It is not a misconfiguration.
The UniFi WAN transit network is VLAN 253. OPNsense is the gateway (.1); the UniFi router WAN port takes a static address (.2).
| Site | Transit subnet | OPNsense GW | UniFi WAN |
|---|---|---|---|
| Site A | 10.10.253.0/24 | 10.10.253.1 | 10.10.253.2 |
| Site B | 10.20.253.0/24 | 10.20.253.1 | 10.20.253.2 |
Migration Approach
Migration proceeds in six phases, one site at a time. Export the OPNsense configuration after every major milestone.
| Phase | Action |
|---|---|
| 1 | Insert OPNsense upstream of UniFi at Site A (ISP → OPNsense → UniFi WAN → existing users) |
| 2 | Repeat at Site B |
| 3 | Build WireGuard site-to-site VPN between sa-fw-01 and sb-fw-01 |
| 4 | Move Proxmox nodes to final VLAN 20 management IPs |
| 5 | Build Proxmox clusters (sa-pve, sb-pve) |
| 6 | Storage, PBS, and Kubernetes |
Full per-phase configuration steps are in Migration Phases. The transit network wiring and handoff details between OPNsense and UniFi are in UniFi Handoff.
Related Pages
- Migration Phases — step-by-step configuration for each phase
- UniFi Handoff — transit network, double-NAT model, and static routing
- WireGuard — site-to-site VPN between
sa-fw-01andsb-fw-01 - Architecture Overview — site roles, E200 workload guidance, and all hard rules
Switch Operations
Operational guide for the Netgear XS716T/XS748T core switches and UniFi access switches: roles, management addresses, VLAN assignments, jumbo frame policy, and common operational gotchas.
Migration Phases
Detailed walkthrough of the six OPNsense migration phases: inserting OPNsense upstream of UniFi at each site, building the WireGuard site-to-site VPN, migrating Proxmox to final management IPs, and standing up clusters, storage, and Kubernetes.