Site B Port Map
Site B port assignments for sb-sw-01 and sb-sw-02, with NIC-to-VLAN wiring for the five compute nodes and the edge device.
sb-sw-01 (Netgear XS748T, 48-port 10 Gb core) carries all infrastructure VLANs for Site B; sb-sw-02 (UniFi USW 24 PoE) handles IPMI, APs, and client access. The tables map every sb-sw-01 port, all six Site B hosts, and the known sb-sw-02 connections.
L2 Core Only: No Routing on sb-sw-01
sb-sw-01 is L2 only: no routing, no DHCP. All inter-VLAN routing is handled by OPNsense on sb-edge-01. sb-sw-02 carries only VLANs 10, 20, 100, 110, and 120: never VLANs 25, 30, 40, 50, 60, 65, 70, 80, or 90.
Per-Node NIC Wiring
Standard Compute Nodes: sb-cmp-01 Through sb-cmp-05
All five compute nodes follow this identical NIC assignment. Each node carries one Intel X710-T4 (4 × 10GBASE-T) for tagged data trunks. The FN8TP onboard SFP+ ports are unused because sb-sw-01 has only four SFP+ combo slots.
| NIC | Connects to | VLANs / Role |
|---|---|---|
| Onboard 1G #1 | sb-sw-01 (Access) | 20 Proxmox Management |
| Onboard 1G #2 | sb-sw-01 (Access) | 25 no GW Corosync heartbeat: dedicated, site-local |
| Onboard 10GBASE-T | sb-sw-01 (Trunk) | 65 no GW Ceph cluster / OSD replication: site-local |
| X710-T4 port 1 | sb-sw-01 (Trunk) | 30 VM Services |
| X710-T4 port 2 | sb-sw-01 (Trunk) | 40 50 Kubernetes Nodes + K8s LB / VIPs |
| X710-T4 port 3 | sb-sw-01 (Trunk) | 60 Storage / Ceph public |
| X710-T4 port 4 | sb-sw-01 (Trunk) | 90 Backup / Replication |
| Dedicated IPMI | sb-sw-02 (Access) | 10 Network Mgmt / IPMI |
Ceph Network Split: Decided 2026-06-06
Ceph public traffic runs on VLAN 60 via the X710-T4; clients attach directly to that VLAN. Ceph cluster traffic runs on VLAN 65 via onboard 10GBASE-T. Both are site-local with no gateway, and 60↔65 routing is prohibited. See VLAN Reference.
sb-edge-01 NIC Wiring
sb-edge-01 (Supermicro SYS-E200-8D, running OPNsense VM sb-fw-01) differs from the compute node pattern: onboard 1G #1 is the WAN uplink to the ISP, so management and Corosync shift to the remaining interfaces.
| NIC | Connects to | VLANs / Role |
|---|---|---|
| Onboard 1G #1 | Site B ISP handoff (WAN) | OPNsense WAN: 1 Gbps; not on any switch |
| Onboard 1G #2 | sb-sw-01 port 2 (Access) | 20 Proxmox Management |
| Onboard 10G #1 | sb-sw-01 port 3 (Access) | 25 no GW Corosync heartbeat |
| Onboard 10G #2 | sb-sw-01 port 1 (Trunk) | All VLANs: OPNsense LAN, VLAN-aware Linux bridge |
| Dedicated IPMI | sb-sw-02 (Access) | 10 Network Mgmt / IPMI |
sb-sw-01: Netgear XS748T Core Switch
48-port 10 Gb switch. Uses 41 of 48 ports: 3 for sb-edge-01, 7 per compute node (5 nodes = 35 ports), plus 3 for the sb-sw-02 uplink, existing USG Pro WAN, and reserved emergency-administration connection. All IPMI connections land on sb-sw-02, not on sb-sw-01.
| Port | Connected device / interface | Port mode | VLAN assignment |
|---|---|---|---|
| 1 | sb-edge-01 OPNsense LAN interface (onboard 10G port 2) | Trunk | Tagged 10 20 25 30 40 50 60 65 70 80 90 100 110 120 253 |
| 2 | sb-edge-01 Proxmox management interface (onboard 1G port 2) | Access | Untagged 20 |
| 3 | sb-edge-01 Corosync (onboard 10G #1) | Access | Untagged 25 |
| 4 | sb-cmp-01 Proxmox management interface (onboard 1G port 1) | Access | Untagged 20 |
| 5 | sb-cmp-01 Corosync (onboard 1G #2) | Access | Untagged 25 |
| 6 | sb-cmp-01 Ceph cluster (onboard 10GBASE-T) | Trunk | Tagged 65 |
| 7 | sb-cmp-01 X710-T4 port 1: VM Services | Trunk | Tagged 30 |
| 8 | sb-cmp-01 X710-T4 port 2: Kubernetes Nodes / K8s LB / VIPs | Trunk | Tagged 40 50 |
| 9 | sb-cmp-01 X710-T4 port 3: Ceph public | Trunk | Tagged 60 |
| 10 | sb-cmp-01 X710-T4 port 4: Backup / Replication | Trunk | Tagged 90 |
| 11 | sb-cmp-02 Proxmox management interface (onboard 1G port 1) | Access | Untagged 20 |
| 12 | sb-cmp-02 Corosync (onboard 1G #2) | Access | Untagged 25 |
| 13 | sb-cmp-02 Ceph cluster (onboard 10GBASE-T) | Trunk | Tagged 65 |
| 14 | sb-cmp-02 X710-T4 port 1: VM Services | Trunk | Tagged 30 |
| 15 | sb-cmp-02 X710-T4 port 2: Kubernetes Nodes / K8s LB / VIPs | Trunk | Tagged 40 50 |
| 16 | sb-cmp-02 X710-T4 port 3: Ceph public | Trunk | Tagged 60 |
| 17 | sb-cmp-02 X710-T4 port 4: Backup / Replication | Trunk | Tagged 90 |
| 18 | sb-cmp-03 Proxmox management interface (onboard 1G port 1) | Access | Untagged 20 |
| 19 | sb-cmp-03 Corosync (onboard 1G #2) | Access | Untagged 25 |
| 20 | sb-cmp-03 Ceph cluster (onboard 10GBASE-T) | Trunk | Tagged 65 |
| 21 | sb-cmp-03 X710-T4 port 1: VM Services | Trunk | Tagged 30 |
| 22 | sb-cmp-03 X710-T4 port 2: Kubernetes Nodes / K8s LB / VIPs | Trunk | Tagged 40 50 |
| 23 | sb-cmp-03 X710-T4 port 3: Ceph public | Trunk | Tagged 60 |
| 24 | sb-cmp-03 X710-T4 port 4: Backup / Replication | Trunk | Tagged 90 |
| 25 | sb-cmp-04 Proxmox management interface (onboard 1G port 1) | Access | Untagged 20 |
| 26 | sb-cmp-04 Corosync (onboard 1G #2) | Access | Untagged 25 |
| 27 | sb-cmp-04 Ceph cluster (onboard 10GBASE-T) | Trunk | Tagged 65 |
| 28 | sb-cmp-04 X710-T4 port 1: VM Services | Trunk | Tagged 30 |
| 29 | sb-cmp-04 X710-T4 port 2: Kubernetes Nodes / K8s LB / VIPs | Trunk | Tagged 40 50 |
| 30 | sb-cmp-04 X710-T4 port 3: Ceph public | Trunk | Tagged 60 |
| 31 | sb-cmp-04 X710-T4 port 4: Backup / Replication | Trunk | Tagged 90 |
| 32 | sb-cmp-05 Proxmox management interface (onboard 1G port 1) | Access | Untagged 20 |
| 33 | sb-cmp-05 Corosync (onboard 1G #2) | Access | Untagged 25 |
| 34 | sb-cmp-05 Ceph cluster (onboard 10GBASE-T) | Trunk | Tagged 65 |
| 35 | sb-cmp-05 X710-T4 port 1: VM Services | Trunk | Tagged 30 |
| 36 | sb-cmp-05 X710-T4 port 2: Kubernetes Nodes / K8s LB / VIPs | Trunk | Tagged 40 50 |
| 37 | sb-cmp-05 X710-T4 port 3: Ceph public | Trunk | Tagged 60 |
| 38 | sb-cmp-05 X710-T4 port 4: Backup / Replication | Trunk | Tagged 90 |
| 39 | sb-sw-02: core uplink | Trunk | Tagged 10 20 100 110 120 |
| 40 | USG Pro WAN: existing VLAN 253 transit | Access | Untagged 253 |
| 41 | Emergency administration connection (spare) | Access | Untagged 10 or 20 |
USG Pro WAN Port: Discrepancy Resolved 2026-07-03
Port 40 is the demoted USG Pro WAN connection, per the vault/14 port table. An earlier discrepancy (the vault/14 WAN/Edge prose said port 29: actually sb-cmp-04 X710-T4 port 2) was corrected in the vault on 2026-07-03; table and prose now agree on port 40.
sb-sw-02: UniFi USW 24 PoE Access Switch
sb-sw-02 handles IPMI / BMC for all six Supermicro nodes, PoE for APs, and client/lab access. It uplinks to sb-sw-01 port 39. The demoted USG Pro serves existing Wi-Fi users and the client LAN. Its WAN interface connects to sb-sw-01 port 40 (VLAN 253), and its LAN side intentionally remains behind double NAT.
The vault does not document specific port numbers within sb-sw-02. The table below records known device assignments by VLAN role.
| Connected device | Port mode | VLAN assignment | Function |
|---|---|---|---|
sb-edge-01 IPMI | Access | 10 | Network Mgmt / IPMI |
sb-cmp-01 IPMI | Access | 10 | Network Mgmt / IPMI |
sb-cmp-02 IPMI | Access | 10 | Network Mgmt / IPMI |
sb-cmp-03 IPMI | Access | 10 | Network Mgmt / IPMI |
sb-cmp-04 IPMI | Access | 10 | Network Mgmt / IPMI |
sb-cmp-05 IPMI | Access | 10 | Network Mgmt / IPMI |
Uplink to sb-sw-01 port 39 | Trunk | 10 20 100 110 120 | Core-switch uplink |
| Wireless access points and client-facing ports | Access or trunk | 100 110 120 | Lab / Trusted Client, IoT, and Guest WiFi |
IPMI Must Not Be Exposed to the Internet
All BMC / IPMI interfaces connect only to sb-sw-02 on VLAN 10 (Network Mgmt / IPMI). This VLAN is unreachable from the internet and must never be routed across the WireGuard tunnel.
VLAN 253: UniFi WAN Transit
VLAN 253 is tagged on sb-edge-01's OPNsense LAN trunk (port 1) so OPNsense can provide the WAN gateway (10.20.253.1) to the demoted USG Pro. The USG Pro WAN port lands on sb-sw-01 port 40 as an untagged VLAN 253 access port. Its LAN side intentionally continues to serve existing Wi-Fi users and clients behind double NAT.
The cutover from the current USG Pro WAN arrangement to OPNsense-as-upstream is a physical cable move at Site B Phase 2: relocate the single ISP handoff cable from the USG Pro to sb-edge-01's onboard 1G #1, then feed the USG Pro its WAN from OPNsense via VLAN 253.
Related Pages
- Site A Port Map: separate three-switch L2 target; do not copy its VLAN 999 profiles or migration sequence to Site B
- Switch Operations: VLAN provisioning and switch management procedures
- VLAN Reference: all 15 VLANs with subnets and gateway assignments
- Per-site Inventory: NIC models, RAM, and storage per host
- IP Tables: per-host IP addresses for all VLANs