Switch Operations
Operational guide for the Netgear XS716T/XS748T core switches and UniFi access switches: roles, management addresses, VLAN assignments, jumbo frame policy, and common operational gotchas.
Netgear and UniFi switches each fill a distinct role across both sites. This page covers their traffic split, management addresses, VLAN assignments, jumbo frame policy, and operational procedures including factory reset, firmware updates, fan noise, and UniFi-specific gotchas.
Switch Roles
The switching layer uses a two-tier model: Netgear core switches carry server, storage, and infrastructure traffic; UniFi and access switches carry client, AP, PoE, and legacy user traffic.
No routing on core switches
sa-sw-01 and sb-sw-01 are L2 core only — no routing, no DHCP. OPNsense on sa-fw-01 / sb-fw-01 is the infrastructure router and firewall.
| Switch | Model | Site | Role |
|---|---|---|---|
sa-sw-01 | Netgear XS716T | Site A | 10 Gb core — Proxmox, storage, K8s, backup, VM |
sb-sw-01 | Netgear XS748T | Site B | 10 Gb core — Proxmox, storage, K8s, backup, VM |
sa-sw-02 | Access switch | Site A | IPMI, Proxmox mgmt, edge uplink |
sa-sw-03 | Access switch | Site A | Compute mgmt, AP (sa-ap-01), spare |
sb-sw-02 | UniFi USW 24 PoE | Site B | Access, IPMI, APs, client devices |
Management Addresses
All switch management interfaces sit on VLAN 10 (Network Mgmt / IPMI). The UniFi routers (sa-gw / sb-gw) also appear here in their demoted bootstrap role.
| Device | Site A | Site B |
|---|---|---|
Netgear core switch (sa-sw-01 / sb-sw-01) | 10.10.10.2 | 10.20.10.2 |
Access switch #1 (sa-sw-02 / sb-sw-02) | 10.10.10.3 | 10.20.10.3 |
Access switch #2 (sa-sw-03) | 10.10.10.4 | — |
Demoted UniFi router mgmt (sa-gw / sb-gw) | 10.10.10.5 | 10.20.10.4 |
AP sa-ap-01 (U7 Pro XGS) | 10.10.10.6 | — |
VLAN Assignments
Infrastructure-only VLANs (25, 30, 40, 50, 60, 65, 70, 80, 90) are exclusive to the core switches. VLANs 10, 20, 100, 110, and 120 appear on both tiers — VLAN 20 (Proxmox Management) is deliberately offloaded to the access switches at Site A. Never trunk the infrastructure-only VLANs through the access switches — Corosync, VM Services, Kubernetes, storage, Ceph, DMZ, monitoring, and backup traffic must stay on the Netgear core.
Core switches (sa-sw-01 / sb-sw-01)
| VLAN | Name |
|---|---|
| 10 | Network Mgmt / IPMI |
| 20 | Proxmox Management |
| 25 | Corosync heartbeat |
| 30 | VM Services |
| 40 | Kubernetes Nodes |
| 50 | K8s LB / VIPs |
| 60 | Storage / Ceph public |
| 65 | Ceph cluster |
| 70 | DMZ |
| 80 | Monitoring |
| 90 | Backup / Replication |
Access switches (sa-sw-02, sa-sw-03, sb-sw-02)
| VLAN | Name |
|---|---|
| 10 | Network Mgmt / IPMI |
| 20 | Proxmox Management |
| 100 | Lab / Trusted Client |
| 110 | IoT |
| 120 | Guest WiFi |
Site A VLAN 20 offload
At Site A, Proxmox Management (VLAN 20) and IPMI (VLAN 10) are served through sa-sw-02 and sa-sw-03 rather than sa-sw-01. This frees ports on the Netgear core for dedicated Corosync uplinks (VLAN 25). See Site A Port Map for full cabling detail.
Jumbo Frames
The default MTU across all VLANs is 1500 bytes. Jumbo frames (9000-byte MTU) are enabled only on storage VLANs 60, 65, and 90.
No jumbo frames on Corosync
VLAN 25 (Corosync heartbeat) must never use jumbo frames. Corosync traffic stays at the 1500-byte default MTU regardless of switch capabilities.
Configure per-port MTU on the Netgear switches to match the intended VLAN:
| Traffic class | VLANs | MTU |
|---|---|---|
| Default / all other | all others | 1500 |
| Storage / Ceph public | 60 | 9000 |
| Ceph cluster | 65 | 9000 |
| Backup / Replication | 90 | 9000 |
| Corosync heartbeat | 25 | 1500 |
Netgear Operational Notes
Factory Reset
The Netgear XS716T / XS748T factory reset requires the switch to be powered on. Hold the recessed reset button for the required duration (check the front-panel label; typically 5–10 seconds until status LEDs cycle). The switch reboots and reverts to factory defaults including the default management IP.
If the Netgear Discovery Tool finds the device but the web UI does not load:
- Confirm your workstation IP is in the same subnet as the switch's factory management IP.
- Try a different browser; older Netgear firmware may require HTTP rather than HTTPS, or present TLS negotiation errors with modern browsers.
- Check whether the switch is in the wrong management VLAN — a partial config may have changed the VLAN before reset.
Firmware Update
Unzip before uploading
Netgear firmware is distributed as a .zip archive. Extract it first; upload the inner image file (.stk or .bin), not the zip itself.
The XS716T / XS748T maintain dual firmware image slots (primary and secondary). Upload the new image to the inactive slot to preserve a rollback path, then set it as the active boot image and reboot. If the updated image fails to boot, fall back by selecting the previous slot from the boot menu.
What MIBs are
Netgear firmware packages sometimes include .mib files. These are SNMP Management Information Base definitions for monitoring systems (Grafana/Prometheus SNMP exporters, etc.) — they are not firmware and do not need to be uploaded to the switch.
Fan Noise
sa-sw-01 (Netgear XS716T) ships with high-speed 40 mm fans that generate noticeable noise at idle.
Do not reverse fan direction without checking airflow
The XS716T has a fixed front-to-back (or back-to-front) airflow path depending on the SKU. Reversing fan direction without confirming the chassis airflow direction from service documentation risks hot-spot formation and thermal shutdown.
Quieting options, in order of preference:
- Confirm fan health — a failing bearing causes high-pitched whine distinct from normal airflow noise. Replace the fan if RPM is erratic.
- Clean the fan and intake vents — dust buildup causes fans to spin at maximum speed.
- Replace with a compatible lower-noise fan — match the physical connector, voltage, and PWM signal. Verify static-pressure rating is not reduced below the switch's thermal requirement.
- Relocate the switch to a closet or rack with acoustic treatment.
UniFi Operational Notes
Network Deletion Client Conflict
Deleting a UniFi network that still has a client or static-IP assignment fails with an error similar to:
Network includes a Client's "40:a6:e8:e6:9f:ec" configuration.
Please remove this first before deleting the Network.The referenced MAC address may not appear in the active clients list. Search in all of the following locations before retrying the delete:
- Clients list (include offline / hidden clients)
- Fixed IP / DHCP reservations manager
- Port profiles that reference the network by name
- Firewall rules that reference the network
- Per-device network overrides under Devices → [device] → Config → Network
Restarting the UniFi router may clear stale DHCP lease timeouts but does not remove stored client configuration entries. The association must be deleted explicitly from one of the above locations.
UniFi Role in the Final Architecture
UniFi switches and APs remain in service permanently for their access, PoE, and Wi-Fi roles. The UniFi routers (sa-gw / sb-gw) are demoted to bootstrap/fallback and sit behind OPNsense on VLAN 253 (UniFi WAN transit).
UniFi must not route infrastructure traffic
OPNsense handles all infrastructure routing, VLAN gateways, firewall policy, DHCP, and site-to-site WireGuard. UniFi routing devices are bootstrap/fallback only. Proxmox and OpenShift nodes must never sit behind UniFi routing.
Related Pages
- Site A Port Map — per-port VLAN and cabling detail for Site A
- Site B Port Map — per-port VLAN and cabling detail for Site B
- VLAN Reference — full VLAN table with subnets and gateways
- IP Tables — master IP reference by host and VLAN
Site B Port Map
Authoritative switch port map for Site B: sb-sw-01 (Netgear XS748T 48-port 10 Gb core) and sb-sw-02 (UniFi USW 24 PoE access), with per-node NIC-to-VLAN wiring for all five compute nodes and the edge device.
Firewall / OPNsense
OPNsense's role as the infrastructure router and firewall at each site, VM placement constraints, the intentional double-NAT UniFi topology, and the six-phase migration approach.