NIC Inventory & Allocation
Network interface card inventory across both sites, including the six 10G multiport cards, two ThinkPad X550-T2 cards, onboard ports, and per-node VLAN wiring.
The inventory includes six 10G multiport add-in cards, all allocated with no spare, and two Intel X550-T2 cards for the ThinkPads. The tables also list onboard ports by platform and the per-node VLAN wiring for both sites.
10G Multiport NIC Accounting
Six 10G multiport add-in cards are in scope. All 6 are allocated; no spare exists and no Site B node carries two cards (confirmed 2026-06-06).
| Card | Count | Destination |
|---|---|---|
| Intel X710-T4 (quad-port 10GBASE-T) | 5 | Site B: one per node: sb-cmp-01 - sb-cmp-05 |
| XL710/X710-class quad-port 10GBASE-T | 1 | Site A: sa-stor-01, existing, already installed |
All 6 Cards Allocated: No Spare
No spare X710-T4 remains. No Site B node carries two cards. Any reallocation requires sourcing an additional card before proceeding.
Thinkpad Add-In NICs
| Host | Card | Ports used |
|---|---|---|
sa-cmp-01 (ThinkPad P51) | Intel X550-T2 | 1 |
sa-cmp-02 (ThinkPad P52) | Intel X550-T2 | 1 |
Each ThinkPad carries one X550-T2 dual-port card; one port is active per host.
Onboard NIC Layouts by Platform
| Platform | Hosts | Onboard 10G | Onboard 1G | Onboard SFP+ | IPMI |
|---|---|---|---|---|---|
| SYS-E200-8D | sa-edge-01, sb-edge-01 | 2 × 10GBASE-T | 2 × 1GbE | - | Dedicated port |
| SYS-5019D-4C-FN8TP | sb-cmp-01, sb-cmp-02 | 2 × 10GBASE-T | 4 × 1GbE | 2 × 10G SFP+ | Dedicated port |
| SYS-5018D-FN4T | sb-cmp-03, sb-cmp-04, sb-cmp-05 | 2 × 10GBASE-T | 2 × 1GbE | - | Dedicated port |
| 5049A-T (X11SPA-T) | sa-stor-01 | 1 × 10GBASE-T (AQC107) | 1 × 1GbE (i210, red RJ45) | - | Dedicated port |
| ThinkPad P51 / P52 | sa-cmp-01, sa-cmp-02 | - | 1 × 1GbE (onboard) | - | None |
`sa-stor-01` AQC107: Do Not Use for Critical Traffic
The onboard 10GBASE-T port on sa-stor-01 uses the Aquantia AQC107 chip, driven by the atlantic kernel driver. This driver is historically unstable under Linux and Proxmox. Do not assign Proxmox management, Corosync, or any other critical VLAN to this port. Use it only for non-critical data, or leave it unused. This is why Corosync on sa-stor-01 stays tagged on XL710 port 1 rather than moving to the onboard 10G.
Site B per-Node Wiring
The table below applies to the five compute/storage nodes (sb-cmp-01 through sb-cmp-05). sb-edge-01 uses onboard NICs only and is covered under Edge Node Wiring (Site A and Site B). The FN8TP onboard SFP+ ports are present but unused.
| Interface | VLAN | Assignment | Notes |
|---|---|---|---|
| Onboard 1G #1 | 20 | Proxmox Management | Host management and Proxmox UI |
| Onboard 1G #2 | 25 | Corosync heartbeat | Dedicated link; no GW |
| Onboard 10GBASE-T | 65 | Ceph cluster | Cluster replication only; no GW |
| X710-T4 p1 | 30 | VM Services | - |
| X710-T4 p2 | 40 / 50 | Kubernetes Nodes / K8s LB / VIPs | Tagged trunk |
| X710-T4 p3 | 60 | Storage / Ceph public | - |
| X710-T4 p4 | 90 | Backup / Replication | - |
| IPMI | 10 | Network Mgmt / IPMI | Connected to sb-sw-02 (UniFi USW 24 PoE) |
Ceph Public vs. Cluster Split
Ceph public traffic (VLAN 60, Storage / Ceph public) runs on X710-T4 p3. Ceph cluster replication (VLAN 65, Ceph cluster) runs on the onboard 10GBASE-T copper port. These are intentionally separate interfaces and separate VLANs.
Edge Node Wiring (Site A and Site B)
sa-edge-01 (SYS-E200-8D)
| Interface | Assignment | Notes |
|---|---|---|
| Onboard 10G #1 | WAN (Fios, 2 Gbps) | ISP uplink; passed through to OPNsense VM (sa-fw-01) |
| Onboard 1G #2 | Corosync heartbeat (VLAN 25) | Dedicated; no GW |
The approved target redistributes Site A NICs across three equal Layer-2 switches. VLANs traverse the RSTP fabric to OPNsense when routing is required; physical attachment does not create a gateway owner. See the authoritative Site A Port Map.
sb-edge-01 (SYS-E200-8D)
| Interface | Assignment | Notes |
|---|---|---|
| Onboard 1G #1 | WAN (ISP, 1 Gbps) | Passed through to OPNsense VM (sb-fw-01); Site B ISP uplink is 1 Gbps |
| Onboard 1G #2 | Corosync heartbeat (VLAN 25) | Dedicated; no GW |
| Onboard 10GBASE-T (×2) | OPNsense LAN trunk | LAN-side trunk to sb-sw-01 for infrastructure VLANs |
Unlike sa-edge-01 where WAN is on 10G #1 (Fios 2 Gbps), sb-edge-01 uses a 1G port for WAN, freeing both onboard 10G ports for OPNsense LAN-side trunking to the Site B core switch.
sa-stor-01 (Supermicro 5049A-T / X11SPA-T)
| Interface | Assignment | Notes |
|---|---|---|
| Onboard 1GbE (i210, red RJ45) | Proxmox Management (VLAN 20) | Only critical onboard port used for management |
| Onboard 10GBASE-T (AQC107) | avoid: non-critical or unused | atlantic driver instability; never assign mgmt or Corosync |
| XL710 p1 | Corosync (VLAN 25) + VM Services (VLAN 30) | Co-tagged trunk; Corosync is not on a dedicated link due to AQC107 instability |
| XL710 p2-p4 | Storage, Backup, and additional services | See Site A Port Map for per-port VLAN detail |
sa-cmp-01 / sa-cmp-02 (Thinkpad P51 / P52)
| Interface | Assignment | Notes |
|---|---|---|
| X550-T2 p1 | Corosync heartbeat (VLAN 25) + trunk | Corosync is shared on this port, not a dedicated link |
No LACP: All Links Are Single
All node and access switch links are single copper connections (Cat6a). No LACP bonding is used anywhere in the design.
Related Pages
- Per-site Inventory: full hardware BOM with roles and platform notes
- Site A Port Map: switch port assignments and VLAN tagging at Site A
- Site B Port Map: switch port assignments and VLAN tagging at Site B
- VLAN Reference: VLAN IDs, subnets, and gateway assignments