Initial Site Bootstrap
The bare-metal → operational-baseline runbook: temp LAN, hardware install, Proxmox, edge VMs (OPNsense, UOS, OpenBao) via Pulumi, and the bao-first secrets migration.
Everything that happens before the phased cutover work in Build Phases: racked hardware on the temp LAN → Proxmox on every node → edge VMs provisioned and configured → repo secrets running bao-first. End state is "Phase 0 complete, Phase 1 ready," with the whole edge stack rebuildable from the repo. Captured 2026-07-01 from the Site A bring-up; this is the repeatable template for Site B.
Historical Runbook; Target Wiring Superseded 2026-07-26
Phase 0 is complete and all three edge VMs are live. The flat-bootstrap procedure remains here as an operational record. All older cutover destinations are superseded by the OPNsense-routed, three-switch L2 design and A0-A11 sequence. Use Current Build State and Site A Port Map for current work.
The flat addresses below describe a lifecycle, not a permanent fan-out. XG6 p4
serves one host or BMC at a time; after the other systems are built, p4 remains
on sa-edge-01 nic0 so its edge VMs stay reachable. OPNsense continues to
receive WAN service from the UCG, but every downstream system moves to final
10.10.x networks before the A8 ONT handoff.
Temp-LAN Plan (Phase-0 Addressing)
Flat network behind the existing UniFi router, with no VLANs yet. At Site A this is the observed bootstrap state, not the approved final topology.
| Site A | Site B | |
|---|---|---|
| Temp subnet | 192.168.0.0/23 (revised 2026-07-04) | 192.168.16.0/24 |
| Gateway (UniFi router) | 192.168.0.1 | 192.168.16.1 |
*-edge-01 Proxmox | 192.168.1.10 | 192.168.16.10 (tentative) |
sa-uos-01 (UOS VM) | 192.168.1.40 | - (no second controller) |
*-fw-01 seed mgmt (OPNsense VM) | 192.168.1.41 | 192.168.16.41 (tentative) |
*-bao-01 (OpenBao VM) | DHCP: lease recorded in BAO_ADDR | DHCP (when built) |
Step 0: Workstation Prerequisites (Tier 0)
Clone the repo, cp .env.local.example .env.local, fill the tier-0 values (PULUMI_CONFIG_PASSPHRASE, Proxmox credential, ANSIBLE_VAULT_PASSWORD), leave the BAO_* knobs empty: every tool runs in pure fallback mode until step 4. ssh-add the operator key (the Proxmox provider authenticates via the agent), then make install.
Step 1: Hardware Install
Install NICs, boot M.2 (512 GB: never a 1.92 TB enterprise SSD), RDIMMs, and data SSDs per the combined port wiring and faceplate guide. Use XG6 p4 as the one-at-a-time bootstrap service lead; do not pre-cable a host to an XG10. Keep XG6 p5/p6 disconnected; no XG10 ever connects to UCG Max or XG6.
IPMI First Contact
Connect one BMC to XG6 p4, change ADMIN/ADMIN (or the sticker password)
immediately, record its bootstrap address, then disconnect it before servicing
the next endpoint. IPMI moves directly to its final VLAN 10 port in A5 and is
never exposed to the internet. macOS iKVM pain: see IPMI / KVM.
Step 2: Proxmox on Bare Metal
Install PVE on every node via IPMI virtual media using the temporary IPs above.
Service and baseline one host at a time through XG6 p4. Attach
sa-edge-01 nic0 to p4 last and leave it there through A4 so the edge VMs
stay available. Add the Pulumi API credential (Datacenter → Permissions → API
Tokens, privilege separation OFF) to .env.local. Apply the host baseline and
hardening one node at a time (make baseline-check LIMIT=<node> →
baseline-apply). Run the edge node last because it will carry the firewall VM.
Step 3: Edge VMs via Pulumi (Order Matters)
All on *-edge-01, all make targets against STACK=site-a|site-b; preview before every up; export and commit config after each milestone.
- OPNsense: provision (
opnsense-provision-set/preview/up: FreeBSD image + cloud-initopnsense-bootstrapconversion, API-ready on the seed IP), then app config (opnsense-config-check/apply: Phase-1 outbound NAT + baseline firewall). - UOS controller (Site A only):
unifi-provision-*, thenunifi-config-setup(first-run owner/console via the UOS API). - OpenBao:
openbao-provision-*(VM boots sealed and uninitialized by design), then the manual init runbook:bao operator init→ Shamir keys + root token → password manager. Pulumi owns the E200 startup policy: OPNsense1/90/180, bao2/60/180, and Site A UOS3/0/180(order/up/down, seconds).
Step 4: Secrets Go bao-first
Run the tier-1 migration for the site (Site A completed it on 2026-07-01). Set BAO_ADDR and the temporary BAO_ADMIN_TOKEN, then run make openbao-config-setup. This creates the KV v2 homelab mount, per-scope policies, and AppRoles. Paste each printed role_id and one-time secret_id pair into .env.local, then run make openbao-config-seed. Remove BAO_ADMIN_TOKEN afterward; the password manager retains it.
Verification Gates
| After step | Gate |
|---|---|
| 0 | make help lists targets; scripts/vault-pass.sh prints, not errors |
| 1 | IPMI reachable on new creds from the temp LAN only |
| 2 | PVE web UI on every node; make baseline-check LIMIT=<node> clean re-run |
| 3.1 | OPNsense API answers on the seed IP; opnsense-config-apply re-run = 0 changes |
| 3.2 | UOS UI on :11443; unifi-config-check clean |
| 3.3 | bao status = unsealed + initialized; VM start order set |
| 4 | Config runs succeed via bao (vault files renamed away) AND via fallback (BAO_ADDR unset) |
Related Pages
- Current Build State: where the build actually stands today
- Build Phases: the phased cutover that follows this runbook
- Secrets: OpenBao: the full secrets architecture