Secrets: OpenBao
Two independent OpenBao instances (sa-bao-01, sb-bao-01) on the edge E200s: secret tiers, cross-site transit auto-unseal, AppRole consumers, and the break-glass fallback.
Each site runs one OpenBao (Vault OSS fork) instance on its edge E200. A small Go binary in a roughly 2 GB VM fits the E200 workload boundary. This placement was decided on 2026-07-02. The sites do not share a stretched Raft cluster: OpenBao has no cross-site replication, and a two-node Raft cluster over WireGuard cannot maintain quorum.
Topology
| Site A | Site B | |
|---|---|---|
| VM | sa-bao-01 on sa-edge-01 | sb-bao-01 on sb-edge-01 |
| Final address | 10.10.30.40/24 (VLAN 30) | 10.20.30.40/24 (VLAN 30) |
| Specs | 2 vCPU / 2 GB / 20 GB, Ubuntu noble | same |
| Storage | single-node integrated Raft | same |
| Version | 2.5.4 (pinned tarball, sha256-verified) | same |
Cross-site disaster recovery uses scheduled bao operator raft snapshot save, shipped to the other site over the backup path (VLAN 90, PBS pattern): each site's secrets are restorable at the other.
Seal / Unseal
Each instance auto-unseals via the other site's transit engine over the WireGuard tunnel: sa-bao-01 points at sb-bao-01 and vice versa. Because the tunnel is the unseal path, Pulumi declares OPNsense order=1,up=90,down=180 before bao order=2,up=60,down=180; Site A UOS follows at order=3,up=0,down=180. Proxmox applies up after the current VM starts, so the firewall's 90-second value is the delay before bao.
Cold-Start Deadlock: Documented Break-Glass
If both sites are down at once, neither can auto-unseal. To recover, seal-migrate one instance back to Shamir with a temporary seal stanza swap and the recovery keys. Unseal it manually, let the other instance auto-unseal against it, and then migrate back. Both sites' recovery keys live in the password manager; this is the documented recovery path.
Interim: Site A Only (2026-07-02)
Site B is non-operational, so sa-bao-01 runs standalone with Shamir manual unseal (3 of 5 keys after the rare VM reboot). This is the first bootstrap step, not temporary work. Transit auto-unseal follows when Site B comes online. The site-b Pulumi stack stays parked at enabled: false.
Secret Tiers
| Tier | Store | Contents |
|---|---|---|
| 0: bootstrap | repo-root .env.local (gitignored) | Proxmox creds, PULUMI_CONFIG_PASSPHRASE, ANSIBLE_VAULT_PASSWORD: everything needed to (re)build bao itself |
| 1: runtime | OpenBao KV v2 (per site) | app/service credentials, API keys, cert material |
Tier 0 can never migrate into bao: bao cannot hold the secrets that build bao.
Consumers (bao-first, Executed 2026-07-01)
The KV v2 mount is homelab, with paths opnsense/site-a, unifi/site-a, and pulumi/proxmox; Site B mirrors under */site-b on sb-bao-01 later. Access is per-consumer read-only via AppRole (opnsense-config, unifi-config, pulumi-provision), with role_id/secret_id in root .env.local.
- Ansible secret variables in
opnsense/configandunifi/configuse KV v2 lookups. When bao is unreachable or sealed, they fall back to the encryptedvault-credentials.ymlfiles. Those files remain as the documented break-glass path and are resynchronized withmake openbao-config-seed. - Pulumi make targets overlay
PROXMOX_VE_*fromhomelab/pulumi/proxmoxand fall back to.env.localon failure. Stack encryption stays on the local passphrase provider because a bao-backed provider cannot work while bao is sealed, andaorxi-openbaocannot depend on the service it builds. - Unsetting
BAO_ADDRforces pure fallback mode everywhere.
Still Deferred
Transit engines and seal migration (needs Site B), TLS via Let's Encrypt DNS-01 (sa-bao-01.core.aorxi.io; never expose port 8200 beyond the local segment while TLS is off), the Raft snapshot timer with cross-site shipping, and the api_addr correction at the VLAN 30 cutover. Init and recovery-key handling stay a manual runbook.
Related Pages
- Initial Site Bootstrap: where bao provisioning and the bao-first migration sit in the build order
- DNS VMs: the internal zone that will name the bao endpoints
- Decisions Log: dated entries for the 2026-07-01/02 secrets decisions
DNS Deployment & Cutover Runbook
Install and cluster the four Technitium DNS instances, create replicated zones, enable signed updates, and migrate clients from OPNsense Unbound.
UniFi OS Server (UOS) Controller
One self-hosted UniFi controller for the whole lab: sa-uos-01 on VLAN 10, pinned UOS 5.1.19, Pulumi-provisioned with API-driven first-run setup.