CA & Certificate Infrastructure

Certificate strategy for the AORXI homelab: Let's Encrypt DNS-01 via Cloudflare for OPNsense, Proxmox, PBS, and Kubernetes; step-ca private CA for IPMI/BMC and internal mTLS.

All TLS certificates in this homelab are publicly trusted via Let's Encrypt using Cloudflare DNS-01 challenges — no HTTP challenge servers required, no inbound firewall holes. A private CA (step-ca) covers the narrow exception: IPMI/BMC firmware that cannot run ACME clients, and optional service-to-service mTLS requiring client certificates.

Strategy Overview

                  ┌──────────────────────────────────────────────────────┐
                  │            Let's Encrypt (publicly trusted)           │
                  │   ACME DNS-01 challenge → Cloudflare aorxi.io zone   │
                  └────────────────────────┬─────────────────────────────┘
                                           │ issues certs for:
                      ┌────────────────────┼────────────────────────┐
                      ▼                    ▼                         ▼
              OPNsense WebGUI       Proxmox / PBS            K8s workloads
              (os-acme-client)      (acme.sh)              (cert-manager)
              sa-fw-01.core…        pve.core.aorxi.io      *.core.aorxi.io


                  ┌──────────────────────────────────────────────────────┐
                  │         step-ca (private CA — internal only)          │
                  │                sa-ca-01  10.10.30.30                 │
                  └────────────────────────┬─────────────────────────────┘
                                           │ issues certs for:
                      ┌────────────────────┴────────────────────────┐
                      ▼                                              ▼
               IPMI BMCs (all sites)                 internal mTLS (optional)
               ipmi-sa-edge-01.core.aorxi.io         service-to-service
               ipmi-sa-stor-01.core.aorxi.io
               ipmi-sb-*.core.aorxi.io

Why LE-first: aorxi.io is on Cloudflare, so DNS-01 works for any subdomain including *.core.aorxi.io — every browser and OS already trusts the LE root, and private CA root distribution is avoided for the vast majority of services.

Where private CA is unavoidable: IPMI/BMC firmware cannot run ACME clients. Uploading a cert signed by a known CA (step-ca) is cleaner than accepting persistent self-signed warnings or regenerating self-signed certs manually. Internal mTLS (e.g., Prometheus scrape targets, Proxmox Backup Server (PBS) node auth) can optionally use step-ca short-lived certs.

Design Decisions

Decision	Choice
Primary cert strategy	Let's Encrypt via Cloudflare DNS-01 — no internet exposure required
Public domain	`aorxi.io` on Cloudflare
Internal zone	`*.core.aorxi.io` — LE issues publicly-trusted cert; Technitium serves A records
Private CA	`step-ca` on `sa-ca-01` (`10.10.30.30`) — IPMI BMCs and internal mTLS only
K8s issuance	cert-manager `ClusterIssuer` per cluster, Cloudflare DNS-01 solver
Non-K8s issuance	OPNsense ACME plugin; acme.sh on Proxmox/PBS nodes
IPMI certs	Private CA only — BMC firmware cannot run ACME
Cert renewal	Automatic everywhere: cert-manager (K8s), acme.sh cron (bare-metal), os-acme-client (OPNsense)
Root trust	LE root already trusted everywhere; step-ca root pushed to managed hosts only

Cloudflare API Token

One token, shared by all ACME clients (OPNsense, acme.sh, cert-manager). Create it at: Cloudflare Dashboard → My Profile → API Tokens → Create Token → Custom token.

Scope the token to aorxi.io only

Never grant Zone → DNS → Edit on all zones. Scope explicitly to the aorxi.io zone to limit blast radius if the token is ever leaked.

Required scopes

Permission	Scope
Zone → DNS → Edit	`aorxi.io` (specific zone only)
Zone → Zone → Read	`aorxi.io`

Store the token value — it is shown only once. Referenced as <CF_API_TOKEN> throughout this page.

Let's Encrypt on OPNsense

OPNsense includes the os-acme-client plugin for built-in ACME cert management.

Install the plugin

System → Firmware → Plugins → search os-acme-client → Install.

Configure the ACME account

Services → ACME Client → Accounts → Add:

Field	Value
Name	`letsencrypt`
E-mail	`<your-email>`
ACME URL	`https://acme-v02.api.letsencrypt.org/directory`

Save → Register.

Configure Cloudflare DNS validation

Services → ACME Client → Challenge Types → Add:

Field	Value
Name	`cloudflare-dns01`
Challenge Type	DNS-01
DNS Service	Cloudflare
API Token	`<CF_API_TOKEN>`

Issue certificates

Services → ACME Client → Certificates → Add — one entry per OPNsense instance:

Cert name	Alt name	Host
`sa-fw-01`	`sa-fw-01.core.aorxi.io`	Site A OPNsense WebGUI
`sb-fw-01`	`sb-fw-01.core.aorxi.io`	Site B OPNsense WebGUI

For each cert: Key Length ec-256, ACME Account letsencrypt, Challenge Type cloudflare-dns01, Auto Renewal enabled.

After issuance, bind to the WebGUI: System → Settings → Administration → SSL Certificate → select the issued cert → Save.

Renewal

The plugin handles renewal automatically via an OPNsense cron job. Verify in Services → ACME Client → Log that renewal fires before 30-day expiry.

Let's Encrypt on Proxmox and PBS

acme.sh is used on bare-metal Proxmox and Proxmox Backup Server (PBS) nodes. It gives more control than the Proxmox UI ACME integration and is consistent across all nodes.

Install acme.sh

Run on each Proxmox or PBS node:

curl https://get.acme.sh | sh -s email=<your-email>
source ~/.bashrc

Set Cloudflare credentials

export CF_Token="<CF_API_TOKEN>"
export CF_Account_ID="<CF_ACCOUNT_ID>"   # from Cloudflare dashboard URL

# Persist for renewals
echo 'export CF_Token="<CF_API_TOKEN>"' >> ~/.acme.sh/account.conf
echo 'export CF_Account_ID="<CF_ACCOUNT_ID>"' >> ~/.acme.sh/account.conf

Issue a cert per node

Substitute the hostname for each node:

# Example: sa-stor-01
acme.sh --issue \
  --dns dns_cf \
  -d sa-stor-01.core.aorxi.io \
  --keylength ec-256 \
  --server letsencrypt

Install into Proxmox

# Proxmox expects cert at /etc/pve/local/pve-ssl.pem and key at pve-ssl.key
acme.sh --install-cert \
  -d sa-stor-01.core.aorxi.io \
  --cert-file   /etc/pve/local/pve-ssl.pem \
  --key-file    /etc/pve/local/pve-ssl.key \
  --reloadcmd   "systemctl restart pveproxy"

Install into PBS

acme.sh --install-cert \
  -d sa-pbs-01.core.aorxi.io \
  --cert-file   /etc/proxmox-backup/proxy.pem \
  --key-file    /etc/proxmox-backup/proxy.key \
  --reloadcmd   "systemctl restart proxmox-backup-proxy"

Renewal

acme.sh installs a cron job at install time. Verify:

crontab -l | grep acme
# Expected: 0 0 * * * /root/.acme.sh/acme.sh --cron --home /root/.acme.sh

Let's Encrypt on Kubernetes

Tentative — K8s clusters not yet deployed

cert-manager and ClusterIssuer configuration is planned for Phase 18+ after K8s clusters are built. Steps below are the intended configuration.

Install cert-manager

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true

Cloudflare API token Secret

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
type: Opaque
stringData:
  api-token: "<CF_API_TOKEN>"

ClusterIssuer

Deploy this in both Site A and Site B clusters:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-cloudflare
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: <your-email>
    privateKeySecretRef:
      name: letsencrypt-cloudflare-key
    solvers:
    - dns01:
        cloudflare:
          apiTokenSecretRef:
            name: cloudflare-api-token
            key: api-token

Wildcard certificates

# *.core.aorxi.io — covers all internal services at both sites
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: core-wildcard
  namespace: cert-manager
spec:
  secretName: core-wildcard-tls
  issuerRef:
    name: letsencrypt-cloudflare
    kind: ClusterIssuer
  dnsNames:
  - "*.core.aorxi.io"
  - "core.aorxi.io"
  renewBefore: 360h   # 15 days
---
# *.aorxi.io — externally exposed services
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: aorxi-wildcard
  namespace: cert-manager
spec:
  secretName: aorxi-wildcard-tls
  issuerRef:
    name: letsencrypt-cloudflare
    kind: ClusterIssuer
  dnsNames:
  - "*.aorxi.io"
  - "aorxi.io"
  renewBefore: 360h

Reference the shared Secret in Ingress resources:

spec:
  tls:
  - hosts:
    - myapp.core.aorxi.io
    secretName: core-wildcard-tls

Or let cert-manager issue per-service certs by annotating the Ingress:

metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-cloudflare
spec:
  tls:
  - hosts:
    - myapp.core.aorxi.io
    secretName: myapp-tls

LE rate limits

LE allows 50 certs/domain/week and 5 duplicate certs/week. Wildcard certs count as one issuance regardless of how many Ingresses reference the Secret. Share the wildcard across namespaces using a Secret copy tool (e.g., reflector or kubed) rather than issuing per-namespace certs.

Private CA — step-ca

step-ca runs on sa-ca-01, a lightweight LXC on VLAN 30 (VM Services). It issues certs for IPMI/BMC interfaces that cannot run ACME clients, and optionally for internal mTLS.

VM specification

Field	Value
VM name	`sa-ca-01`
Host	`sa-stor-01`
VLAN	30 (VM Services)
IP	`10.10.30.30`
Specs	1 vCPU, 512 MB RAM, 8 GB disk
OS	Ubuntu 22.04 LXC

Install

# On sa-ca-01 — check https://github.com/smallstep/certificates/releases for current version
wget https://dl.smallstep.com/gh-release/cli/gh-release-header/v0.27.0/step-cli_0.27.0_amd64.deb
wget https://dl.smallstep.com/gh-release/certificates/gh-release-header/v0.27.0/step-ca_0.27.0_amd64.deb
dpkg -i step-cli_0.27.0_amd64.deb step-ca_0.27.0_amd64.deb

Initialize the CA

step ca init \
  --name "Homelab Internal CA" \
  --dns "sa-ca-01.core.aorxi.io,10.10.30.30" \
  --address ":9000" \
  --provisioner "admin@aorxi.io" \
  --deployment-type standalone

This generates:

/root/.step/certs/root_ca.crt — root certificate (distribute to managed hosts)
/root/.step/certs/intermediate_ca.crt — intermediate used for signing
/root/.step/secrets/ — private keys (back these up)

Record the provisioner password in Bitwarden or equivalent — it cannot be recovered.

Run as a service

cat > /etc/systemd/system/step-ca.service << 'EOF'
[Unit]
Description=step-ca
After=network-online.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/step-ca /root/.step/config/ca.json
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl enable --now step-ca

Verify: curl -k https://10.10.30.30:9000/health

Issue IPMI certificates

# Get the root CA fingerprint (needed to trust the CA URL)
step certificate fingerprint /root/.step/certs/root_ca.crt

# Issue cert for an IPMI host — valid 8760h (1 year)
step ca certificate \
  "ipmi-sa-stor-01.core.aorxi.io" \
  ipmi-sa-stor-01.crt \
  ipmi-sa-stor-01.key \
  --ca-url https://10.10.30.30:9000 \
  --root /root/.step/certs/root_ca.crt \
  --not-after 8760h \
  --provisioner "admin@aorxi.io"

Upload the resulting .crt and .key files to the BMC web interface: Supermicro: Configuration → SSL Certificate → Upload (PEM format — step-ca outputs PEM by default).

Repeat for each BMC host.

IPMI certificate renewal

IPMI certs do not auto-renew. Schedule a yearly reminder or a cron that re-issues and re-uploads via the Supermicro Redfish API.

Test Redfish endpoint before relying on it

The curl-based Redfish cert upload below is illustrative. Behavior varies by BMC firmware version. Test against your specific board before scripting this into production renewal.

# Re-issue cert
step ca certificate "ipmi-sa-stor-01.core.aorxi.io" new.crt new.key \
  --ca-url https://10.10.30.30:9000 \
  --root /root/.step/certs/root_ca.crt \
  --provisioner "admin@aorxi.io" \
  --not-after 8760h

# Upload via Redfish (adjust for firmware version)
curl -k -u ADMIN:<password> \
  -X POST https://10.10.10.20/redfish/v1/Managers/1/NetworkProtocol/HTTPS/Certificates \
  -H "Content-Type: application/json" \
  -d "{\"CertificateString\": \"$(cat new.crt new.key)\", \"CertificateType\": \"PEM\"}"

Root CA trust distribution

Push the root CA cert to every host that needs to verify private-CA-issued certs: admin workstations and any service doing mTLS.

# Debian/Ubuntu hosts (Proxmox nodes, DNS VMs, etc.)
scp /root/.step/certs/root_ca.crt root@<host>:/usr/local/share/ca-certificates/homelab-root-ca.crt
ssh root@<host> update-ca-certificates

# macOS admin workstation
security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain homelab-root-ca.crt

Firefox uses its own cert store — import manually via Preferences → Privacy & Security → Certificates → Import.

Do not expose step-ca to the internet

Port 9000 on sa-ca-01 must never be reachable from the internet. Access is restricted to trusted admin VLANs and Site B over WireGuard.

Certificate Inventory

Service	Issuer	Hostname	Renewal
OPNsense Site A	Let's Encrypt	`sa-fw-01.core.aorxi.io`	Auto (os-acme-client)
OPNsense Site B	Let's Encrypt	`sb-fw-01.core.aorxi.io`	Auto (os-acme-client)
Proxmox `sa-edge-01`	Let's Encrypt	`sa-edge-01.core.aorxi.io`	Auto (acme.sh cron)
Proxmox `sa-cmp-01`	Let's Encrypt	`sa-cmp-01.core.aorxi.io`	Auto (acme.sh cron)
Proxmox `sa-cmp-02`	Let's Encrypt	`sa-cmp-02.core.aorxi.io`	Auto (acme.sh cron)
Proxmox `sa-stor-01`	Let's Encrypt	`sa-stor-01.core.aorxi.io`	Auto (acme.sh cron)
Proxmox `sb-edge-01`	Let's Encrypt	`sb-edge-01.core.aorxi.io`	Auto (acme.sh cron)
Proxmox `sb-cmp-01` through `sb-cmp-05`	Let's Encrypt	`sb-cmp-0x.core.aorxi.io`	Auto (acme.sh cron)
PBS `sa-pbs-01`	Let's Encrypt	`sa-pbs-01.core.aorxi.io`	Auto (acme.sh cron)
K8s wildcard Site A	Let's Encrypt	`*.core.aorxi.io`	Auto (cert-manager)
K8s wildcard Site B	Let's Encrypt	`*.core.aorxi.io`	Auto (cert-manager)
IPMI `sa-edge-01`	step-ca	`ipmi-sa-edge-01.core.aorxi.io`	Manual/yearly
IPMI `sa-stor-01`	step-ca	`ipmi-sa-stor-01.core.aorxi.io`	Manual/yearly
IPMI `sb-edge-01`	step-ca	`ipmi-sb-edge-01.core.aorxi.io`	Manual/yearly
IPMI `sb-cmp-01` through `sb-cmp-05`	step-ca	`ipmi-sb-cmp-0x.core.aorxi.io`	Manual/yearly

Firewall Rules

ACME DNS-01 (all clients → Cloudflare)

DNS-01 challenge is outbound only — no inbound rules needed. All ACME clients call out to:

acme-v02.api.letsencrypt.org TCP 443
api.cloudflare.com TCP 443

OPNsense passes this by default on LAN/WAN. Verify the VLAN each acme.sh host sits on permits outbound HTTPS.

step-ca (admin VLANs → VLAN 30 only)

Proto	Source	Dest	Port	Purpose
TCP	VLANs 10 20 100	`10.10.30.30`	9000	Certificate issuance API
TCP	`10.20.30.0/24` (Site B DNS VMs)	`10.10.30.30`	9000	Cross-site cert issuance (over WireGuard)

Deployment Sequence

step-ca can be deployed any time after Phase 5 (DNS up). IPMI certs are a day-2 improvement — BMCs work with their self-signed certs in the interim.

Provision sa-ca-01 LXC on sa-stor-01, VLAN 30, 10.10.30.30/24
Install step-ca, initialize CA
Add sa-ca-01.core.aorxi.io → 10.10.30.30 A record in Technitium
Distribute root CA cert to admin workstations and Proxmox nodes
Issue and upload IPMI certs per host
Configure acme.sh on all Proxmox/PBS nodes; verify first issuance
Configure os-acme-client on both OPNsense instances; verify first issuance
Deploy cert-manager and ClusterIssuers after K8s clusters are up (Phase 18+)
Issue and deploy wildcard certs; verify ingress TLS

Operational Notes

Cloudflare token rotation

If the token is rotated, update it in all three places:

OPNsense ACME plugin (Services → ACME Client → Challenge Types)
Each node's ~/.acme.sh/account.conf
The cloudflare-api-token K8s Secret in both clusters

The Secret update triggers cert-manager to re-validate on the next renewal cycle. Force an immediate renewal if needed:

cmctl renew <cert-name> -n cert-manager

LE staging for testing

Test with LE staging before hitting production rate limits

Before running issuance against the production LE API (50 certs/domain/week), validate the DNS-01 flow against the staging server first. Staging certs are not browser-trusted but confirm the pipeline works without burning quota.

acme.sh: --server letsencrypt_test
cert-manager ClusterIssuer server: https://acme-staging-v02.api.letsencrypt.org/directory

step-ca backup

The private keys in /root/.step/secrets/ must be backed up. If sa-ca-01 is lost without a backup, all private-CA-issued certs become unverifiable and IPMI BMCs revert to self-signed on next reboot. Back up /root/.step/ to PBS-A, encrypted.

Cert expiry monitoring

Add Prometheus blackbox_exporter TLS probes against all HTTPS endpoints. Alert at 30 days remaining. The Grafana cert-manager dashboard (available at grafana.com) covers K8s certs natively.

DNS VMs — Technitium DNS serving core.aorxi.io A records
IPMI / KVM — accessing IPMI interfaces where private CA certs are installed
Build Phases — ordered sequence including CA and cert deployment steps
IP Tables — sa-ca-01 at 10.10.30.30, VLAN 30

CA & Certificate Infrastructure

On this page