Files
Homelabv4/infrastructure/authentik/manifests/ha-postgres-app-pushsecret.yaml
T
Scooby HuskyandClaude Sonnet 5 cbf7c07711 Vault -> VPS secret pipeline: replaces manual kubectl/ssh secret copies
User correctly flagged this: manually copying secrets between home and
the VPS defeats the entire point of having Vault. Every VPS secret this
session (MinIO creds, Cloudflare token, CNPG certs, the pg-authentik-app
password) was a one-off kubectl create secret copy-paste, because the
VPS's k3s cluster had no Vault/ESO pipeline at all - this builds one.

- infrastructure/authentik/manifests/ha-postgres-app-pushsecret.yaml:
  home pushes pg-authentik-app's password into Vault at
  secret/vps/pg-authentik-app (PushSecret, not just Get - this password is
  CNPG-generated, not Vault-native, so it has to originate from a push).

- infrastructure/vps-eso/manifests/clustersecretstore.yaml: ESO on the
  VPS (installed via helm, out-of-band like k3s/cert-manager - see the
  file's own header) authenticates to home's Vault via AppRole (not
  Kubernetes auth - the VPS is a separate cluster with no federation to
  home's API server). Reachable via a new public
  https://vault.kube.huskypup.net record - Vault was deliberately kept
  off the public internet before this, explicitly confirmed with the user
  before opening it. Traffic goes through the same Istio ingress gateway
  already serving other public hosts, so unlike the ha-authentik-postgres
  NodePort case, no PeerAuthentication/AuthorizationPolicy change was
  needed - it arrives as a normal in-mesh call from the gateway's own
  identity, not raw external TCP to a pod.

  Found and fixed a real Vault gotcha while wiring this up: tried to
  remove the AppRole's token_bound_cidrs restriction (added first, before
  discovering Vault can't see the VPS's real source IP through the
  gateway - it only ever sees the gateway's own pod IP) by omitting the
  field from a follow-up  - that does NOT clear it, the
  AppRole role endpoint preserves omitted fields rather than resetting
  them to default. Had to explicitly write token_bound_cidrs=. Spent a
  while chasing a misleading 403 permission denied on auth/token/
  lookup-self before finding this - vault token capabilities said read
  was allowed (policy was fine), the actual rejection was IP-bound token
  use from an unbound context.

- infrastructure/vps-standby/authentik/manifests/
  pg-authentik-app-externalsecret.yaml: pulls it back down, Merge policy
  (only overwrites the password key - host/dbname/username stay local,
  CNPG still needs its own local -rw hostname for internal use).

- argocd-apps/vps-standby/vps-eso.yaml: new Application for the
  ClusterSecretStore.

Verified end-to-end: password now matches between home and VPS's
pg-authentik-app secrets via this pipeline (not the earlier manual patch),
confirmed by comparing both live secret values after ESO's sync.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-20 18:26:26 -05:00

37 lines
1.5 KiB
YAML

# Multi-site active failover pilot (see
# /home/scooby/.claude/plans/jiggly-snacking-iverson.md) - closes the gap
# the user correctly flagged: manually kubectl/ssh-copying secrets between
# home and the VPS defeats the entire point of having Vault. This is the
# first piece of a real Vault -> VPS pipeline (infrastructure/vps-eso/ has
# the other half, ESO running on the VPS itself pulling this back down).
#
# pg-authentik-app is CNPG-generated, not Vault-native - each cluster
# (home and VPS) independently generates its own random password for the
# `app` role at bootstrap time. Since VPS's Postgres now REPLICATES from
# home (WAL includes role/password changes), the two clusters' actual live
# passwords are only in sync because of that replication - but the two
# clusters' K8S SECRET OBJECTS never resync on their own (confirmed live
# 2026-08-20: VPS's copy was stale). Pushing home's value into Vault, with
# VPS's ESO pulling it back down on refreshInterval, makes this self-
# healing instead of a manual one-time fix that goes stale again on the
# next password rotation.
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
name: pg-authentik-app-to-vps
namespace: authentik
spec:
refreshInterval: 5m
secretStoreRefs:
- name: vault-backend
kind: ClusterSecretStore
selector:
secret:
name: pg-authentik-app
data:
- match:
secretKey: password
remoteRef:
remoteKey: vps/pg-authentik-app
property: password