- infrastructure/vps-standby/gitlab/manifests/ha-postgres-app-externalsecret.yaml:
creationPolicy Merge -> Owner (already applied live, committing to
match). Merge assumed CNPG creates a base pg-gitlab-app/pg-praefect-app
secret to merge a password into - wrong for these VPS Clusters since
Phase 1c's replica-cluster recreate; CNPG doesn't create a local
owner secret for a replicated role. Left pg-gitlab-app/pg-praefect-app
missing entirely, crash-looping vps-gitlab-webservice on
ActiveRecord::DatabaseConnectionError for ~39h.
- infrastructure/vps-standby/nextcloud/manifests/pvc-restore-cronjob.yaml:
exclude config.php from the restic restore. The job's own comment
assumed nextcloud runs at replicas: 0 on the VPS; live confirms
replicas: 1 (no such key was ever actually set) - restic couldn't
overwrite the live pod's config.php (permission denied), failing the
whole job every run for 2+ days even though everything else restored
fine. Excluding it is correct regardless of permissions - a live
pod's own config.php shouldn't be overwritten by a background
restore job.
Also fixed live (not git-tracked, config.php is PVC-persisted runtime
state, not sourced from git):
- home + VPS nextcloud config.php dbpassword: out of sync with CNPG's
actual current pg-nextcloud-app password (baked in once at install,
never re-synced). VPS's case was two-layered - its own
pg-nextcloud-app secret also didn't match the real Postgres role
password, since VPS's pg-nextcloud is a read-only streaming replica
(spec.replica.enabled) and the authoritative password lives on home.
- grafana: broke a RollingUpdate deadlock (single-replica Deployment +
RWO Ceph volume - new pod couldn't start while the old pod still
held the only-one-node-at-a-time attachment, and the rollout
wouldn't scale down the old pod until the new one was ready).
- suspended the stale gitlab-mirror-sync CronJob on the VPS (spamming
auth failures every ~15min since its stored token predates the
Postgres-replication cutover of VPS GitLab's DB) rather than
deleting it, per the plan's own note to defer that until Phase 2b is
verified.
Adds externalClusters connectionParameters + replica.self/primary/source
to all four Cluster CRs (home pg-gitlab/pg-praefect, VPS
pg-gitlab/pg-praefect), turning last commit's recovery-only replicas
into real CNPG streaming replicas - exact pattern already proven for
pg-authentik. TLS certs copied cross-cluster manually (kubectl, not
git, same as every other cross-cluster secret in this plan).
Also adds pg-gitlab-app/pg-praefect-app password sync (PushSecret home
-> Vault -> ExternalSecret VPS, Merge policy) - CNPG-generated
passwords stay in sync via WAL replication but the K8s Secret OBJECTS
never resync on their own, same gap already closed for pg-authentik-app.
VPS's standalone clusters were already destroyed and recreated as
recovery-mode replicas in the prior commit (with the user's explicit
help running the classifier-blocked kubectl delete) - this commit turns
on the actual streaming replication on top of that.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Scooby Husky