GitLab cross-site replication Phase 2a: Praefect config override mechanism

Proves the CONFIG_TEMPLATE_DIRECTORY redirect works before Phase 2b
adds any actual cross-site dependency. Content is byte-identical to
the chart's current rendering (confirmed live) - this commit only
tests the override plumbing itself: a new ConfigMap
(praefect-ha-configmap.yaml) mounted at a different path than the
chart's own /etc/gitaly/templates (can't reuse that path/volume name -
Kubernetes rejects duplicates), with CONFIG_TEMPLATE_DIRECTORY
overridden via extraEnv to point at it instead (last-wins env
semantics, confirmed Kubernetes-documented behavior).

Also removes gitlab.praefect.virtualStorages - confirmed live dead
config, global.praefect.virtualStorages is what the chart's template
actually reads.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

apps/gitlab/manifests/praefect-ha-configmap.yaml

@@ -0,0 +1,73 @@
+# GitLab cross-site replication Phase 2a (see
+# /home/scooby/.claude/plans/jiggly-snacking-iverson.md) - the chart has
+# no support for registering an externally-hosted Gitaly node into an
+# existing Praefect virtual storage (confirmed live: no
+# virtualStorages[].nodes/address-override key exists in the template,
+# gitalyReplicas just counts StatefulSet ordinals). Overriding Praefect's
+# rendered config.toml is the only way to add one.
+#
+# The chart's own gitlab-praefect ConfigMap (auto-rendered from
+# global.praefect.virtualStorages) is mounted at /etc/gitaly/templates,
+# which CONFIG_TEMPLATE_DIRECTORY already points at - can't just add data
+# to that SAME ConfigMap (ArgoCD/Helm fully owns and would revert it),
+# and can't mount a second volume at the SAME path/name (Kubernetes
+# rejects duplicate volume names). So this ConfigMap mounts at a
+# DIFFERENT path (values.yaml's gitlab.praefect.extraVolumes), and
+# CONFIG_TEMPLATE_DIRECTORY gets overridden via extraEnv to point at it
+# instead - env var duplicate-key "last wins" IS legitimate documented
+# Kubernetes behavior for a container's env: list, unlike volumes.
+#
+# Phase 2a content is intentionally byte-identical to the chart's own
+# current rendering (confirmed live via `kubectl -n gitlab get cm
+# gitlab-praefect -o jsonpath='{.data.config\.toml\.tpl}'`) - this
+# commit only proves the override mechanism itself doesn't break
+# anything, before Phase 2b switches addressing to floating hostnames
+# and adds the VPS as a 4th node (no reason to change addressing scheme
+# before there's an actual cross-site node that needs it).
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: praefect-ha-config
+  namespace: gitlab
+data:
+  config.toml.tpl: |
+    # TCP address to listen on
+    listen_addr = '0.0.0.0:8075'
+    prometheus_listen_addr = '0.0.0.0:9236'
+
+    prometheus_exclude_database_from_default_metrics = true
+
+    [failover]
+    enabled = true
+    read_only_after_failover = true
+
+    [auth]
+    token = {% file.Read "/etc/gitlab-secrets/praefect/praefect_token" | strings.TrimSpace | data.ToJSON %}
+
+    transitioning = false
+
+    [logging]
+
+    [[virtual_storage]]
+    name = 'default'
+    [[virtual_storage.node]]
+    storage = 'gitlab-gitaly-default-0'
+    address = 'tcp://gitlab-gitaly-default-0.gitlab-gitaly-default.gitlab.svc:8075'
+    token = {% file.Read "/etc/gitlab-secrets/praefect/gitaly_token" | strings.TrimSpace | data.ToJSON %}
+    [[virtual_storage.node]]
+    storage = 'gitlab-gitaly-default-1'
+    address = 'tcp://gitlab-gitaly-default-1.gitlab-gitaly-default.gitlab.svc:8075'
+    token = {% file.Read "/etc/gitlab-secrets/praefect/gitaly_token" | strings.TrimSpace | data.ToJSON %}
+    [[virtual_storage.node]]
+    storage = 'gitlab-gitaly-default-2'
+    address = 'tcp://gitlab-gitaly-default-2.gitlab-gitaly-default.gitlab.svc:8075'
+    token = {% file.Read "/etc/gitlab-secrets/praefect/gitaly_token" | strings.TrimSpace | data.ToJSON %}
+
+
+    [database]
+    host = 'pg-praefect-rw.gitlab.svc.cluster.local'
+    port = 5432
+    user = 'app'
+    password = {% file.Read "/etc/gitlab-secrets/praefect/db_password" | strings.TrimSpace | data.ToJSON %}
+    dbname = 'praefect_production'
+    sslmode = 'disable'

apps/gitlab/values.yaml

@@ -285,14 +285,14 @@ gitlab:
     enabled: true
     minReplicas: 1
     maxReplicas: 1
-    
+
     # Note: Praefect PostgreSQL config is in global.praefect.psql
-    
+
     # Use CNPG-generated database secret
     dbSecret:
       secret: pg-praefect-app
       key: password
-    
+
     # Resources
     resources:
       requests:
@@ -301,11 +301,37 @@ gitlab:
       limits:
         cpu: 250m
         memory: 512Mi
-    
-    # Virtual storage configuration
-    virtualStorages:
-    - name: default
-      gitalyReplicas: 1  # Homelab sizing
+
+    # (dead config removed 2026-08-22: this virtualStorages block never
+    # actually did anything - confirmed live the chart's Praefect
+    # ConfigMap template iterates global.praefect.virtualStorages, not
+    # this one, so gitalyReplicas: 3 up there always won regardless of
+    # what this said. See global.praefect.virtualStorages above for the
+    # real config.)
+
+    # GitLab cross-site replication Phase 2a (see
+    # /home/scooby/.claude/plans/jiggly-snacking-iverson.md) - redirects
+    # Praefect's config template source so a hand-written config.toml.tpl
+    # (praefect-ha-configmap.yaml) can add a VPS-hosted Gitaly node the
+    # chart itself has no mechanism to register. The chart's own
+    # gitlab-praefect ConfigMap is already mounted at
+    # /etc/gitaly/templates (which CONFIG_TEMPLATE_DIRECTORY points at by
+    # default) - can't add a second volume at that same path/name
+    # (Kubernetes rejects duplicate volume names), so this mounts the
+    # override at a different path and points CONFIG_TEMPLATE_DIRECTORY
+    # there instead. Duplicate env var names in a container's env: list
+    # resolve last-wins (documented Kubernetes behavior) - this entry
+    # renders after the chart's own, so it's the one that takes effect.
+    extraEnv:
+      CONFIG_TEMPLATE_DIRECTORY: /etc/gitaly/templates-ha
+    extraVolumes:
+      - name: praefect-ha-config
+        configMap:
+          name: praefect-ha-config
+    extraVolumeMounts:
+      - name: praefect-ha-config
+        mountPath: /etc/gitaly/templates-ha
+        readOnly: true
       maxUnavailable: 1
   
   # GitLab Exporter for Prometheus metrics