Files
Homelabv4/apps/gitlab/manifests/praefect-ha-configmap.yaml
T
Scooby HuskyandClaude Sonnet 5 bce3861083 GitLab cross-site replication Phase 2b: enable Gitaly + workhorse TLS
Per the approved plan (Istio federation scoped separately, current
NodePort/PeerAuth/cert-manager pattern continues for these 2 remaining
links). Real Let's Encrypt certs (not self-signed) via the same DNS-01
ClusterIssuer used everywhere else - both sites already trust Let's
Encrypt's public CA, no custom CA distribution needed.

- global.gitaly.tls.enabled: adds tls_listen_addr (8076) alongside the
  existing plaintext listener (8075, stays for in-cluster traffic
  that doesn't need it). Requires
  global.praefect.virtualStorages[0].tlsSecretName (chart's own NOTES.txt
  validation rejects the render without it, confirmed live).
- global.workhorse.tls.enabled + global.workhorse.host: Gitaly's [gitlab]
  url (internal-API callback for gitlab-shell hooks) switches from an
  unpublishable *.svc.cluster.local name to a real floating hostname
  (gitlab-internal-api.ha.huskypup.net) that Let's Encrypt CAN certify -
  confirmed live this replaces the plaintext [[listeners]] block with
  [listeners.tls] on the SAME port (8181), not a second port like Gitaly.
- praefect-ha-configmap.yaml (the manually-patched live config) updated
  to tls:// + floating hostnames for all 3 home nodes, so home's own
  Praefect uses the identical addresses it'll hand the VPS as peer
  addresses once Phase 2b's 4th node lands.
- New CoreDNS rewrite for gitlab-internal-api.ha.huskypup.net (same
  local-rewrite pattern as pg-*/gitaly-* already have).

Sequencing note: applying live - Certificates must issue before the
TLS-enabling values land, or Gitaly/webservice pods fail to start
(missing secret for their init container's cert copy).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-22 14:24:13 -05:00

89 lines
3.9 KiB
YAML

# GitLab cross-site replication (see
# /home/scooby/.claude/plans/jiggly-snacking-iverson.md) - the chart has
# no support for registering an externally-hosted Gitaly node into an
# existing Praefect virtual storage (confirmed live: no
# virtualStorages[].nodes/address-override key exists in the template,
# gitalyReplicas just counts StatefulSet ordinals). Overriding Praefect's
# rendered config.toml is the only way to add one.
#
# A values-only CONFIG_TEMPLATE_DIRECTORY redirect (extraEnv +
# extraVolumes/extraVolumeMounts) was tried first and confirmed NOT to
# work: charts/gitlab/charts/praefect/templates/statefulset.yaml's own
# volumes: list never calls the gitlab.extraVolumes helper (only
# volumeMounts does) - a chart limitation specific to this subchart, no
# values-only fix exists.
#
# THIS ConfigMap (praefect-ha-config) is deployed as a harmless,
# otherwise-unused object - it exists purely as a git-tracked reference
# copy of the content that actually matters. The REAL live config lives
# on the chart's own gitlab-praefect ConfigMap, which
# argocd-apps/apps/gitlab.yaml's ignoreDifferences now exempts from
# ArgoCD's normal drift-reconciliation (its `data` field specifically) -
# apply this file's content to it directly:
# kubectl -n gitlab patch configmap gitlab-praefect --type merge \
# -p "{\"data\":{\"config.toml.tpl\":\"$(kubectl -n gitlab get cm \
# praefect-ha-config -o jsonpath='{.data.config\.toml\.tpl}' | \
# python3 -c 'import sys,json; print(json.dumps(sys.stdin.read())[1:-1])')\"}}"
# (or simpler: kubectl -n gitlab get cm praefect-ha-config -o
# jsonpath='{.data}' | kubectl -n gitlab patch cm gitlab-praefect --type
# merge -p "{\"data\":$(cat -)}")
#
# Content below has since moved past byte-identical: nodes 0-2 now use
# tls:// on port 8076 (matching global.gitaly.tls.enabled in
# values.yaml) and the floating hostnames gitaly-{0,1,2}.ha.huskypup.net
# (CoreDNS-rewritten locally for home's own traffic, same reasoning as
# pg-gitlab.ha.huskypup.net) rather than in-cluster DNS names or the
# plaintext scheme - this is what makes the SAME addresses usable once
# the VPS joins as a 4th node, without home's own Praefect needing a
# different config than what it hands the VPS as peer addresses. Praefect
# trusts the cert via the pod's normal system CA bundle - it's a real
# Let's Encrypt cert (gitaly-tls-certificate.yaml), not self-signed, so
# no extra CA trust config is needed here.
apiVersion: v1
kind: ConfigMap
metadata:
name: praefect-ha-config
namespace: gitlab
data:
config.toml.tpl: |
# TCP address to listen on
listen_addr = '0.0.0.0:8075'
prometheus_listen_addr = '0.0.0.0:9236'
prometheus_exclude_database_from_default_metrics = true
[failover]
enabled = true
read_only_after_failover = true
[auth]
token = {% file.Read "/etc/gitlab-secrets/praefect/praefect_token" | strings.TrimSpace | data.ToJSON %}
transitioning = false
[logging]
[[virtual_storage]]
name = 'default'
[[virtual_storage.node]]
storage = 'gitlab-gitaly-default-0'
address = 'tls://gitaly-0.ha.huskypup.net:8076'
token = {% file.Read "/etc/gitlab-secrets/praefect/gitaly_token" | strings.TrimSpace | data.ToJSON %}
[[virtual_storage.node]]
storage = 'gitlab-gitaly-default-1'
address = 'tls://gitaly-1.ha.huskypup.net:8076'
token = {% file.Read "/etc/gitlab-secrets/praefect/gitaly_token" | strings.TrimSpace | data.ToJSON %}
[[virtual_storage.node]]
storage = 'gitlab-gitaly-default-2'
address = 'tls://gitaly-2.ha.huskypup.net:8076'
token = {% file.Read "/etc/gitlab-secrets/praefect/gitaly_token" | strings.TrimSpace | data.ToJSON %}
[database]
host = 'pg-praefect-rw.gitlab.svc.cluster.local'
port = 5432
user = 'app'
password = {% file.Read "/etc/gitlab-secrets/praefect/db_password" | strings.TrimSpace | data.ToJSON %}
dbname = 'praefect_production'
sslmode = 'disable'