Fix CNPG operator -> instance status communication + add CNPG health check

Two related fixes, found while working through why authentik/n8n/gitlab
ArgoCD Applications showed Unknown/stuck health despite pods being fine:

1. ArgoCD has no built-in health check for CNPG's Cluster CRD, so it
   always reported "Unknown" - and since app-level health rolls up to
   the worst resource status, every app with a CNPG Cluster showed
   Unknown/Progressing regardless of actual state. Added a Lua health
   check (resource.customizations.health.postgresql.cnpg.io_Cluster)
   reading .status.conditions[Ready] / .status.phase.

2. Once that started reporting real status instead of masking it,
   pg-authentik showed a genuine problem: CNPG's operator couldn't
   reach its Postgres instances' status API (port 8000) - "Cannot
   extract Pod status ... context deadline exceeded" - because:
   a) authentik-ingress's CiliumNetworkPolicy never allowed cnpg-system
      as a source (crowdsec-ingress already had this exception,
      authentik-ingress was just missing it - inconsistency, not
      deliberate).
   b) Even after fixing (a), still blocked - pg-authentik's pods are
      ambient-mesh-enrolled, so the connection actually goes through
      ztunnel's HBONE tunnel (port 15008) first, same underlying issue
      as the argocd-redis fix from earlier today. Rather than keep
      finding and patching this per-namespace as it recurs, added a
      cluster-wide CiliumClusterwideNetworkPolicy allowing HBONE
      broadly - ztunnel's own mTLS/SPIFFE identity verification is the
      real security boundary for mesh traffic; Cilium blocking the
      tunnel port itself was only breaking legitimate traffic, not
      adding meaningful protection on top.

Verified live: CNPG operator immediately went from failing status
extraction to successfully reconciling (recreating a pod to reattach
its PVC) once both fixes were in place.
This commit is contained in:
Scooby Husky
2026-08-17 16:41:11 -05:00
parent 99b8ca6be9
commit 7b3669e9a6
3 changed files with 80 additions and 0 deletions
@@ -0,0 +1,34 @@
# Allow ztunnel's HBONE tunnel port (15008) cluster-wide.
#
# Istio ambient mode routes ALL pod-to-pod traffic through ztunnel's mTLS
# HBONE tunnel on port 15008 first, regardless of which "real" port the
# traffic is ultimately for. Per-namespace CiliumNetworkPolicies that only
# allow the application port (e.g. 6379 for Redis, 8000 for CNPG's instance
# status API) block the HBONE tunnel itself even when the intended port is
# allowed - ztunnel's own encrypted connection never gets through.
#
# Hit this same bug twice in one session (2026-08-17): argocd-redis (fixed
# with a namespace-scoped NetworkPolicy) and cnpg-system -> pg-authentik's
# status port (fixed the same way first, still broken - needed this too).
# Rather than keep discovering and patching this per-namespace, allow HBONE
# broadly: ztunnel already enforces real security here via mTLS + SPIFFE
# identity verification (and any CiliumNetworkPolicy/AuthorizationPolicy
# ingress rules on the target still apply to the traffic's real intent) -
# Cilium blocking the tunnel port itself only breaks legitimate mesh traffic,
# it doesn't add a meaningful security boundary on top of what ztunnel
# already does.
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: allow-hbone
spec:
description: "Allows Istio ambient's ztunnel HBONE tunnel port cluster-wide - see comments"
enableDefaultDeny:
egress: false
ingress: false
endpointSelector: {}
ingress:
- toPorts:
- ports:
- port: "15008"
protocol: TCP