mirror of
https://gitlab.kube.huskypup.net/Scooby/Homelabv4.git
synced 2026-08-21 11:36:50 +00:00
Scooby Husky
983671ef50
Root cause (confirmed via ztunnel logs): the argo-cd chart's default
NetworkPolicies only allow each component's application port (e.g. 6379
for redis), but Istio ambient mode routes ALL pod-to-pod traffic through
ztunnel's HBONE tunnel on port 15008 first - so the tunnel itself was
being blocked even though the "real" port was allowed. Every inter-pod
connection in the argocd namespace hung for exactly 10s then reset;
ztunnel's own log named it directly ("connection timed out, maybe a
NetworkPolicy is blocking HBONE port 15008"). This broke argocd-server's
Redis-backed session/cluster-info caching cluster-wide and was silently
preventing the root Application from picking up new child Applications.
Fix: additive NetworkPolicy allowing ingress on 15008 for all argocd
pods (NetworkPolicies union across multiple policies selecting a pod,
so this doesn't touch/replace the chart's rendered ones - safe across
Helm upgrades).
Also: dropped argocd from the namespace-enrollment job's waypoint list.
argocd's only AuthorizationPolicy (allow-argocd-access) is a plain
source-namespace/IP match with no L7 rules - its own status shows
"attached to ztunnel", not waypoint - so forcing L7 waypoint processing
onto the namespace was unnecessary overhead, not a security requirement.
ztunnel's mTLS still fully covers it. (This was a red herring for the
HBONE bug itself, not the fix, but a valid simplification found along
the way.)
67 lines
3.9 KiB
YAML
67 lines
3.9 KiB
YAML
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: istio-mesh-app-enrollment
|
|
namespace: istio-system
|
|
annotations:
|
|
argocd.argoproj.io/hook: PostSync
|
|
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
|
|
argocd.argoproj.io/sync-wave: "1"
|
|
spec:
|
|
backoffLimit: 3
|
|
ttlSecondsAfterFinished: 300
|
|
template:
|
|
spec:
|
|
serviceAccountName: argocd-hook-sa
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: enroll
|
|
image: alpine/k8s:1.32.13
|
|
command:
|
|
- /bin/bash
|
|
- -c
|
|
- |
|
|
set -euo pipefail
|
|
echo "=== Enrolling app namespaces in Istio ambient mesh ==="
|
|
|
|
# Enroll app namespaces in ambient mesh
|
|
APP_NAMESPACES=(argocd gitlab n8n nextcloud teslamate home-assistant frigate guacamole cattle-system)
|
|
for ns in "${APP_NAMESPACES[@]}"; do
|
|
echo "Enrolling $ns in ambient mesh..."
|
|
kubectl label namespace "$ns" istio.io/dataplane-mode=ambient --overwrite 2>/dev/null || true
|
|
kubectl label namespace "$ns" istio-injection- 2>/dev/null || true
|
|
done
|
|
|
|
# Attach waypoint proxies for L7 policy enforcement.
|
|
# argocd excluded: its only AuthorizationPolicy (allow-argocd-access)
|
|
# is a plain source-namespace/IP match with no L7 rules - it's
|
|
# enforced directly by ztunnel (see its status: "attached to
|
|
# ztunnel", not waypoint). Forcing waypoint L7 processing onto the
|
|
# namespace anyway broke argocd-redis: waypoint doesn't handle
|
|
# Redis's long-lived RESP protocol well, causing ~10s hang-then-
|
|
# close on every connection (i/o timeout errors in argocd-server,
|
|
# cluster info/session caching failing, new Applications from git
|
|
# never getting picked up). ztunnel's plain mTLS still fully
|
|
# covers argocd's actual security requirement here.
|
|
echo "Attaching waypoint proxies to app namespaces..."
|
|
WAYPOINT_APP_NAMESPACES=(gitlab n8n nextcloud teslamate home-assistant frigate guacamole)
|
|
for ns in "${WAYPOINT_APP_NAMESPACES[@]}"; do
|
|
kubectl label namespace "$ns" istio.io/use-waypoint=waypoint --overwrite 2>/dev/null || true
|
|
done
|
|
kubectl label namespace argocd istio.io/use-waypoint- 2>/dev/null || true
|
|
|
|
# Annotate services for NetBird operator auto-discovery (netbird.io/expose triggers the operator)
|
|
echo "Annotating services for NetBird operator exposure..."
|
|
kubectl annotate svc -n gitlab gitlab-webservice-default netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n grafana grafana netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n home-assistant home-assistant netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n guacamole guacamole netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n nextcloud nextcloud netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n argocd argocd-server netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n frigate frigate netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n teslamate teslamate netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n home-assistant esphome netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n prometheus kube-prometheus-stack-prometheus netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
kubectl annotate svc -n vault vault netbird.io/expose="true" --overwrite 2>/dev/null || true
|
|
echo "App namespace enrollment and NetBird annotations complete"
|