mirror of
https://gitlab.kube.huskypup.net/Scooby/Homelabv4.git
synced 2026-08-21 11:36:50 +00:00
Scooby HuskyandClaude Sonnet 5
018eb3570f
Root cause of tonight's earlier CrowdSec/CNPG-backup workarounds: no node in the cluster had any route into the Netbird mesh CIDR (100.108.0.0/16) for pod-originated traffic. The per-namespace netbird 'router' pods are inbound-only infrastructure (external peers reaching K8s services); their own architecture has no reverse path. Fix, in two parts: 1. infrastructure/netbird/manifests/egress-daemonset.yaml - one netbird client per node, hostNetwork so its wt0 interface lives in the node's real network namespace, plus a sidecar that adds a host route sending 100.108.0.0/16 out via it. hostNetwork requires a scoped Kyverno PolicyException (infrastructure/kyverno/policies/netbird-egress-exception.yaml) to the disallow-host-namespaces STIG policy - narrowly for this one DaemonSet by name, not a namespace-wide exclusion. 2. Discovered the route alone wasn't enough for k3s NodePort traffic (vps-minio:30900): Netbird manages its own nftables ACLs independent of iptables/Kyverno, and its forward chain (netbird-rt-fwd) only permits *established* connections through a peer acting as a router - never new ones, by design, unless a Netbird 'Network Route' policy is explicitly configured (it isn't, for this VPS). Locally-terminated connections (tinyproxy) go through a separate, already-permissive ACL chain, which is why the CrowdSec proxy fix from earlier tonight worked. Replicated that working pattern for MinIO: minio-forward.service on the VPS host (systemd, socat) forwards 100.108.113.41:9000 -> MinIO's ClusterIP, avoiding the NodePort path entirely. Re-enabled everything that was disabled/suspended earlier tonight because of this gap, pointed at the new endpoint: - CrowdSec CAPI/console-enroll (removed DISABLE_ONLINE_API, restored the VPS proxy env vars) - n8n/nextcloud/authentik CNPG backup.barmanObjectStore - vault-raft-snapshot CronJob (unsuspended) - nextcloud PVC content sync CronJob endpoint vps-minio.netbird.internal is retired everywhere - it was never actually resolvable (Netbird has no DNS configured) even before today's routing fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
39 lines
1.5 KiB
YAML
39 lines
1.5 KiB
YAML
# Scoped exception to the disallow-host-namespaces STIG policy (V-242386),
|
|
# narrowly for the netbird-egress DaemonSet only - not a namespace-wide
|
|
# exclusion like crowdsec's (see disallow-host-namespaces ClusterPolicy).
|
|
#
|
|
# Why hostNetwork is genuinely needed here: no node in this cluster has any
|
|
# route into the Netbird mesh CIDR (100.108.0.0/16) for pod-originated
|
|
# egress traffic - confirmed 2026-08-17 while debugging CrowdSec/CNPG
|
|
# backups reaching the VPS. The existing per-namespace "router" pods
|
|
# (NBRoutingPeer-based) are inbound-only and run fine in ordinary pod
|
|
# network namespaces. An EGRESS gateway is architecturally different: its
|
|
# wt0 interface must live in each node's real network namespace so the
|
|
# node's own kernel routing table can send matched pod traffic out through
|
|
# it directly - a pod-netns router would need SNAT/forwarding indirection
|
|
# on top, and per-pod route injection isn't possible without hostPID
|
|
# (also STIG-blocked) or a Cilium egress-gateway feature that doesn't
|
|
# actually solve this (it selects source IP for already-routable traffic,
|
|
# it doesn't create the missing route).
|
|
apiVersion: kyverno.io/v2
|
|
kind: PolicyException
|
|
metadata:
|
|
name: netbird-egress-exemption
|
|
namespace: kyverno
|
|
spec:
|
|
exceptions:
|
|
- policyName: disallow-host-namespaces
|
|
ruleNames:
|
|
- host-namespaces
|
|
- autogen-host-namespaces
|
|
match:
|
|
any:
|
|
- resources:
|
|
kinds:
|
|
- DaemonSet
|
|
- Pod
|
|
namespaces:
|
|
- netbird
|
|
names:
|
|
- netbird-egress*
|