adaptive_export 0.16.4-rc1: kssync replaces the ks-ch-sync CronJob

[ConstanzeTU]

Rolls adaptive-export to 0.16.4-rc1, which mirrors kubescape state into ClickHouse from AE instead of soc's ks-ch-sync CronJob.

Why

ks-ch-sync is a python:3.11-slim pod that every 2 minutes LISTs every ContainerProfile, GETs each one individually, and INSERTs all of them unconditionally. Measured on edge4: 234 profiles against 624,447 stored rows — about 2,669 copies of each profile, holding 271 distinct contents. Every row carries the container's whole learned behaviour as text, which is why kubescape_profiles reached 501 MB, and why one dashboard panel reading it exhausted the ClickHouse server's memory budget and killed the container (exit 137).

The amplification was never the language. It was a timer with no change detection.

What changes

• Writes a row only when the content differs from the last row written.
• GETs an object only when its resourceVersion has moved — removing ~235 API calls per run as well.
• Not an informer: the storage apiserver truncates spec on a LIST (execs/opens/egress come back empty), so an informer's cache would hold truncated objects and mirror empty profiles. LIST for metadata, GET for content.
• Leader-only, using AE's existing cluster-setup leader. Every pod running it would store one row per node per change. The leader stamps its own node as hostname, which the PEM requires (WHERE hostname = gethostname()), and logs it.

Contents

• k8s/vizier/adaptive_export/kustomization.yaml: newTag 0.16.3-rc1 → 0.16.4-rc1
• k8s/vizier/bootstrap/adaptive_export_role.yaml: read-only grants kssync needs — containerprofiles and rogueartifacts (get/list, cluster-scoped because the CRDs are), and configmaps restricted by resourceNames to node-agent-bundle-policy rather than a blanket grant.

Deploy order matters

ADAPTIVE_KSSYNC_DISABLE=true or remove the ks-ch-sync CronJob before or with this roll. Both writing at once doubles every changed row.

Verifying

The log line reports listed, fetched and written separately. On a steady cluster the correct reading is non-zero listed, zero written — that is the fix working, and it is indistinguishable from a broken sync unless both numbers are read.

Not in this PR

No table engine change. event_time leading the sorting key is what makes every re-write a distinct row; collapsing it would destroy the profile history, which is currently the only record of how a profile evolved. Separate decision, separate image.

🤖 Generated with Claude Code

https://claude.ai/code/session_014GDT6HWiFmRxmUaGFSKjPY