Skip to content

feat(aisix)!: drop default resources, document workers sizing the token-estimate pool, refuse utilization autoscaling without requests - #429

Draft
jarvis9443 wants to merge 5 commits into
mainfrom
docs/aisix-workers-estimate-pool-1968
Draft

jarvis9443 wants to merge 5 commits into
mainfrom
docs/aisix-workers-estimate-pool-1968

Conversation

@jarvis9443

@jarvis9443 jarvis9443 commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

This drops the gateway chart's default resources and makes a utilization HPA or KEDA cpu/memory trigger refuse to render without the request it measures against. Breaking for anyone who relied on the old defaults.

From api7/aisix#1437 the gateway sizes its token-estimate pool by proxy.workers, one thread per worker, and each thread holds one tokenizer instance per encoding it has counted with (about 17 MiB cl100k, 27 MiB o200k, 39 MiB o200k_harmony resident). With config.proxy.workers unset the gateway uses every CPU the process sees, which without a CPU limit is the whole node, so the old fixed limits.memory: 1Gi could OOM a pod on a large node. No single CPU or memory figure fits every deployment, so the chart stops guessing:

  • resources defaults to {} (previously requests: {cpu: 500m, memory: 256Mi}, limits: {memory: 1Gi}). The pod renders with no resources block unless you set one. The values comment explains why and what to size.
  • config.proxy.workers stays declared with no default (null); its comment now says it sizes both the proxy workers and the token-estimate pool, and to set it to the CPUs allocated to the pod.
  • With autoscaling.enabled, a targetCPUUtilizationPercentage without resources.requests.cpu, or a targetMemoryUtilizationPercentage without resources.requests.memory, now fails the render (... requires resources.requests.cpu: utilization is measured against the request, so without one the HorizontalPodAutoscaler never scales ...). Before, such an HPA deployed and silently never scaled. Since targetCPUUtilizationPercentage defaults to 70, autoscaling.enabled=true alone now needs resources.requests.cpu.
  • The same applies to KEDA: with keda.enabled, a cpu or memory trigger whose metricType is Utilization (KEDA's default when metricType is omitted; the deprecated metadata.type is honoured too) fails the render unless the matching resources.requests entry is set. metricType: AverageValue and non-resource triggers such as prometheus are unaffected.
  • The README's "Autoscale on CPU" example sets requests and workers, and aisix-render-checks.sh covers the empty default, the HPA and KEDA refusals, and the cases that still render. The render-succeeds checks in that script now print their result; a >/dev/null on the check line was swallowing it.

Upgrading: a release that took the chart's resource defaults gets none after helm upgrade (the pod becomes BestEffort) and, if it autoscales on a CPU or memory utilization target (HPA or KEDA), fails to render until the matching requests are set. Set resources (at least requests) and config.proxy.workers yourself before upgrading. --reuse-values keeps whatever the old release had.

Held for the release that ships api7/aisix#1437: the workers comment describes behaviour older gateways do not have. charts/aisix-cp is untouched.

🤖 Generated with Claude Code

@jarvis9443 jarvis9443 added the aisix-hold-for-release Gateway-key chart change: merges with the release that ships the key label Nov 15, 2026
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Pre-merge checks: Passed 1 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
E2e Test Quality Review Warning Summary: The PR's changed behavior has no E2E test. The only new tests are Helm render checks in .github/scripts/aisix-render-checks.sh. Criterion 1 (E2E Completeness, Blocking) is not met. Unit… Add E2E coverage that runs on the kind CI cluster: 1. Install charts/aisix with autoscaling.enabled=true and a resources.requests block. Assert that the HPA reports a current utilization value and that the pod has the expected `reques…
Passed Passed checks: 1 passed

Check name Status Explanation
Security Check Passed Summary: No Security Check failure condition applies to this pull request. The PR changes the charts/aisix Helm chart. It drops the default resources, adds render-time refusals for utilization…
Full details: Security Check

Explanation

Summary: No Security Check failure condition applies to this pull request. The PR changes the charts/aisix Helm chart. It drops the default resources, adds render-time refusals for utilization-based HPA and KEDA triggers without matching requests, updates the config.proxy.workers comment and README, and extends .github/scripts/aisix-render-checks.sh. Evidence gathered: - The repository checkout has one commit (b11edaa), so a PR base diff could not be isolated from history. I relied on the PR description and the current contents of the changed files. - charts/aisix/templates/hpa.yaml and charts/aisix/templates/scaledobject.yaml only map values to HPA and ScaledObject fields. Neither touches secrets, logging, authorization, ownership, TLS, or secret references. - .github/scripts/aisix-render-checks.sh runs helm template assertions. Its only credential-like values are placeholder fixtures such as k1, k2, and user. The script prints check results and does not log real secrets. - The PR does not add or change any of the Go gateway code paths the check targets. Those paths cover logging and redaction, DB encryption, endpoint authorization, cross-resource ownership, TLS flags, shared-resource safety, and $env:// or $secret:// resolution. Category review: 1. Sensitive data exposure: No issues found in the changed files. 2. Unencrypted secrets in DB: Not applicable. No DB model or migration is changed. 3. Authorization bypass: Not applicable. No endpoint or handler is changed. 4. Cross-resource access: Not applicable. 5. TLS/crypto configuration: Not applicable. 6. Shared resource safety: Not applicable. 7. Secret reference resolution: Not applicable. The PR does not change secret reference handling. Caveat: The charts/aisix-cp chart and the pre-existing admin-key and configSecrets handling are outside this PR, as the PR description states. I did not review them for this check.

Full details: E2e Test Quality Review

Explanation

Summary: The PR's changed behavior has no E2E test. The only new tests are Helm render checks in .github/scripts/aisix-render-checks.sh. Criterion 1 (E2E Completeness, Blocking) is not met. Unit-level tests are not sufficient under this check. Evidence - The repository has one commit (git rev-list --count HEAD = 1). The base is unavailable, so the full PR patch could not be isolated. I judged from the current tree and the PR description. The findings below apply to the current state of the files. - .github/scripts/aisix-render-checks.sh runs helm template and parses the output with Python. The checks for this PR's behavior are at lines 111-132: - Line 111-112: the default render has no resources. - Lines 113-126: CPU and memory HPA refusals, and KEDA Utilization refusals (including the omitted metricType case). - Lines 117-118, 127-132: the passing cases (HPA, KEDA AverageValue, KEDA prometheus). - None of these checks deploys a pod or HPA, or observes scheduling, QoS class, or scaling. - The refusals are fail calls in charts/aisix/templates/_helpers.tpl (lines 441, 444, 457). They run at template time only. - .github/workflows/ci.yaml runs ct lint, the render script, and aisix-config-boot.sh. It also runs kind-based helm install steps for standalone, listeners, and admin (lines 127-221). These installs use standalone-values.yaml. They are real deployments, but none of them asserts the changed behavior. No CI value file sets autoscaling, keda, or resources (rg found only standalone-values.yaml:20 and a comment). The PR does not add any assertion on QoS class, HPA status, or the refusal at install time. - aisix-config-boot.sh checks config loading only. It does not test resources, workers, or autoscaling. Per-criterion 1. E2E Completeness (Blocking): not met. The flow from values to a running pod with requests, or to a working HPA or KEDA scaler, is not tested. The tests stop at rendered YAML. 2. Scenario Coverage (Major): partly met. Covered: empty default, both HPA refusals, the KEDA refusals, the passing cases. Not covered: targetCPUUtilizationPercentage=0, resources.requests with only memory set and cpu missing, and config.proxy.workers left at null. 3. Readability: New check names are clear. Older checks use long inline Python eval strings. These predate this PR. 4. Test Relationships: Each check renders independently. No hidden order found. 5. Assertion Readability: The new assertions are simple. 6. Real Services vs Mocking: Not applicable to render tests. 7. Error Handling: The refuses helper checks the non-zero exit and the message. Adequate for render-time refusals. 8. PR Scope: Not fully assessable without the base. 9. Concurrency Safety: Not applicable to chart templates and values. Prior failure disposition: The prior FAIL is not addressed. The current tree has the same render-only test layer and no new E2E test. The prior claims hold on the current tree. The prior run's statement that the fail guards are not in hpa.yaml is also confirmed: they are in _helpers.tpl.

Resolution

Add E2E coverage that runs on the kind CI cluster: 1. Install charts/aisix with autoscaling.enabled=true and a resources.requests block. Assert that the HPA reports a current utilization value and that the pod has the expected requests. 2. Install with default values. Assert that the pod QoS class is BestEffort. This tests the upgrade impact the PR description states. 3. Run the refusal cases (for example, autoscaling.enabled=true with no resources.requests.cpu) through helm install. Assert a non-zero exit and the expected message. 4. Add boundary cases to .github/scripts/aisix-render-checks.sh: targetCPUUtilizationPercentage=0, resources.requests with only memory set, and config.proxy.workers left at null. Then re-run the check.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

…hout the matching request

The gateway sizes its token-estimate pool by config.proxy.workers, one thread
per worker each holding its own tokenizer (about 17/27/39 MiB resident for
cl100k/o200k/o200k_harmony). With workers unset it follows every CPU the pod
sees, so the chart's fixed 1Gi memory limit could OOM a pod on a large node.
No single CPU or memory figure fits every deployment, so resources now
defaults to {} and the operator sets requests and config.proxy.workers.

An HPA with a CPU or memory utilization target needs the matching request;
the render now fails instead of deploying an HPA that never scales.
@jarvis9443 jarvis9443 changed the title docs(aisix): say config.proxy.workers also sizes the token-estimate pool feat(aisix)!: drop default resources, document workers sizing the token-estimate pool, refuse a utilization HPA without requests Nov 19, 2026
… matching request

KEDA measures a cpu or memory trigger with metricType Utilization (its default
when metricType is omitted) against the container's request, exactly as the
chart's own HPA does, so without one the HPA KEDA creates never scales.

Also print the result of the render-succeeds checks, which their >/dev/null
redirect was silencing.
@jarvis9443 jarvis9443 changed the title feat(aisix)!: drop default resources, document workers sizing the token-estimate pool, refuse a utilization HPA without requests feat(aisix)!: drop default resources, document workers sizing the token-estimate pool, refuse utilization autoscaling without requests Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aisix-hold-for-release Gateway-key chart change: merges with the release that ships the key

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant