Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/scripts/aisix-render-checks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ check() {
if "$@"; then echo "ok $what"; else echo "FAIL $what"; fails=$((fails + 1)); fi
}

renders() { render "$@" >/dev/null; }

refuses() {
local want=$1
shift
Expand Down Expand Up @@ -106,5 +108,27 @@ check "a dotted key under config is refused" \
refuses "contains a dot" --set 'config.cache\.redis\.password=x'
check "configSecrets outside the allow-list is refused" \
refuses "not a credential-bearing setting" --set 'configSecrets.upstream\.timeout_ms.secretName=a' --set 'configSecrets.upstream\.timeout_ms.key=b'
check "the default render sets no resources" \
query '"resources" not in pod["containers"][0]'
check "a CPU-utilization HPA without resources.requests.cpu is refused" \
refuses "requires resources.requests.cpu" --set autoscaling.enabled=true
check "a memory-utilization HPA without resources.requests.memory is refused" \
refuses "requires resources.requests.memory" --set autoscaling.enabled=true --set autoscaling.targetCPUUtilizationPercentage=null --set autoscaling.targetMemoryUtilizationPercentage=80 --set resources.requests.cpu=1
check "an HPA renders once the matching requests are set" \
renders --set autoscaling.enabled=true --set autoscaling.targetMemoryUtilizationPercentage=80 --set resources.requests.cpu=1 --set resources.requests.memory=1Gi
check "an HPA with only extraMetrics needs no requests" \
renders --set autoscaling.enabled=true --set autoscaling.targetCPUUtilizationPercentage=null --set 'autoscaling.extraMetrics[0].type=Pods'
check "a KEDA cpu Utilization trigger without resources.requests.cpu is refused" \
refuses "requires resources.requests.cpu" --set keda.enabled=true --set autoscaling.enabled=false --set 'keda.triggers[0].type=cpu' --set 'keda.triggers[0].metricType=Utilization' --set 'keda.triggers[0].metadata.value=60'
check "a KEDA memory Utilization trigger without resources.requests.memory is refused" \
refuses "requires resources.requests.memory" --set keda.enabled=true --set autoscaling.enabled=false --set 'keda.triggers[0].type=memory' --set 'keda.triggers[0].metricType=Utilization' --set 'keda.triggers[0].metadata.value=60' --set resources.requests.cpu=1
check "a KEDA cpu trigger with metricType omitted counts as Utilization and is refused" \
refuses "requires resources.requests.cpu" --set keda.enabled=true --set autoscaling.enabled=false --set 'keda.triggers[0].type=cpu' --set 'keda.triggers[0].metadata.value=60'
check "a KEDA cpu AverageValue trigger needs no request" \
renders --set keda.enabled=true --set autoscaling.enabled=false --set 'keda.triggers[0].type=cpu' --set 'keda.triggers[0].metricType=AverageValue' --set 'keda.triggers[0].metadata.value=500m'
check "KEDA cpu and memory Utilization triggers render once the requests are set" \
renders --set keda.enabled=true --set autoscaling.enabled=false --set 'keda.triggers[0].type=cpu' --set 'keda.triggers[0].metadata.value=60' --set 'keda.triggers[1].type=memory' --set 'keda.triggers[1].metadata.value=60' --set resources.requests.cpu=1 --set resources.requests.memory=1Gi
check "a KEDA prometheus trigger needs no request" \
renders --set keda.enabled=true --set autoscaling.enabled=false --set 'keda.triggers[0].type=prometheus' --set 'keda.triggers[0].metadata.serverAddress=http://p:9090' --set 'keda.triggers[0].metadata.query=up' --set 'keda.triggers[0].metadata.threshold=1'

[ "$fails" -eq 0 ] || { echo "$fails render check(s) failed"; exit 1; }
24 changes: 19 additions & 5 deletions charts/aisix/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,9 +286,20 @@ autoscaling:
minReplicas: 2
maxReplicas: 20
targetCPUUtilizationPercentage: 70
resources:
requests:
cpu: "2"
memory: 1Gi
config:
proxy:
workers: 2
```

Requires `metrics-server` in the cluster. A replica the autoscaler adds is kept out
Requires `metrics-server` in the cluster. The utilization target is measured
against `resources.requests.cpu`, which the chart does not default, so the render
fails without it (likewise `resources.requests.memory` for a memory target). The
numbers above are placeholders — size them from your own traffic, and keep
`config.proxy.workers` equal to the CPUs you request. A replica the autoscaler adds is kept out
of the Service until its readiness probe passes, which happens only after it has
loaded its configuration from the control plane — so a scaling event never routes
traffic to a gateway that cannot serve it.
Expand Down Expand Up @@ -339,6 +350,11 @@ A streamed request is counted in `aisix_llm_requests_total` when the stream
ends, not when it starts, so during a burst of long streams this signal lags by
roughly one stream length. The steady-state rate is unaffected.

A KEDA `cpu` or `memory` trigger with `metricType: Utilization` — KEDA's default
when `metricType` is omitted — is measured against the matching
`resources.requests` entry, which the chart does not default, so the render fails
without it. `metricType: AverageValue` needs no request.

`autoscaling` and `keda` are mutually exclusive — enabling both fails the render
rather than letting two controllers fight over the replica count.

Expand Down Expand Up @@ -575,7 +591,7 @@ rejects. New installs set the value in its own key in `values.yaml` or under
| config.proxy.request_id.accept_headers | list | `["x-aisix-request-id"]` | Headers a caller may supply its own request id in. Add `x-request-id` only when no proxy in front stamps it |
| config.proxy.thread_per_core | bool | `nil` | Thread-per-core serving topology. Null uses the gateway default (on for Linux) |
| config.proxy.url_rewrites | list | `[]` | Entry-level URL rewrite rules, first match wins, e.g. `[{name: per-server-mcp, match: "^/mcp-servers/([^/]+)/mcp$", rewrite: "/mcp/$1"}]` |
| config.proxy.workers | int | `nil` | Worker count. Null uses the gateway default |
| config.proxy.workers | int | `nil` | Worker count. No default: null uses the gateway default, every CPU the process sees — the node's, unless a CPU limit caps it. Also sizes the token-estimate pool: one thread per worker, each holding one tokenizer instance per encoding it has counted with (about 17 MiB cl100k, 27 MiB o200k, 39 MiB o200k_harmony resident). Set it to the CPUs allocated to the pod, and size `resources` memory for it |
| config.ratelimit.concurrency_ttl_secs | int | `300` | Seconds a concurrency-limit slot is held at most |
| config.ratelimit.redis | object | `nil` | Extra settings for the shared rate-limit Redis selected by `rateLimit.backend: redis`, e.g. `{timeout_secs: 2}` or `{mode: cluster, nodes: [...]}`. Same keys as `cache.redis`; the URL comes from `rateLimit.redis`, `username` and `password` go through `configSecrets` |
| config.shutdown.min_drain_secs | int | `30` | Seconds the gateway keeps accepting after SIGTERM while `/readyz` reports 503. `terminationGracePeriodSeconds` must cover it |
Expand Down Expand Up @@ -662,9 +678,7 @@ rejects. New installs set the value in its own key in `values.yaml` or under
| readinessProbe.failureThreshold | int | `3` | |
| readinessProbe.periodSeconds | int | `3` | |
| replicaCount | int | `2` | Number of gateway replicas. Ignored once `autoscaling.enabled` or `keda.enabled` is true — the autoscaler owns the replica count from then on and the Deployment omits `spec.replicas` so `helm upgrade` cannot reset it. |
| resources.limits.memory | string | `"1Gi"` | |
| resources.requests.cpu | string | `"500m"` | |
| resources.requests.memory | string | `"256Mi"` | |
| resources | object | `{}` | Gateway container resources. No default requests or limits: neither CPU nor memory has a value that fits every deployment — both depend on the cluster and the traffic, and memory grows with `config.proxy.workers` (each worker's token-estimate thread holds its own tokenizer, see that key). Set `requests` from your own load, and `config.proxy.workers` to the CPUs you allocate. A CPU limit throttles the proxy under CFS, adding tail latency. Autoscaling on a CPU or memory utilization target needs the matching request: the render fails without it. |
| securityContext.allowPrivilegeEscalation | bool | `false` | |
| securityContext.capabilities.add[0] | string | `"NET_BIND_SERVICE"` | |
| securityContext.capabilities.drop[0] | string | `"ALL"` | |
Expand Down
18 changes: 17 additions & 1 deletion charts/aisix/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -280,9 +280,20 @@ autoscaling:
minReplicas: 2
maxReplicas: 20
targetCPUUtilizationPercentage: 70
resources:
requests:
cpu: "2"
memory: 1Gi
config:
proxy:
workers: 2
```

Requires `metrics-server` in the cluster. A replica the autoscaler adds is kept out
Requires `metrics-server` in the cluster. The utilization target is measured
against `resources.requests.cpu`, which the chart does not default, so the render
fails without it (likewise `resources.requests.memory` for a memory target). The
numbers above are placeholders — size them from your own traffic, and keep
`config.proxy.workers` equal to the CPUs you request. A replica the autoscaler adds is kept out
of the Service until its readiness probe passes, which happens only after it has
loaded its configuration from the control plane — so a scaling event never routes
traffic to a gateway that cannot serve it.
Expand Down Expand Up @@ -333,6 +344,11 @@ A streamed request is counted in `aisix_llm_requests_total` when the stream
ends, not when it starts, so during a burst of long streams this signal lags by
roughly one stream length. The steady-state rate is unaffected.

A KEDA `cpu` or `memory` trigger with `metricType: Utilization` — KEDA's default
when `metricType` is omitted — is measured against the matching
`resources.requests` entry, which the chart does not default, so the render fails
without it. `metricType: AverageValue` needs no request.

`autoscaling` and `keda` are mutually exclusive — enabling both fails the render
rather than letting two controllers fight over the replica count.

Expand Down
21 changes: 21 additions & 0 deletions charts/aisix/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -435,9 +435,30 @@ Reject value combinations that render successfully but cannot run.
{{- if and .Values.autoscaling.enabled .Values.keda.enabled }}
{{- fail "autoscaling.enabled and keda.enabled are mutually exclusive: two controllers writing spec.replicas fight over the replica count" }}
{{- end }}
{{- if .Values.autoscaling.enabled }}
{{- $requests := dig "requests" dict (.Values.resources | default dict) }}
{{- if and .Values.autoscaling.targetCPUUtilizationPercentage (not $requests.cpu) }}
{{- fail "autoscaling.targetCPUUtilizationPercentage requires resources.requests.cpu: utilization is measured against the request, so without one the HorizontalPodAutoscaler never scales. Set resources.requests.cpu, or set autoscaling.targetCPUUtilizationPercentage=null" }}
{{- end }}
{{- if and .Values.autoscaling.targetMemoryUtilizationPercentage (not $requests.memory) }}
{{- fail "autoscaling.targetMemoryUtilizationPercentage requires resources.requests.memory: utilization is measured against the request, so without one the HorizontalPodAutoscaler never scales. Set resources.requests.memory, or set autoscaling.targetMemoryUtilizationPercentage=null" }}
{{- end }}
{{- end }}
{{- if and .Values.keda.enabled (not .Values.keda.triggers) }}
{{- fail "keda.enabled requires at least one entry in keda.triggers" }}
{{- end }}
{{- if .Values.keda.enabled }}
{{- $requests := dig "requests" dict (.Values.resources | default dict) }}
{{- range .Values.keda.triggers }}
{{- if has (toString .type) (list "cpu" "memory") }}
{{- /* KEDA defaults a resource trigger to Utilization; metadata.type is its deprecated spelling of metricType */}}
{{- $metricType := .metricType | default (dig "type" "" (.metadata | default dict)) | default "Utilization" }}
{{- if and (eq (toString $metricType) "Utilization") (not (get $requests (toString .type))) }}
{{- fail (printf "a keda.triggers %s trigger with metricType Utilization requires resources.requests.%s: utilization is measured against the request, so without one KEDA's HorizontalPodAutoscaler never scales. Set resources.requests.%s, or use metricType: AverageValue" .type .type .type) }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- $config := .Values.config | default dict }}
{{- if eq .Values.rateLimit.backend "redis" }}
{{- $mode := "single" }}
Expand Down
25 changes: 15 additions & 10 deletions charts/aisix/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,12 @@ config:
# -- (bool) Thread-per-core serving topology. Null uses the gateway default (on
# for Linux)
thread_per_core: null
# -- (int) Worker count. Null uses the gateway default
# -- (int) Worker count. No default: null uses the gateway default, every CPU
# the process sees — the node's, unless a CPU limit caps it. Also sizes the
# token-estimate pool: one thread per worker, each holding one tokenizer
# instance per encoding it has counted with (about 17 MiB cl100k, 27 MiB
# o200k, 39 MiB o200k_harmony resident). Set it to the CPUs allocated to the
# pod, and size `resources` memory for it
workers: null
# -- Entry-level URL rewrite rules, first match wins, e.g.
# `[{name: per-server-mcp, match: "^/mcp-servers/([^/]+)/mcp$", rewrite: "/mcp/$1"}]`
Expand Down Expand Up @@ -506,15 +511,15 @@ serviceAccount:
# -- ServiceAccount annotations
annotations: {}

## No CPU limit by default: CFS throttling adds tail latency on a proxy and
## suppresses the very CPU signal the HorizontalPodAutoscaler scales on. The
## CPU *request* is mandatory for CPU-target autoscaling and is set below.
resources:
requests:
cpu: 500m
memory: 256Mi
limits:
memory: 1Gi
# -- Gateway container resources. No default requests or limits: neither CPU
# nor memory has a value that fits every deployment — both depend on the
# cluster and the traffic, and memory
# grows with `config.proxy.workers` (each worker's token-estimate thread holds
# its own tokenizer, see that key). Set `requests` from your own load, and
# `config.proxy.workers` to the CPUs you allocate. A CPU limit throttles the
# proxy under CFS, adding tail latency. Autoscaling on a CPU or memory
# utilization target needs the matching request: the render fails without it.
resources: {}

## The image already runs as uid 10001 and declares it numerically, which is
## what `runAsNonRoot` needs to verify without the uid being repeated here.
Expand Down
Loading