From 7039028b8b2609f59dec9829b6ff85ed3ef8f1e8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sebasti=C3=A1n=20Ramiro=20Entrerrios=20Garc=C3=ADa?= <100527906+Sebas1705@users.noreply.github.com> Date: Fri, 13 Nov 2026 23:44:00 +0100 Subject: [PATCH 1/2] docs: describe only the remaining ai-engine llama target routing control --- .../capabilities/edge/public-edge-routing.md | 23 +- .../operations/runtime-routing-control.md | 153 ++++--------- guides/communication/service-communication.md | 21 +- .../runtime-routing-and-service-targeting.md | 216 ++++-------------- 4 files changed, 107 insertions(+), 306 deletions(-) diff --git a/guides/capabilities/edge/public-edge-routing.md b/guides/capabilities/edge/public-edge-routing.md index 721fe98..8986236 100644 --- a/guides/capabilities/edge/public-edge-routing.md +++ b/guides/capabilities/edge/public-edge-routing.md @@ -1,17 +1,17 @@ # Public Edge Routing -Last updated: 2026-05-03. +Last updated: 2026-11-13. ## Purpose -This capability provides the stable ingress boundary for mobile and backoffice clients while keeping downstream services private, replaceable, and operationally reroutable. +This capability provides the stable ingress boundary for mobile and backoffice clients while keeping downstream services private and replaceable. ## Functional objective - accept public traffic through one edge service - apply edge auth, CORS, timeout, and forwarding policy before reaching BFFs - expose internal AI proxy routes without making ai-engine public -- preserve a stable public surface even when runtime targets change underneath +- preserve a stable public surface when downstream targets change ## Ownership and boundaries @@ -33,8 +33,8 @@ This capability provides the stable ingress boundary for mobile and backoffice c ### Explicit boundaries -- owns public ingress, route registration, forwarding, and AI proxy target resolution in `api-gateway/src/app/routes/proxy.ts` -- owns persisted AI target overrides in `api-gateway/src/app/services/routingStateStore.ts` +- owns public ingress and the route table in `api-gateway/src/app/routes/routeTable.ts`, with forwarding and AI proxy target resolution in `proxy.ts` +- still owns a persisted target override for the `/internal/ai-engine/*` proxy in `api-gateway/src/app/services/routingStateStore.ts`; `bff-backoffice` no longer writes to it (see [Runtime routing and service targeting](../../../operations/runtime-routing-and-service-targeting.md)) - delegates business orchestration to BFFs and AI execution to ai-engine - does not own domain validation, persistence, or operator UI semantics @@ -48,7 +48,7 @@ This capability provides the stable ingress boundary for mobile and backoffice c | HTTP route | `/v1/mobile/*` | public ingress toward `bff-mobile` | | HTTP route | `/v1/backoffice/*` | public ingress toward `bff-backoffice` | | HTTP route | `/internal/ai-engine/*` | internal proxy for generation, ingest, catalogs, health, and stats | -| HTTP route | `/internal/admin/ai-engine/target` | privileged runtime target read/update/delete | +| HTTP route | `/internal/admin/ai-engine/target` | privileged read/update/delete of the gateway's own AI proxy target (`API_GATEWAY_ADMIN_TOKEN`); not used by the Backoffice flow | ### Inputs @@ -61,7 +61,7 @@ This capability provides the stable ingress boundary for mobile and backoffice c - forwarded public responses for mobile and backoffice clients - forwarded internal responses for ai-engine generation, ingest, health, and stats -- persisted side effects when operators mutate the AI target through admin routes +- persisted side effects when the gateway admin target route is used ### Contract sources @@ -74,7 +74,7 @@ This capability provides the stable ingress boundary for mobile and backoffice c 1. A public client reaches `api-gateway` on a `/v1/mobile/*` or `/v1/backoffice/*` route. 2. The gateway applies edge auth and request policy in `server.ts`, including health-route exemptions and route classification for rate limiting. -3. `proxy.ts` resolves the correct upstream target, using persisted AI overrides for `/internal/ai-engine/*` and static BFF URLs for public BFF paths. +3. `proxy.ts` resolves the upstream from the route table, using the gateway AI target (environment default or persisted override) for `/internal/ai-engine/*` and static BFF URLs for public BFF paths. 4. The gateway forwards the request through shared HTTP helpers and route-specific circuit breakers. 5. The response is returned to the caller while preserving edge-visible status and correlation context. @@ -82,19 +82,19 @@ This capability provides the stable ingress boundary for mobile and backoffice c ### Owned persistence -- gateway routing state persisted by `RoutingStateStore` +- gateway AI proxy target persisted by `RoutingStateStore` ### Runtime state - default upstream URLs from `api-gateway/src/app/config.ts` -- effective AI API and stats targets computed from env defaults plus persisted overrides +- effective AI API and stats targets for `/internal/ai-engine/*` computed from env defaults plus the persisted gateway override - per-upstream circuit-breaker state inside the gateway process ### State risks - stale AI target after restart or redeploy because the routing state file survives - gateway target drift relative to BFF or ai-engine runtime state -- public route availability diverging from operator expectations when the gateway lags behind downstream route additions +- public route availability diverging from operator expectations when the gateway lags behind downstream route additions or removals: the route table still lists some `bff-backoffice` paths that no longer exist (`/kubernetes/overview`, `/ai-engine/presets*`, `/ai-engine/connections*`, `/service-targets*`), which end in an upstream error; cleanup is pending ## Observability evidence @@ -111,6 +111,7 @@ This capability provides the stable ingress boundary for mobile and backoffice c ### Verification evidence - `api-gateway/src/tests/proxy.test.ts` +- `api-gateway/src/tests/proxyRouteEquivalence.test.ts` - `api-gateway/src/tests/routingStateStore.test.ts` - focused health-route tests in `api-gateway/src/tests/health.test.ts` diff --git a/guides/capabilities/operations/runtime-routing-control.md b/guides/capabilities/operations/runtime-routing-control.md index 651126c..632b4d3 100644 --- a/guides/capabilities/operations/runtime-routing-control.md +++ b/guides/capabilities/operations/runtime-routing-control.md @@ -1,139 +1,70 @@ # Runtime Routing Control -Last updated: 2026-05-03. +Last updated: 2026-11-13. ## Purpose -This capability lets operators change effective service and AI routing at runtime, persist those overrides, and return safely to defaults without rebuilding the platform. - -## Functional objective - -- persist and apply runtime target overrides and presets -- expose the effective routing picture to operators -- support controlled rollback from overrides back to default topology +This capability lets operators point `ai-engine-api` at a different llama runtime without a redeploy, persist that override, and return to the environment default. It is the only runtime routing control that remains; the full flow is in [Runtime routing and service targeting](../../../operations/runtime-routing-and-service-targeting.md). ## Ownership and boundaries -### Main owners - -- `bff-backoffice` -- `api-gateway` -- `ai-engine` - -### Upstream actors - -- backoffice operators - -### Explicit boundaries +| Owner | Responsibility | Code | +|---|---|---| +| `Backoffice` | llama target card (`AiEngineTargetControl`) in AI diagnostics | `Backoffice/src/src/ui/components/AiEngineTargetControl.tsx` | +| `api-gateway` | plain forwarding of `/v1/backoffice/*` to the BFF | `api-gateway/src/app/routes/routeTable.ts` | +| `bff-backoffice` | validation, host allowlist, probe, sync to ai-engine | `bff-backoffice/src/app/routes/backoffice.ts` | +| `ai-engine-api` | owns and persists the override | `ai-engine/src/ai_engine/api/app.py` | -- owns BFF-side service target and AI preset management in `bff-backoffice/src/app/routes/backoffice.ts` -- owns gateway-side AI target persistence in `api-gateway/src/app/services/routingStateStore.ts` -- owns ai-engine local llama target override in `ai-engine` -- does not replace deploy-time configuration management in `platform-infra`, but can override runtime behavior after deploy +Out of scope: deploy-time configuration (`platform-infra`) and any BFF-side routing state (none exists). ## Entrypoints and contracts -### Main entrypoints - -| Type | Location | Notes | +| Type | Route | Notes | |---|---|---| -| HTTP route | `GET/PUT/DELETE /v1/backoffice/ai-engine/target` | BFF AI target inspection and mutation | -| HTTP route | `GET/POST/PUT/DELETE /v1/backoffice/ai-engine/presets*` | shared AI preset management | -| HTTP route | `/v1/backoffice/service-targets/:service` | runtime service override control | -| HTTP route | `GET/PUT/DELETE /internal/admin/ai-engine/target` | gateway-owned AI target control | -| HTTP route | ai-engine internal llama target routes | ai-engine local runtime override | +| HTTP | `GET/PUT/DELETE /v1/backoffice/ai-engine/target` | read, apply, clear | +| HTTP | `POST /v1/backoffice/ai-engine/probe` | non-mutating reachability check of `/v1/models` | +| HTTP | `GET /v1/backoffice/ai-engine/plugin/capabilities` | effective plugin runtime capabilities | +| HTTP (private) | `GET/PUT/DELETE /internal/admin/llama-target` on `ai-engine-api` | called by the BFF with `x-api-key` | -### Inputs - -- operator target changes and preset payloads validated by Zod schemas -- current runtime state from BFF routing store, gateway routing store, and ai-engine target store -- service identifiers such as `ai-engine-api`, `ai-engine-stats`, and other BFF-managed upstreams - -### Outputs - -- persisted override files and preset state -- effective runtime target changes -- reset-to-default side effects when overrides are cleared - -### Contract sources - -- `bff-backoffice/src/app/routes/backoffice.ts` -- `bff-backoffice/src/app/services/routingStateStore.ts` -- `api-gateway/src/app/services/routingStateStore.ts` -- `api-gateway/src/app/routes/proxy.ts` +Payload for `PUT` and `POST probe`: `host`, `protocol` (`http` or `https`), `port`, optional `label`, validated with Zod in the BFF. ## Main functional flow -1. An operator changes a target or preset in backoffice. -2. `bff-backoffice` validates the payload and persists the change in its routing-state store. -3. When the change affects AI execution, the BFF synchronizes runtime state toward `api-gateway` and ai-engine as needed. -4. Subsequent requests use the new effective target without a redeploy. -5. Operators verify health and can clear the override to fall back to defaults. - -## Persistence and runtime state - -### Owned persistence +1. The operator edits the llama target card and optionally probes it. +2. The BFF checks the host against `ALLOWED_ROUTING_TARGET_HOSTS` and calls `ai-engine-api`. +3. `ai-engine-api` persists the override in `AI_ENGINE_LLAMA_TARGET_STATE_FILE` and applies it. +4. The BFF clears its upstream cache and breakers and returns the read-back target. +5. `DELETE` restores the environment default. -- BFF service-target overrides and AI preset records -- gateway AI target override file -- ai-engine local llama target override file +## Persistence and risks -### Runtime state +- State lives only in `ai-engine-api` (file on mounted storage); it survives restarts and redeploys. +- A stale override after a rollout leaves generation pointing at an unexpected runtime while pods look healthy. +- Rolling back images does not roll back the override. -- effective topology is computed from environment defaults plus one or more persisted overrides -- overrides survive restart and redeploy unless explicitly cleared -- BFF, gateway, and ai-engine can each hold part of the routing truth +## Failure modes -### State risks - -- stale override after a healthy rollout leaves runtime using an unexpected target -- sync-order mistakes across BFF, gateway, and ai-engine create split routing state -- UI can display one topology while the actual traffic path uses another - -## Observability evidence - -### Health signals - -- health probes executed after target changes - -### Operational signals - -- target and preset readback routes in the BFF and gateway -- operator probe route `POST /v1/backoffice/ai-engine/probe` -- runtime-target comparison in backoffice diagnostics - -### Verification evidence - -- `bff-backoffice/src/tests/routingStateStore.test.ts` -- `bff-backoffice/src/tests/backofficeRouteGuards.test.ts` -- `api-gateway/src/tests/routingStateStore.test.ts` -- `api-gateway/src/tests/proxy.test.ts` - -## Failure modes and risks - -| Failure mode | Likely cause | User/operator impact | Detection path | Recovery path | -|---|---|---|---|---| -| Wrong target configured | operator error or stale preset | runtime requests fail while pods stay healthy | target read routes and smoke requests | reset to default or apply a known-good preset | -| State mismatch across owners | failed sync or concurrent edits | UI shows one topology while runtime uses another | compare BFF, gateway, and ai-engine target reads | resynchronize holders and retest | -| Override survives too long | runtime holder not cleared after rollout | traffic keeps pointing to a temporary host | post-deploy routing inspection | explicitly clear override and verify | +| Failure mode | Likely cause | Detection | Recovery | +|---|---|---|---| +| `400` on `PUT` | host not allowed or invalid payload | response message | fix the payload or `ALLOWED_ROUTING_TARGET_HOSTS` | +| Probe not reachable | wrong host/port or runtime down | probe result | correct the target | +| Target set but generation fails | llama runtime unreachable or warming up | smoke request, `ai-engine-api` health | `DELETE` the override or fix the runtime | +| Admin call times out | `ai-engine-api` hung | BFF error citing the timeout (`AI_ENGINE_ADMIN_TIMEOUT_MS`) | restore `ai-engine-api` | ## Security and access model -- routing mutation paths are high-impact admin operations and should remain role-gated and auditable -- preset and target payloads can reveal sensitive internal topology and must stay on protected surfaces -- route guards in BFF and gateway are part of the control boundary, not optional UI behavior - -## Deployment and rollback implications +- Mutating routes are high-impact admin operations behind the backoffice authentication path. +- Targets are restricted by `ALLOWED_ROUTING_TARGET_HOSTS` (denied outside development when empty). +- `ai-engine-api` internal admin routes are not public. -### Deployment impact +## Observability and verification -- routing overrides can outlive container rollouts and therefore change the effective behavior of an otherwise healthy deployment -- gateway, BFF, and ai-engine changes must preserve routing-state compatibility across restarts +- Readback via `GET /v1/backoffice/ai-engine/target`; reachability via the probe route. +- Tests: `bff-backoffice/src/tests/backoffice.test.ts`, `backofficeBranches.test.ts`, `routingAllowlist.test.ts`; `ai-engine/src/tests/test_api/test_llama_target_store.py`; `Backoffice/src/src/test/ai-diagnostics-panel.integration.test.tsx`. -### Rollback impact +## History -- rolling back binaries does not automatically roll back persisted routing state -- safe rollback often requires both image rollback and explicit override reset or reapplication +Service-target overrides, shared presets and connections, the Kubernetes overview, routing history, and the matching Backoffice tab and presets selector were removed (`bff-backoffice` PR #9, `Backoffice` PR #7). The gateway still carries a separate `/internal/admin/ai-engine/target` store and some dead forwarding entries; see the operations reference. ## Related capabilities @@ -142,13 +73,7 @@ This capability lets operators change effective service and AI routing at runtim - [AI observability and diagnostics](../ai/ai-observability-and-diagnostics.md) - [Backoffice operational diagnostics](./backoffice-operational-diagnostics.md) -## Open questions - -- whether all routing transitions are audited consistently enough across BFF, gateway, and ai-engine holders -- whether concurrent operator edits need stronger conflict semantics and UI signaling - ## References - [Platform capability matrix](../../foundations/platform-capability-matrix.md) - [Platform functional overview](../../foundations/platform-functional-overview.md) -- [Backoffice operational diagnostics](./backoffice-operational-diagnostics.md) diff --git a/guides/communication/service-communication.md b/guides/communication/service-communication.md index 696795e..3bd8acc 100644 --- a/guides/communication/service-communication.md +++ b/guides/communication/service-communication.md @@ -1,6 +1,6 @@ # Service Communication Model -Last updated: 2026-04-05. +Last updated: 2026-11-13. ## Synchronous request flow @@ -16,7 +16,7 @@ Last updated: 2026-04-05. | Client applications | `api-gateway` | Public boundary only | | `api-gateway` | `bff-mobile`, `bff-backoffice`, internal ai-engine proxy routes | No direct client access to domain services | | `bff-mobile` | domain services | BFF-to-BFF calls are not allowed | -| `bff-backoffice` | domain services, ai diagnostics endpoints, gateway admin sync | Backoffice runtime control path is explicit | +| `bff-backoffice` | domain services, ai diagnostics endpoints, `ai-engine-api` llama target admin routes | Backoffice runtime control path is explicit | | quiz/word-pass services | `ai-engine-api`, `ai-engine-stats` | Internal AI producer traffic | | `ai-engine-api` | llama runtime | Model execution path, may be externalized | @@ -57,12 +57,17 @@ Last updated: 2026-04-05. ### Runtime control routes +Only the llama target remains as a runtime control. See [Runtime routing and service targeting](../../operations/runtime-routing-and-service-targeting.md). + | Method | Route | Owner | |---|---|---| -| GET/PUT/DELETE | `/internal/admin/ai-engine/target` | `api-gateway` | -| GET/PUT/DELETE | `/v1/backoffice/ai-engine/target` | `bff-backoffice` | -| GET/POST/PUT/DELETE | `/v1/backoffice/ai-engine/presets` | `bff-backoffice` | -| GET/PUT/DELETE | `/v1/backoffice/service-targets/:service` | `bff-backoffice` | +| GET/PUT/DELETE | `/v1/backoffice/ai-engine/target` | `bff-backoffice` (forwarded by the gateway) | +| POST | `/v1/backoffice/ai-engine/probe` | `bff-backoffice` | +| GET | `/v1/backoffice/ai-engine/plugin/capabilities` | `bff-backoffice` | +| GET/PUT/DELETE | `/internal/admin/llama-target` | `ai-engine-api` (private, called by the BFF) | +| GET/PUT/DELETE | `/internal/admin/ai-engine/target` | `api-gateway` (separate gateway target store, no longer used by the BFF; cleanup pending) | + +History: service targets, ai-engine presets and connections, and the Kubernetes overview were removed from `bff-backoffice` and the Backoffice UI. ## Asynchronous flow @@ -81,7 +86,7 @@ For generation-related traffic, the logical dependency chain is: 5. llama runtime returns model output 6. `ai-engine-api` records stats and returns validated payloads upstream -This means an apparently healthy deploy can still fail functionally if the persisted live AI target or llama target points to an unreachable runtime. +This means an apparently healthy deploy can still fail functionally if the persisted llama target points to an unreachable runtime. ## Communication rules @@ -89,7 +94,7 @@ This means an apparently healthy deploy can still fail functionally if the persi - BFF-to-BFF calls are disallowed. - Timeouts and retry strategy must be explicit per dependency. - `x-correlation-id` must be propagated end-to-end. -- Environment defaults and runtime overrides must be observable independently. +- Environment defaults and the llama target override must be observable independently. - Auth/security headers to preserve when applicable: - `authorization` - `x-firebase-id-token` diff --git a/operations/runtime-routing-and-service-targeting.md b/operations/runtime-routing-and-service-targeting.md index e0ff45a..d2a3949 100644 --- a/operations/runtime-routing-and-service-targeting.md +++ b/operations/runtime-routing-and-service-targeting.md @@ -1,204 +1,74 @@ # Runtime Routing And Service Targeting -Last updated: 2026-05-03. +Last updated: 2026-11-13. ## Purpose -This document explains the runtime routing controls that exist outside the static Kubernetes manifests. It is intentionally separate from deployment documentation because the effective topology can differ from the deployed topology, especially for `ai-engine`. +This document explains the one runtime routing control that exists outside the static Kubernetes manifests: the `ai-engine-api` llama target. Everything else is wired declaratively through `platform-infra` and environment variables. ## When to use this -- when diagnosing traffic that does not match the deployed Kubernetes manifests -- when operators need to apply, inspect, or clear runtime target overrides -- when validating how `bff-backoffice`, `api-gateway`, and `ai-engine-api` interact as runtime state holders +- when diagnosing generation traffic that does not match the deployed Kubernetes manifests +- when an operator needs to apply, probe, inspect, or clear the llama target +- when validating how `Backoffice`, `bff-backoffice`, and `ai-engine-api` interact for that target -## Why this exists +## What exists -Most services are deployed declaratively through `platform-infra`. However, the platform also supports operational retargeting for selected upstreams when a service must temporarily point to: +The llama target is the host, port, and protocol that `ai-engine-api` uses to reach the model runtime (llama-compatible server). It defaults to the environment value and can be overridden at runtime, for example to point staging at a workstation-hosted model while API and stats stay in-cluster (see [ADR 0004](../architecture/decision-records/0004-split-ai-runtime.md)). -- a workstation-hosted dependency -- an alternate node -- a relay endpoint -- a controlled fallback instance +### Operator flow -The current primary use case is `ai-engine` externalization in staging. +1. The operator edits the llama target card in the `Backoffice` AI diagnostics view. +2. `Backoffice` calls `api-gateway`, which forwards `/v1/backoffice/ai-engine/*` to `bff-backoffice` as a plain proxy route. +3. `bff-backoffice` validates the payload, checks the host against `ALLOWED_ROUTING_TARGET_HOSTS`, and calls `ai-engine-api` at `/internal/admin/llama-target` with the service API key (`x-api-key`). +4. `ai-engine-api` rebuilds its llama URL, persists the override, and applies it to its runtime client. +5. The BFF clears its upstream cache and breaker state and returns the effective target read back from `ai-engine-api`. -## Components involved +### Routes -### `bff-backoffice` +| Layer | Method and route | Notes | +|---|---|---| +| BFF (public through the gateway) | `GET /v1/backoffice/ai-engine/target` | effective target read back from `ai-engine-api` | +| BFF | `PUT /v1/backoffice/ai-engine/target` | body `host`, `protocol` (`http` or `https`, default `http`), `port` (default `7002`), optional `label` | +| BFF | `DELETE /v1/backoffice/ai-engine/target` | clear the override and return to the environment default | +| BFF | `POST /v1/backoffice/ai-engine/probe` | same body as `PUT`; checks `/v1/models` without changing state | +| BFF | `GET /v1/backoffice/ai-engine/plugin/capabilities` | reads `ai-engine-api` `/internal/plugin/capabilities`; `502` when unreachable | +| `ai-engine-api` (private) | `GET/PUT/DELETE /internal/admin/llama-target` | owner of the override; not exposed publicly | +| `ai-engine-api` (private) | `GET /internal/plugin/capabilities` | effective plugin runtime capabilities | -Responsibilities: +## Persistence -- expose operational APIs for service target inspection and override -- persist shared routing state for backoffice-driven upstream changes -- persist reusable ai-engine destination presets shared across browsers and operators +- `ai-engine-api` stores the override in `AI_ENGINE_LLAMA_TARGET_STATE_FILE` (default `data/llama-target-state.json`), so it survives pod recreation. `DELETE` resets the file and falls back to the environment URL. +- `bff-backoffice` keeps no routing state: it has no routing-state store on `main`. -### `api-gateway` - -Responsibilities: - -- hold the active live ai-engine target used by the runtime request path -- enforce approved-host rules through environment-level allowlists -- proxy operator-facing ai-engine target, preset, connection, probe, and plugin-capability routes to `bff-backoffice` - -### `ai-engine-api` - -Responsibilities: - -- hold the active llama target used by generation requests -- preserve effective model-runtime selection across restart or pod recreation -- allow split-AI staging to keep API/stats in-cluster while the model runtime remains elsewhere - -### `backoffice` - -Responsibilities: - -- expose operator-facing controls in diagnostics and service overview panels -- allow shared preset management for ai-engine destinations -- allow live runtime target application without a full redeploy - -## Current runtime routing use cases - -### Use case 1: staging ai-engine runs on an external workstation - -Effective path: - -1. backoffice operator selects a shared ai-engine preset -2. `bff-backoffice` calls the compatibility route to apply the target -3. `api-gateway` stores the live target -4. requests to ai-engine are forwarded to the configured host and ports - -### Use case 2: backoffice needs a shared list of candidate ai-engine destinations - -The preset list is stored in `bff-backoffice` persisted runtime state so all browsers see the same options. - -### Use case 3: operator wants to revert to environment defaults - -The routing layer supports returning from override state back to environment-derived endpoints. +## Security boundaries -### Use case 4: split-AI staging keeps API in-cluster but llama external +- `PUT` is rejected unless the host matches `ALLOWED_ROUTING_TARGET_HOSTS` (exact host, `*.suffix`, or IPv4 CIDR). When the variable is empty, overrides are denied outside development. +- The probe route performs a server-side request to an operator-supplied host; keep it on the authenticated backoffice surface. -Effective path: +## Gateway state (current, cleanup pending) -1. cluster services call `ai-engine-api` -2. `ai-engine-api` resolves its active llama target -3. model traffic reaches the selected external or alternate llama runtime -4. API/stats remain observable through the cluster even if model execution lives elsewhere +`api-gateway` still has a second, separate target store on `main`: `GET/PUT/DELETE /internal/admin/ai-engine/target` (guarded by `API_GATEWAY_ADMIN_TOKEN`, persisted in `GATEWAY_ROUTING_STATE_FILE`) that can redirect the `/internal/ai-engine/*` proxy. `bff-backoffice` no longer calls it, so the Backoffice flow above does not touch it. The gateway route table also still forwards a few paths that `bff-backoffice` no longer serves (`/kubernetes/overview`, `/ai-engine/presets*`, `/ai-engine/connections*`, `/service-targets*`); those requests end in an upstream error. No pull request that removes them is open in `AxiomNode/api-gateway` as of this date, so treat this as current behavior and an open cleanup item. ## Validation -- current effective targets can be read back from the runtime control surfaces -- gateway and backoffice diagnostics reflect the selected target state clearly -- end-to-end traffic reaches the intended upstream after an override or reset -- llama reachability should be checked through `/v1/backoffice/ai-engine/probe` and runtime capability readback when diagnosing split-AI mode +- `GET /v1/backoffice/ai-engine/target` shows the effective target after a change or reset. +- `POST /v1/backoffice/ai-engine/probe` checks reachability before applying. +- A generation smoke request confirms end-to-end traffic reaches the intended runtime. ## Risks and cautions -- persisted overrides can outlive pod restarts and redeploys -- operators can misread effective topology if they only inspect manifests -- rollback is incomplete if images are reverted but runtime target state is left unchanged - -## Related documents - -- `docs/operations/deployment-strategy.md` -- `docs/architecture/target-architecture.md` -- `docs/guides/capabilities/operations/runtime-routing-control.md` - -## Persistence model - -### `bff-backoffice` - -Stores routing state in a small persistent JSON file on mounted storage. - -Persists: - -- configurable service target overrides -- shared ai-engine destination presets - -### `api-gateway` - -Stores the active ai-engine target in its own persistent JSON file on mounted storage. - -Persists: - -- active ai-engine API and stats host/port target - -This design survives pod recreation. It is not limited to in-memory process state. - -### `ai-engine-api` - -Stores the active llama target in its own persistent runtime state file on mounted storage. - -Persists: - -- active llama host/port target used by generation traffic - -This means the AI execution path can diverge from environment defaults even when the API deployment itself is healthy. - -## Security boundaries - -Runtime retargeting is constrained by allowlists. - -Key rule: - -- operators may change where traffic points only within the bounds allowed by `ALLOWED_ROUTING_TARGET_HOSTS` - -This prevents unrestricted arbitrary SSRF-style retargeting. - -## Operational model - -```mermaid -sequenceDiagram - participant Operator - participant Backoffice - participant BFF as bff-backoffice - participant Gateway as api-gateway - participant API as ai-engine-api - participant Llama as External or in-cluster llama - - Operator->>Backoffice: Select saved ai-engine destination - Backoffice->>BFF: PUT /v1/backoffice/ai-engine/target - BFF->>Gateway: Sync live ai-engine target - Gateway-->>BFF: Ack - BFF-->>Backoffice: Current runtime target - Backoffice-->>Operator: Updated runtime topology - Gateway->>API: Forward runtime traffic - API->>Llama: Use active llama target -``` - -## Design trade-offs - -Benefits: - -- fast operational recovery without full cluster redeploy -- shared operator-visible presets instead of private browser-only state -- preserved staging productivity when AI infrastructure is temporarily external - -Costs: - -- runtime state is no longer fully described by Kubernetes manifests alone -- operators must understand both declarative deploy state and runtime override state -- documentation and observability must explicitly surface the active target source - -## Rules for maintainability - -- keep runtime targeting limited to clearly justified services -- persist overrides and presets; do not rely on ad hoc manual pod edits -- expose current source state as `env` or `override` -- document all operator-facing control paths -- prefer shared backend persistence over browser-local state for team-operated controls - -## Recommended documentation practice +- the persisted override outlives pod restarts and redeploys +- rolling back images does not clear the override; reset it explicitly +- operators reading only manifests will miss the effective llama target -When a new runtime routing capability is introduced, update at least: +## History -- architecture overview -- deployment and CI/CD documentation -- operator runbook or diagnostics documentation -- persisted state or secret ownership notes +The BFF used to also expose service-target overrides (`/service-targets*`), shared ai-engine presets and connections (`/ai-engine/presets*`, `/ai-engine/connections*`), a Kubernetes overview (`/kubernetes/overview`), and routing history (`/routing/history`), with the Backoffice Kubernetes tab and presets selector. These were removed in `bff-backoffice` PR #9 and `Backoffice` PR #7; only the llama target remains. ## Related documents -- `docs/architecture/target-architecture.md` -- `docs/operations/cicd-workflow-map.md` -- `platform-infra/kubernetes/README.md` \ No newline at end of file +- [Runtime routing control](../guides/capabilities/operations/runtime-routing-control.md) +- [Deployment strategy](./deployment-strategy.md) +- [Target architecture](../architecture/target-architecture.md) +- [CI/CD workflow map](./cicd-workflow-map.md) From c0aac6fc6d9541b53bf4b69fbac2dfc04fab7266 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sebasti=C3=A1n=20Ramiro=20Entrerrios=20Garc=C3=ADa?= <100527906+Sebas1705@users.noreply.github.com> Date: Sat, 14 Nov 2026 01:37:59 +0100 Subject: [PATCH 2/2] docs: describe the gateway without routing state after the dead route cleanup --- architecture/key-sequence-flows.md | 2 +- .../capabilities/edge/public-edge-routing.md | 24 +++++++------------ .../operations/runtime-routing-control.md | 2 +- guides/communication/service-communication.md | 3 +-- operations/functionality-review-baseline.md | 2 +- .../runtime-routing-and-service-targeting.md | 4 ++-- 6 files changed, 14 insertions(+), 23 deletions(-) diff --git a/architecture/key-sequence-flows.md b/architecture/key-sequence-flows.md index d61653c..1ce64ff 100644 --- a/architecture/key-sequence-flows.md +++ b/architecture/key-sequence-flows.md @@ -41,7 +41,7 @@ Related: `POST /v1/mobile/games/quiz/generate` (and `/wordpass/generate`) is als ## SF-02 Backoffice operator changes the runtime AI target (UC-09) -There are two independent targets. This flow covers the llama target that `ai-engine-api` uses for generation. The gateway keeps its own ai-engine API/stats target, changed with `PUT /internal/admin/ai-engine/target` (guarded by `API_GATEWAY_ADMIN_TOKEN` when set); it is not changed by this flow. +This flow covers the llama target that `ai-engine-api` uses for generation. The gateway keeps no routing state: its ai-engine API/stats upstreams come only from environment variables. ```mermaid sequenceDiagram diff --git a/guides/capabilities/edge/public-edge-routing.md b/guides/capabilities/edge/public-edge-routing.md index 8986236..9feac90 100644 --- a/guides/capabilities/edge/public-edge-routing.md +++ b/guides/capabilities/edge/public-edge-routing.md @@ -34,7 +34,7 @@ This capability provides the stable ingress boundary for mobile and backoffice c ### Explicit boundaries - owns public ingress and the route table in `api-gateway/src/app/routes/routeTable.ts`, with forwarding and AI proxy target resolution in `proxy.ts` -- still owns a persisted target override for the `/internal/ai-engine/*` proxy in `api-gateway/src/app/services/routingStateStore.ts`; `bff-backoffice` no longer writes to it (see [Runtime routing and service targeting](../../../operations/runtime-routing-and-service-targeting.md)) +- resolves the `/internal/ai-engine/*` upstreams only from `AI_ENGINE_API_URL` and `AI_ENGINE_STATS_URL`; it holds no persisted routing state (the runtime llama target lives in `ai-engine-api`, see [Runtime routing and service targeting](../../../operations/runtime-routing-and-service-targeting.md)) - delegates business orchestration to BFFs and AI execution to ai-engine - does not own domain validation, persistence, or operator UI semantics @@ -48,20 +48,17 @@ This capability provides the stable ingress boundary for mobile and backoffice c | HTTP route | `/v1/mobile/*` | public ingress toward `bff-mobile` | | HTTP route | `/v1/backoffice/*` | public ingress toward `bff-backoffice` | | HTTP route | `/internal/ai-engine/*` | internal proxy for generation, ingest, catalogs, health, and stats | -| HTTP route | `/internal/admin/ai-engine/target` | privileged read/update/delete of the gateway's own AI proxy target (`API_GATEWAY_ADMIN_TOKEN`); not used by the Backoffice flow | ### Inputs - public HTTP requests from mobile and backoffice clients - edge auth tokens and correlation metadata - environment defaults for BFF and AI upstream URLs -- persisted AI routing override loaded from `GATEWAY_ROUTING_STATE_FILE` ### Outputs - forwarded public responses for mobile and backoffice clients - forwarded internal responses for ai-engine generation, ingest, health, and stats -- persisted side effects when the gateway admin target route is used ### Contract sources @@ -74,7 +71,7 @@ This capability provides the stable ingress boundary for mobile and backoffice c 1. A public client reaches `api-gateway` on a `/v1/mobile/*` or `/v1/backoffice/*` route. 2. The gateway applies edge auth and request policy in `server.ts`, including health-route exemptions and route classification for rate limiting. -3. `proxy.ts` resolves the upstream from the route table, using the gateway AI target (environment default or persisted override) for `/internal/ai-engine/*` and static BFF URLs for public BFF paths. +3. `proxy.ts` resolves the upstream from the route table, using the environment-configured AI targets for `/internal/ai-engine/*` and static BFF URLs for public BFF paths. 4. The gateway forwards the request through shared HTTP helpers and route-specific circuit breakers. 5. The response is returned to the caller while preserving edge-visible status and correlation context. @@ -82,19 +79,17 @@ This capability provides the stable ingress boundary for mobile and backoffice c ### Owned persistence -- gateway AI proxy target persisted by `RoutingStateStore` +- none ### Runtime state -- default upstream URLs from `api-gateway/src/app/config.ts` -- effective AI API and stats targets for `/internal/ai-engine/*` computed from env defaults plus the persisted gateway override +- upstream URLs from `api-gateway/src/app/config.ts` (BFF URLs, `AI_ENGINE_API_URL`, `AI_ENGINE_STATS_URL`) - per-upstream circuit-breaker state inside the gateway process ### State risks -- stale AI target after restart or redeploy because the routing state file survives -- gateway target drift relative to BFF or ai-engine runtime state -- public route availability diverging from operator expectations when the gateway lags behind downstream route additions or removals: the route table still lists some `bff-backoffice` paths that no longer exist (`/kubernetes/overview`, `/ai-engine/presets*`, `/ai-engine/connections*`, `/service-targets*`), which end in an upstream error; cleanup is pending +- gateway upstream URLs drifting from the real BFF or ai-engine addresses after a redeploy +- public route availability diverging from operator expectations when the gateway lags behind downstream route additions or removals (the route table is the single list to update) ## Observability evidence @@ -105,14 +100,12 @@ This capability provides the stable ingress boundary for mobile and backoffice c ### Operational signals - gateway logs and upstream timeout/failure traces -- admin readback from `/internal/admin/ai-engine/target` - route-aware rate-limiter classification in `api-gateway/src/app/services/rateLimiter.ts` ### Verification evidence - `api-gateway/src/tests/proxy.test.ts` - `api-gateway/src/tests/proxyRouteEquivalence.test.ts` -- `api-gateway/src/tests/routingStateStore.test.ts` - focused health-route tests in `api-gateway/src/tests/health.test.ts` ## Failure modes and risks @@ -120,13 +113,12 @@ This capability provides the stable ingress boundary for mobile and backoffice c | Failure mode | Likely cause | User/operator impact | Detection path | Recovery path | |---|---|---|---|---| | Forwarding failure | BFF unreachable, timeout, or broken proxy route | client-visible outage or edge 404/5xx | proxy tests, logs, smoke requests | restore upstream or add/fix the missing proxy route | -| Wrong AI target | stale persisted override or bad admin update | internal AI routes fail while public gateway stays healthy | `/internal/admin/ai-engine/target`, AI smoke checks | clear or replace the persisted override | +| Wrong AI target | `AI_ENGINE_API_URL` or `AI_ENGINE_STATS_URL` points at the wrong host | internal AI routes fail while public gateway stays healthy | AI smoke checks | fix the env value and redeploy | | Partial route exposure | downstream route exists but gateway route is missing | feature appears broken from public edge only | end-to-end probe through gateway | add the corresponding gateway proxy route and retest | ## Security and access model -- public edge access is separated from privileged admin routing control -- `/internal/admin/ai-engine/target` is protected by the gateway admin token path, not public client auth +- the gateway exposes no admin routing control; the runtime llama target is changed through the authenticated Backoffice surface and lives in `ai-engine-api` - internal AI proxy routes stay private to the platform and are not directly exposed on public client flows ## Deployment and rollback implications diff --git a/guides/capabilities/operations/runtime-routing-control.md b/guides/capabilities/operations/runtime-routing-control.md index 632b4d3..8a2a70b 100644 --- a/guides/capabilities/operations/runtime-routing-control.md +++ b/guides/capabilities/operations/runtime-routing-control.md @@ -64,7 +64,7 @@ Payload for `PUT` and `POST probe`: `host`, `protocol` (`http` or `https`), `por ## History -Service-target overrides, shared presets and connections, the Kubernetes overview, routing history, and the matching Backoffice tab and presets selector were removed (`bff-backoffice` PR #9, `Backoffice` PR #7). The gateway still carries a separate `/internal/admin/ai-engine/target` store and some dead forwarding entries; see the operations reference. +Service-target overrides, shared presets and connections, the Kubernetes overview, routing history, and the matching Backoffice tab and presets selector were removed (`bff-backoffice` PR #9, `Backoffice` PR #7). The matching dead forwarding entries and the gateway's own `/internal/admin/ai-engine/target` store were removed from `api-gateway` (PR #8). ## Related capabilities diff --git a/guides/communication/service-communication.md b/guides/communication/service-communication.md index 3bd8acc..26f5f5c 100644 --- a/guides/communication/service-communication.md +++ b/guides/communication/service-communication.md @@ -65,9 +65,8 @@ Only the llama target remains as a runtime control. See [Runtime routing and ser | POST | `/v1/backoffice/ai-engine/probe` | `bff-backoffice` | | GET | `/v1/backoffice/ai-engine/plugin/capabilities` | `bff-backoffice` | | GET/PUT/DELETE | `/internal/admin/llama-target` | `ai-engine-api` (private, called by the BFF) | -| GET/PUT/DELETE | `/internal/admin/ai-engine/target` | `api-gateway` (separate gateway target store, no longer used by the BFF; cleanup pending) | -History: service targets, ai-engine presets and connections, and the Kubernetes overview were removed from `bff-backoffice` and the Backoffice UI. +History: service targets, ai-engine presets and connections, and the Kubernetes overview were removed from `bff-backoffice`, the Backoffice UI and the `api-gateway` route table (the gateway's own `/internal/admin/ai-engine/target` store was removed as well). ## Asynchronous flow diff --git a/operations/functionality-review-baseline.md b/operations/functionality-review-baseline.md index 3422c2a..affa274 100644 --- a/operations/functionality-review-baseline.md +++ b/operations/functionality-review-baseline.md @@ -43,7 +43,7 @@ How easy it is to prove that the capability works through tests, smoke checks, r | Word-pass inventory and generation | `microservice-wordpass`, `ai-engine`, `runtime-seed` | BFF/backoffice/tooling -> word-pass service -> `ai-engine` or repair path | `generationService.test.ts`, `gamesRoutes.test.ts`, `modelGenerationJob.test.ts`, workflow `WF-05` | High | Medium | Low | Coverage verification is currently blocked; repair and post-generation maintenance are important but still weakly formalized for dry-run and audit evidence. | | AI content generation | `ai-engine`, `microservice-quizz`, `microservice-wordpass`, `api-gateway` | internal service or operator path -> `ai-engine-api` -> llama runtime | `ai-engine/src/tests/test_api/test_app.py`, `test_games/test_generator.py`, `test_api/test_llama_target_store.py`, large pytest tree | High | Medium | Medium | Strong unit coverage exists, but split-runtime state makes operational proof harder; true-false generation still lacks a clearly documented consumer path. | | AI observability and diagnostics | `ai-engine-stats`, `bff-backoffice`, `backoffice` | operator -> backoffice -> `bff-backoffice` -> ai stats and service probes | `backoffice` diagnostics panel tests, `bff-backoffice/src/tests/backoffice.test.ts`, `ai-engine` observability tests | Medium | Medium | Medium | Partial-data and degraded-data labeling rules are still not explicit enough; metric ownership across raw telemetry and UI panels remains diffuse. | -| Runtime routing control | `bff-backoffice`, `api-gateway`, `ai-engine` | operator -> backoffice -> `bff-backoffice` -> gateway / ai-engine | `routingStateStore.test.ts` in gateway and BFF, `test_llama_target_store.py`, workflow `WF-06` | Medium | Low | Medium | Three runtime state holders can all affect effective topology; audit-trail consistency and concurrent-edit semantics are not yet clear enough. | +| Runtime routing control | `bff-backoffice`, `api-gateway`, `ai-engine` | operator -> backoffice -> `bff-backoffice` -> gateway / ai-engine | `test_llama_target_store.py` and the BFF target tests, workflow `WF-06` | Medium | Low | Medium | The llama target state in `ai-engine-api` is the only runtime state holder affecting topology; audit-trail consistency and concurrent-edit semantics are not yet clear enough. | | Backoffice operational diagnostics | `backoffice`, `bff-backoffice`, `api-gateway` | operator UI -> gateway -> `bff-backoffice` -> services/k8s/AI | `service-overview-panel.integration.test.tsx`, `service-console-panel.integration.test.tsx`, `bff-backoffice/src/tests/backoffice.test.ts` | Medium | Medium | Medium | The UI is well covered locally, but there is still no single explicit smoke matrix proving the whole diagnostics chain through gateway, BFF, and downstream services. | | Contract governance | `contracts-and-schemas`, `shared-sdk-client` | schema change -> validation -> SDK generation -> consumer repos | `scripts/validate_contracts.py`, TypeScript generation builds, CI validations | Medium | Medium | Low | `shared-sdk-client/typescript` still has build-only verification and no test or coverage script; some effective contracts remain implicitly service-owned instead of centrally versioned. | | Deployment and rollout automation | `platform-infra`, covered runtime repos | service repo CI -> `platform-infra` build/push -> deploy | service CI workflows, `platform-infra` validation, smoke scripts such as `smoke-stg-edge.sh` | High | Medium | Medium | Covered services deploy well, but post-deploy smoke depth is uneven by capability and split-AI topologies complicate equivalent validation. | diff --git a/operations/runtime-routing-and-service-targeting.md b/operations/runtime-routing-and-service-targeting.md index d2a3949..fb999c7 100644 --- a/operations/runtime-routing-and-service-targeting.md +++ b/operations/runtime-routing-and-service-targeting.md @@ -46,9 +46,9 @@ The llama target is the host, port, and protocol that `ai-engine-api` uses to re - `PUT` is rejected unless the host matches `ALLOWED_ROUTING_TARGET_HOSTS` (exact host, `*.suffix`, or IPv4 CIDR). When the variable is empty, overrides are denied outside development. - The probe route performs a server-side request to an operator-supplied host; keep it on the authenticated backoffice surface. -## Gateway state (current, cleanup pending) +## Gateway -`api-gateway` still has a second, separate target store on `main`: `GET/PUT/DELETE /internal/admin/ai-engine/target` (guarded by `API_GATEWAY_ADMIN_TOKEN`, persisted in `GATEWAY_ROUTING_STATE_FILE`) that can redirect the `/internal/ai-engine/*` proxy. `bff-backoffice` no longer calls it, so the Backoffice flow above does not touch it. The gateway route table also still forwards a few paths that `bff-backoffice` no longer serves (`/kubernetes/overview`, `/ai-engine/presets*`, `/ai-engine/connections*`, `/service-targets*`); those requests end in an upstream error. No pull request that removes them is open in `AxiomNode/api-gateway` as of this date, so treat this as current behavior and an open cleanup item. +`api-gateway` holds no routing state: it resolves the `/internal/ai-engine/*` upstreams only from `AI_ENGINE_API_URL` and `AI_ENGINE_STATS_URL`, and its route table no longer lists the removed `bff-backoffice` paths (`/kubernetes/overview`, `/ai-engine/presets*`, `/ai-engine/connections*`, `/service-targets*`). Its former `/internal/admin/ai-engine/target` store, `API_GATEWAY_ADMIN_TOKEN` and `GATEWAY_ROUTING_STATE_FILE` were removed in `api-gateway` PR #8. ## Validation