Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Central architecture and operations documentation for AxiomNode.

## CI/CD notes
- Docs now have a markdown link validation workflow; keep relative links resolvable.
- Docs should track real workflow behavior (service CI -> platform-infra -> stg deploy by default, prod by manual promotion).
- Docs should track real workflow behavior (service CI dispatches the `platform-infra` image build on merge; deploys are manual-only via `deploy.yaml` `workflow_dispatch`, and no staging host exists today).

## LLM editing rules
- Keep content concise, actionable, and in English.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Backoffice access is role-gated through `microservice-users`. The current role m
### Current runtime reality to document explicitly

- The deployed Kubernetes topology and the effective runtime topology are not always identical.
- `api-gateway`, `bff-backoffice`, and `ai-engine-api` can all hold persisted runtime targeting state.
- Only `ai-engine-api` holds persisted runtime targeting state (the llama target); `api-gateway` and `bff-backoffice` hold none.
- Staging may run `ai-engine-api` and `ai-engine-stats` in-cluster while the llama runtime remains external, or it may render the optional full in-cluster AI overlay for controlled diagnostics.
- Documentation must distinguish environment defaults, persisted overrides, and live effective targets.

Expand Down
18 changes: 8 additions & 10 deletions architecture/c4-views.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# C4 Views

Last updated: 2026-04-22.
Last updated: 2026-11-14.

This document captures the AxiomNode platform using the C4 model: System Context (Level 1), Containers (Level 2), and Components (Level 3) for the most relevant containers. Level 4 (code) is intentionally omitted; it lives in each repository.

Expand All @@ -25,7 +25,7 @@ Rel(player, axiom, "Plays games / consults stats", "HTTPS")
Rel(operator, axiom, "Operates and monitors", "HTTPS")
Rel(platformEng, github, "Pushes code, dispatches builds")
Rel(github, ghcr, "Publishes images")
Rel(github, axiom, "Deploys to k3s")
Rel(github, axiom, "Manual deploy (workflow_dispatch; no staging host today)")
Rel(axiom, llama, "Generation requests", "HTTP")
Rel(player, firebase, "Auth / push", "HTTPS")
```
Expand All @@ -40,7 +40,7 @@ Person(player, "Mobile Player")
Person(operator, "Backoffice Operator")

System_Boundary(edge, "Edge") {
Container(gateway, "api-gateway", "Node/Express", "Single public ingress, auth, CORS, routing, persists active ai-engine target")
Container(gateway, "api-gateway", "Node/Express", "Single public ingress, auth, CORS, routing; stateless (no routing state)")
}

System_Boundary(experience, "Experience / BFF") {
Expand Down Expand Up @@ -78,15 +78,15 @@ Rel(operator, spa, "HTTPS")
Rel(spa, gateway, "HTTPS")
Rel(gateway, bffMobile, "HTTP")
Rel(gateway, bffBack, "HTTP")
Rel(gateway, aiApi, "HTTP (live target)")
Rel(gateway, aiApi, "HTTP (/internal/ai-engine, upstream from env)")
Rel(bffMobile, quiz, "HTTP")
Rel(bffMobile, wp, "HTTP")
Rel(bffMobile, users, "HTTP")
Rel(bffBack, users, "HTTP")
Rel(bffBack, quiz, "HTTP")
Rel(bffBack, wp, "HTTP")
Rel(bffBack, aiStats, "HTTP")
Rel(bffBack, gateway, "HTTP (control plane mutations)")
Rel(bffBack, aiApi, "HTTP (llama target, plugin capabilities)")
Rel(quiz, gateway, "HTTP (generation, /internal/ai-engine)")
Rel(wp, gateway, "HTTP (generation, /internal/ai-engine)")
Rel(aiApi, llama, "HTTP")
Expand All @@ -108,14 +108,12 @@ Rel(sdk, bffBack, "imports")
- `RouteRegistry` — declarative route map, version-aware.
- `Forwarder` (`proxy` SDK module) — `forwardHttp` with structured upstream errors.
- `ContractGuards` — Zod-based validation for shared public query contracts.
- `RuntimeTargetStore` — persists active ai-engine API/stats target on mounted PV.
- `EdgePolicy` — auth, CORS, rate-limit hooks.

### bff-backoffice

- `OperatorAuth` — session and role resolution against `microservice-users`.
- `ServiceTargetOverrideStore` — persists per-route upstream overrides.
- `AIPresetStore` — reusable ai-engine destination presets.
- `LlamaTargetControl` — proxies llama target read/set/reset/probe to `ai-engine-api` `/internal/admin/llama-target`; keeps no routing state.
- `MetricsAggregator` — adapts upstream metrics caches with normalized TTL keys.
- `RowsQueryEngine` — UI-side filter with precomputed lowercase row strings.

Expand All @@ -124,7 +122,7 @@ Rel(sdk, bffBack, "imports")
- `RAGRetriever` — multilingual MiniLM embeddings + metadata-aware oversample/rerank.
- `Optimizer` — query enrichment with category name, model, corpus signature.
- `CacheManager` — keyed by category, embedding model, corpus signature.
- `LlamaClient` — points to active llama target (persisted).
- `LlamaClient` — points to active llama target (persisted in `AI_ENGINE_LLAMA_TARGET_STATE_FILE`).
- `Diagnostics` — structured counters surfaced via FastAPI hooks.

### microservice-quizz / wordpass
Expand All @@ -137,7 +135,7 @@ Rel(sdk, bffBack, "imports")
### platform-infra

- `BuildPushWorkflow` — receives dispatch, builds, publishes `:sha`/`:stg`/`:prod`.
- `DeployWorkflow` — `kubectl apply -k overlays/{env}` with smoke checks.
- `DeployWorkflow` — manual-only (`workflow_dispatch`): `kubectl apply -k overlays/{env}` with smoke checks; no staging host exists today.
- `Overlays` — `dev`, `stg`, `prod` (+ optional full-AI overlay).
- `SealedSecrets` — per-overlay sealed material.

Expand Down
2 changes: 2 additions & 0 deletions architecture/decision-records/0001-single-public-edge.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ Run a single public ingress, `api-gateway`, as the only externally reachable HTT
- `api-gateway` becomes a routing-state holder (live AI target).
- Operational dependency: gateway downtime equals platform downtime; mitigated by k3s replicas and probes.

> **Superseded in part (2026-11-14).** The gateway no longer holds a live AI target: its persisted routing state was removed (`api-gateway` PR #8) and it resolves ai-engine upstreams from `AI_ENGINE_API_URL` and `AI_ENGINE_STATS_URL`. See [Runtime routing and service targeting](../../operations/runtime-routing-and-service-targeting.md).

## Related

- [Repository map](../repository-map.md)
Expand Down
2 changes: 2 additions & 0 deletions architecture/decision-records/0005-runtime-control-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
- Status: Accepted
- Date: 2026-04-22

> **Superseded in part (2026-11-14).** Only the third bullet of the decision still holds: `ai-engine-api` persists the active llama target, and operators change it through `bff-backoffice`. The `bff-backoffice` service-target overrides and shared presets and the `api-gateway` persisted ai-engine target were removed (`bff-backoffice` PR #9, `Backoffice` PR #7, `api-gateway` PR #8); the gateway and the BFF now hold no routing state. The text below is kept as the original record. Current behavior: [Runtime routing and service targeting](../../operations/runtime-routing-and-service-targeting.md).

## Context

We need to change AI upstream routing without redeploying the platform (e.g., point `api-gateway` to a different `ai-engine-api`, or `ai-engine-api` to a different llama target). GitOps alone forces a build/deploy cycle that is too slow for AI recovery scenarios.
Expand Down
21 changes: 10 additions & 11 deletions architecture/domain-model.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Domain Model

Last updated: 2026-04-22.
Last updated: 2026-11-14.

High-level domain model of the AxiomNode platform. Each bounded context is owned by exactly one microservice; cross-context references travel by ID and never by foreign-key joins across databases.

Expand All @@ -12,7 +12,7 @@ High-level domain model of the AxiomNode platform. Each bounded context is owned
| Quiz Gameplay | `microservice-quizz` | PostgreSQL (quizz DB) | Indirect |
| Wordpass Gameplay | `microservice-wordpass` | PostgreSQL (wordpass DB) | Indirect |
| AI Generation | `ai-engine` | Redis cache + vector store + on-disk corpus | Internal only |
| Runtime Routing State | `api-gateway`, `bff-backoffice`, `ai-engine-api` | Mounted PV (per service) | Operator only |
| Runtime Routing State (llama target) | `ai-engine-api` | Mounted PV | Operator only |
| Contracts | `contracts-and-schemas` | Git | n/a |

## Entities (logical view)
Expand Down Expand Up @@ -70,11 +70,9 @@ classDiagram
+latencyMs: int
+cacheHit: bool
}
class RoutingTarget {
+scope: enum (gateway-ai, bff-override, ai-llama)
+targetUrl: string
+presetId: string?
+updatedBy: string
class LlamaTarget {
+url: string
+label: string?
+updatedAt: datetime
}

Expand Down Expand Up @@ -113,12 +111,13 @@ classDiagram
- `cacheKey` uniquely identifies a generation request shape.
- `corpusSignature` change invalidates dependent cached entries logically.

### RoutingTarget (control-plane state, distributed)
### LlamaTarget (control-plane state, owned by `ai-engine-api`)

- Root: `RoutingTarget` keyed by `scope`.
- Root: `LlamaTarget`, persisted as a single JSON file (`AI_ENGINE_LLAMA_TARGET_STATE_FILE`, fields `url`, `label`, `updatedAt`).
- Invariants:
- At most one active target per scope per service instance share the mounted PV.
- Drift between scopes is allowed but observable (operators can see effective vs declared).
- At most one active override; deleting it falls back to the environment URL.
- The override can differ from the declared manifests; operators read the effective value through `GET /v1/backoffice/ai-engine/target`.
- The gateway and `bff-backoffice` keep no routing state (service-target overrides and presets were removed).

## Cross-context contracts

Expand Down
4 changes: 2 additions & 2 deletions architecture/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Common terms used across AxiomNode documentation.
| Full in-cluster AI | Optional overlay that runs llama inside k3s for diagnostics. |
| Effective topology | The routing actually in use, including persisted runtime overrides. |
| Declared topology | The topology described by Kubernetes manifests in `platform-infra`. |
| Routing target | Persisted runtime URL for an upstream (gateway-ai, bff-override, ai-llama). |
| Control plane (runtime) | Backoffice-driven set of routing mutations that survive pod restart. |
| Routing target | Persisted runtime URL for the ai-engine llama upstream (the only runtime routing target). |
| Control plane (runtime) | Backoffice-driven llama target mutation (set, reset) that survives pod restart. |
| Cache key | Composite key `(category_id, embedding_model, corpus_signature, request_shape)` used by `ai-engine-api`. |
| Corpus signature | Stable hash of the curated corpus used for retrieval; participates in cache keys. |
| GHCR | GitHub Container Registry, where service images are published. |
Expand Down
19 changes: 9 additions & 10 deletions architecture/repository-map.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Repository Map

Last updated: 2026-04-19.
Last updated: 2026-11-14.

## Runtime repositories

Expand All @@ -19,8 +19,8 @@ Last updated: 2026-04-19.

| Repository | Persisted runtime state | Why it matters |
|---|---|---|
| `api-gateway` | Active ai-engine API/stats target | Determines where live AI traffic is forwarded |
| `bff-backoffice` | Service target overrides and shared ai-engine presets | Shapes operator-visible effective topology |
| `api-gateway` | None (upstreams come from `AI_ENGINE_API_URL` and `AI_ENGINE_STATS_URL`) | Stateless edge; no routing store |
| `bff-backoffice` | None (only an audit trail of admin actions) | Proxies the llama target control to `ai-engine-api` |
| `ai-engine` | Active llama target inside `ai-engine-api` | Controls where generation calls reach the model runtime |

## Client and operator entry points
Expand All @@ -30,7 +30,7 @@ Last updated: 2026-04-19.
| Mobile app | `api-gateway` | `api-gateway` -> `bff-mobile` -> quiz/word-pass services |
| Backoffice SPA | `api-gateway` | `api-gateway` -> `bff-backoffice` -> users/services/ai diagnostics |
| Internal AI producers | service-local clients | quiz/word-pass -> `ai-engine-api` |
| Operators changing AI routing | `backoffice` through `bff-backoffice` | backoffice -> `bff-backoffice` -> `api-gateway` / `ai-engine-api` |
| Operators changing the llama target | `backoffice` through `bff-backoffice` | backoffice -> `api-gateway` -> `bff-backoffice` -> `ai-engine-api` |

## Delivery-chain classification

Expand All @@ -48,9 +48,9 @@ These repositories participate in the current automatic image publication and st
- `microservice-quizz`
- `microservice-wordpass`

### Outside the current automatic staging chain
### Outside the image-build chain

- `mobile-app`: validated in its own CI, but not published as a k3s runtime image through `platform-infra`
- `mobile-app`: validated in its own CI, but not published as a runtime image through `platform-infra`
- external llama runtime: still operationally external in the split staging topology unless the optional full in-cluster AI overlay is rendered deliberately

## Platform repositories
Expand Down Expand Up @@ -79,10 +79,9 @@ For AI-related flows, the effective runtime topology may differ from the deploye

Reason:

- `api-gateway` can hold the active ai-engine target at runtime
- `bff-backoffice` can persist shared ai-engine presets and service-target overrides
- `ai-engine-api` can hold the active llama target at runtime
- staging can therefore operate in split-AI mode even while the rest of the services remain auto-deployed on k3s
- `ai-engine-api` can hold the active llama target at runtime (the only runtime routing state)
- `api-gateway` and `bff-backoffice` hold none
- a deployment can therefore run in split-AI mode, with the llama runtime outside the cluster, even though the manifests describe in-cluster services

## Current CI/CD automation map

Expand Down
28 changes: 13 additions & 15 deletions architecture/target-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,24 +143,22 @@ That distinction is important: deployment topology and effective runtime topolog

The current architecture contains a narrow operational state plane outside static manifests:

- `bff-backoffice` persists service-target overrides and shared ai-engine presets
- `api-gateway` persists the live ai-engine API and stats target used by forwarded runtime traffic
- `ai-engine-api` persists the active llama target used by generation requests
- `ai-engine-api` persists the active llama target used by generation requests (`AI_ENGINE_LLAMA_TARGET_STATE_FILE`)

These states survive pod recreation through mounted storage and must be documented as part of the effective topology.
`api-gateway` and `bff-backoffice` hold no routing state: the gateway resolves ai-engine upstreams from `AI_ENGINE_API_URL` and `AI_ENGINE_STATS_URL`. Service-target overrides, shared presets and connections, and the gateway live target were removed.

The llama target survives pod recreation through mounted storage and must be documented as part of the effective topology.

### Runtime routing control plane

Backoffice operational tooling now acts as a lightweight runtime control plane for non-static upstream selection.
Backoffice operational tooling acts as a lightweight runtime control plane for exactly one setting, the `ai-engine-api` llama target.

Current capabilities:

- `bff-backoffice` can persist shared upstream routing overrides
- `api-gateway` can persist the live `ai-engine` target used by the platform
- backoffice operators can select and manage reusable ai-engine destination presets
- routing state survives pod recreation because it is stored on mounted persistent volumes
- backoffice operators can read, probe, set and reset the llama target through `bff-backoffice` (`GET/PUT/DELETE /v1/backoffice/ai-engine/target`, `POST /v1/backoffice/ai-engine/probe`)
- `ai-engine-api` persists the override on a mounted volume, so it survives pod recreation

This control-plane behavior is intentionally narrow in scope: it does not replace GitOps or declarative infrastructure, but it provides pragmatic runtime flexibility for operational AI connectivity.
This control-plane behavior is intentionally narrow in scope: it does not replace GitOps or declarative infrastructure. See [Runtime routing and service targeting](../operations/runtime-routing-and-service-targeting.md).

### Delivery architecture

Expand Down Expand Up @@ -196,13 +194,13 @@ This preserves local service autonomy without fragmenting the deployment contrac

### Traceability

- automatic staging deploys are tied to immutable image SHAs
- images are tagged with immutable short SHAs; deploys are manual-only, so the tag a deploy uses is chosen by the operator
- service CI, central packaging, and deployment workflows form a clear audit chain

### Operability

- smoke tests and rollout checks are embedded in deployment workflows
- runtime overrides for AI connectivity can be changed without rebuilding or redeploying the entire stack
- the llama target can be changed without rebuilding or redeploying the stack

### Evolvability

Expand All @@ -211,9 +209,9 @@ This preserves local service autonomy without fragmenting the deployment contrac

### Current architectural constraints

- staging automatic deploys are pinned to immutable short-SHA artifacts, but manual recovery paths can still use mutable environment tags
- runtime overrides introduce a second operational state plane that must be documented and observed
- not every repository in the workspace is part of the GHCR-to-k3s delivery chain
- deploys are manual-only (`deploy.yaml` `workflow_dispatch`), so image tags are chosen by the operator; the thesis-era automatic path pinned immutable short-SHA artifacts
- the llama target override introduces a second operational state plane that must be documented and observed
- not every repository in the workspace is part of the GHCR image-build chain
- AI runtime behavior in staging must always be described in terms of both in-cluster services and external llama reachability

### Documentation implications
Expand Down
6 changes: 3 additions & 3 deletions architecture/use-cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,10 @@ This document enumerates the primary use cases supported by the AxiomNode platfo

### UC-09 Change runtime AI target (Backoffice Operator)

- Trigger: Operator selects a new ai-engine-api/stats target or llama upstream from the backoffice control plane.
- Path: `backoffice -> bff-backoffice -> api-gateway` (persists active target on PV) and/or `bff-backoffice -> ai-engine-api` (persists active llama target).
- Trigger: Operator selects a new llama upstream from the backoffice control plane.
- Path: `backoffice -> api-gateway -> bff-backoffice -> ai-engine-api` (persists active llama target on a mounted volume).
- Post-conditions: Effective routing diverges from declarative manifests; persisted on mounted volume; survives pod recreation.
- Risk: state drift versus manifests; must be observable via stats endpoints and runbook.
- Risk: state drift versus manifests; must be observable via `GET /v1/backoffice/ai-engine/target` and the runbook.

### UC-10 Generate quiz/wordpass via internal service (Internal Service)

Expand Down
Loading
Loading