Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .github/workflows/e2e_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,7 @@ jobs:
E2E_OPENAI_MODEL: ${{ vars.E2E_OPENAI_MODEL }}
E2E_DEPLOYMENT_MODE: ${{ matrix.mode }}
FAISS_VECTOR_STORE_ID: ${{ vars.FAISS_VECTOR_STORE_ID }}
# Override via repo Actions variable E2E_LLAMA_HOSTNAME; default matches server-mode lightspeed-stack.yaml
E2E_LLAMA_HOSTNAME: ${{ vars.E2E_LLAMA_HOSTNAME || 'llama-stack' }}
E2E_OGX_HOSTNAME: ogx

steps:
- uses: actions/checkout@v7
Expand Down Expand Up @@ -289,8 +288,8 @@ jobs:
echo "=== Test failure logs ==="

if [ "${{ matrix.mode }}" == "server" ]; then
echo "=== llama-stack logs ==="
docker compose logs llama-stack
echo "=== OGX logs ==="
docker compose logs ogx
echo ""
echo "=== lightspeed-stack logs ==="
docker compose logs lightspeed-stack
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/e2e_tests_providers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -355,8 +355,8 @@ jobs:
echo "=== Test failure logs ==="

if [ "${{ matrix.mode }}" == "server" ]; then
echo "=== llama-stack logs ==="
docker compose logs llama-stack
echo "=== OGX logs ==="
docker compose logs ogx
echo ""
echo "=== lightspeed-stack logs ==="
docker compose logs lightspeed-stack
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/e2e_tests_rhaiis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -249,8 +249,8 @@ jobs:
echo "=== Test failure logs ==="

if [ "${{ matrix.mode }}" == "server" ]; then
echo "=== llama-stack logs ==="
docker compose logs llama-stack
echo "=== OGX logs ==="
docker compose logs ogx
echo ""
echo "=== lightspeed-stack logs ==="
docker compose logs lightspeed-stack
Expand Down
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ dev/
# VSCode
.vscode/

# Llama related - when running the stack as lib client
# OGX related - when running the stack as lib client
.llama

# Database files
Expand All @@ -195,7 +195,7 @@ requirements.*.backup
# Local run files
local-run.yaml

# Synthesized Llama Stack run.yaml written by unified library mode (LCORE-2336)
# Synthesized OGX run.yaml written by unified library mode (LCORE-2336)
.generated/

# Sisyphus planning files
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
default: '{"components": [{"name":"lightspeed-stack-0-8", "containerImage": "quay.io/example/lightspeed-stack-0-8:latest"}]}'
type: string
- name: llama-stack-image
description: 'Llama Stack runs from source on UBI (init container clones repo and installs deps). Kept for logging/backwards compatibility.'
description: 'OGX runs from source on UBI (init container clones repo and installs deps). Parameter name kept for backwards compatibility.'
default: 'run-from-source (UBI)'
type: string
- name: test-name
Expand Down Expand Up @@ -81,7 +81,7 @@ spec:
- name: instanceType
value: "m5.large"
- name: get-stack-images
description: Extract lightspeed-stack image and commit from SNAPSHOT (Llama Stack runs from source in-pod)
description: Extract lightspeed-stack image and commit from SNAPSHOT (OGX runs from source in-pod)
runAfter:
- provision-cluster
params:
Expand Down
23 changes: 12 additions & 11 deletions deploy/ogx/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Llama Stack container image
# OGX container image

`test.containerfile` builds the Llama Stack server image used by
`docker-compose.yaml` (server mode, e.g. for the e2e suite). Besides the
Llama Stack distribution itself, the image bundles the pieces needed to
generate its run configuration at container start:
`test.containerfile` builds the OGX server image used by `docker-compose.yaml`
(server mode, e.g. for the e2e suite). Besides the OGX distribution itself, the
image bundles the pieces needed to generate its run configuration at container
start:

- `/opt/app-root/ogx_configuration.py` — the config-generation
script (copied from `src/ogx_configuration.py`).
Expand All @@ -21,19 +21,20 @@ shape:

- **Unified mode** — the `lightspeed-stack.yaml` carries a *synthesis
input* (a non-empty `inference.providers` or `vector_store.providers`,
or a `llama_stack.config` block). The full `run.yaml` is synthesized
from it; no external `run.yaml` mount is needed.
or an `ogx.config` block). The full `run.yaml` is synthesized from it;
no external `run.yaml` mount is needed.
- **Legacy mode** — no synthesis input present. The mounted `run.yaml`
(`$LLAMA_STACK_CONFIG`, default `/opt/app-root/run.yaml`) is enriched
with lightspeed dynamic values (BYOK RAG, Solr/OKP, Azure Entra ID).
(`$OGX_CONFIG`, with deprecated fallback `$LLAMA_STACK_CONFIG`, default
`/opt/app-root/run.yaml`) is enriched with lightspeed dynamic values
(BYOK RAG, Solr/OKP, Azure Entra ID).

The repository `docker-compose.yaml` mounts both files and works for
either mode — with a unified `lightspeed-stack.yaml` the `run.yaml`
mount is simply ignored. A unified-only deployment needs just:

```yaml
services:
llama-stack:
ogx:
build:
context: .
dockerfile: deploy/ogx/test.containerfile
Expand All @@ -48,5 +49,5 @@ services:
The compose file also mounts host copies of the script, the baseline
data directory, and the entrypoint over their baked-in counterparts, so
`docker compose up` picks up local changes to any of them without an
image rebuild. Rebuild (`docker compose build llama-stack`) when
image rebuild. Rebuild (`docker compose build ogx`) when
dependencies (`pyproject.toml` / `uv.lock`) or the providers change.
22 changes: 11 additions & 11 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
services:
# Red Hat OGX distribution with FAISS
llama-stack:
ogx:
build:
context: .
dockerfile: deploy/ogx/test.containerfile
platform: linux/amd64
container_name: llama-stack
container_name: ogx
ports:
- "8321:8321" # Expose OGX on 8321 (adjust if needed)
depends_on:
Expand All @@ -14,7 +14,7 @@ services:
volumes:
# Used in legacy mode only: with a unified lightspeed-stack.yaml (one
# carrying inference.providers / vector_store.providers / a
# llama_stack.config block) the entrypoint synthesizes the run config
# ogx.config block) the entrypoint synthesizes the run config
# from lightspeed-stack.yaml and this mount is ignored
- ./run.yaml:/opt/app-root/run.yaml:z
# Host copies so `docker compose up` picks up script changes without rebuilding OGX
Expand All @@ -24,7 +24,7 @@ services:
- ${GCP_KEYS_PATH:-./tmp/.gcp-keys-dummy}:/opt/app-root/.gcp-keys:ro
- ./lightspeed-stack.yaml:/opt/app-root/lightspeed-stack.yaml:ro,z
# Writable OGX storage (rag/ work copy is re-seeded by entrypoint each start)
- llama-storage:/opt/app-root/src/.llama/storage
- ogx-storage:/opt/app-root/src/.llama/storage
Comment thread
asimurka marked this conversation as resolved.
# Read-only e2e FAISS fixtures — never mount these as the live KV_RAG_PATH
- ./tests/e2e/rag:/opt/app-root/src/.llama/storage/.e2e-rag-seed:ro,z
- mock-tls-certs:/certs:ro
Expand Down Expand Up @@ -80,8 +80,8 @@ services:
- SOLR_CONTENT_FIELD=${SOLR_CONTENT_FIELD:-}
- SOLR_EMBEDDING_MODEL=${SOLR_EMBEDDING_MODEL:-}
- SOLR_EMBEDDING_DIM=${SOLR_EMBEDDING_DIM:-}
#required for enrichment
- E2E_LLAMA_HOSTNAME=${E2E_LLAMA_HOSTNAME:-llama-stack}
# Required for enrichment; E2E_LLAMA_HOSTNAME is the external/GitHub Actions name
- E2E_OGX_HOSTNAME=${E2E_OGX_HOSTNAME:-${E2E_LLAMA_HOSTNAME:-ogx}}
networks:
- lightspeednet
healthcheck:
Expand Down Expand Up @@ -111,17 +111,17 @@ services:
- CLIENT_SECRET=${CLIENT_SECRET:-}
# FAISS vector store ID (used by inline RAG config)
- FAISS_VECTOR_STORE_ID=${FAISS_VECTOR_STORE_ID:-}
# Substituted in mounted lightspeed-stack.yaml (llama_stack.url); GitHub Actions sets via vars
- E2E_LLAMA_HOSTNAME=${E2E_LLAMA_HOSTNAME:-llama-stack}
- E2E_LLAMA_PORT=${E2E_LLAMA_PORT:-8321}
# Substituted in mounted lightspeed-stack.yaml (ogx.url); GitHub Actions sets E2E_LLAMA_HOSTNAME
- E2E_OGX_HOSTNAME=${E2E_OGX_HOSTNAME:-${E2E_LLAMA_HOSTNAME:-ogx}}
- E2E_OGX_PORT=${E2E_OGX_PORT:-${E2E_LLAMA_PORT:-8321}}
# OpenTelemetry configuration (tracing disabled by default)
- OTEL_EXPORTER_OTLP_ENDPOINT=${OTEL_EXPORTER_OTLP_ENDPOINT:-}
- OTEL_EXPORTER_OTLP_PROTOCOL=${OTEL_EXPORTER_OTLP_PROTOCOL:-}
- OTEL_SERVICE_NAME=${OTEL_SERVICE_NAME:-}
- OTEL_ANONYMIZATION_SECRET=${OTEL_ANONYMIZATION_SECRET:-lightspeed-stack-otel-anonymization-dev-default}
- OTEL_SDK_DISABLED=${OTEL_SDK_DISABLED:-true}
depends_on:
llama-stack:
ogx:
condition: service_healthy
mock-mcp:
condition: service_healthy
Expand Down Expand Up @@ -186,7 +186,7 @@ services:


volumes:
llama-storage:
ogx-storage:
mock-tls-certs:

networks:
Expand Down
8 changes: 4 additions & 4 deletions docs/design/ogx-config-merge/ogx-config-merge.md
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,7 @@ None at the REST API surface. Internal API additions in
- `migrate_config_dumb(run_yaml_path, lightspeed_yaml_path, output_path)
-> None` — dumb-mode migration (lossless round-trip).
- `deep_merge_list_replace(base, overlay) -> dict` — merge helper.
- `apply_high_level_inference(ls_config, inference)` — high-level expansion.
- `apply_high_level_inference(ogx_config, inference)` — high-level expansion.
- `load_default_baseline() -> dict` — loads `src/data/default_run.yaml`.

CLI additions in `src/lightspeed_stack.py`:
Expand Down Expand Up @@ -474,11 +474,11 @@ not have had a full release with a working migration path. Releases:
3. Run `dedupe_providers_vector_io` on the baseline.
4. Apply existing enrichment: `enrich_byok_rag`, `enrich_solr` (Azure
Entra ID intentionally stays separate because it's a `.env`
side-effect, not an `ls_config` mutation).
side-effect, not an `ogx_config` mutation).
5. If top-level `inference.providers` is non-empty →
`apply_high_level_inference(ls_config, lcs_config["inference"])`.
`apply_high_level_inference(ogx_config, lcs_config["inference"])`.
6. If `unified` and `unified.native_override` non-empty →
`deep_merge_list_replace(ls_config, native_override)`.
`deep_merge_list_replace(ogx_config, native_override)`.
7. `dedupe_providers_vector_io` again for good measure.
8. Return the final dict.

Expand Down
10 changes: 5 additions & 5 deletions docs/devel_doc/conversations_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,17 +115,17 @@ When a user makes a query **without** providing a `conversation_id`:
```python
# No conversation_id provided - create a new conversation first
conversation = await client.conversations.create(metadata={})
llama_stack_conv_id = conversation.id
ogx_conv_id = conversation.id
# Store the normalized version
conversation_id = normalize_conversation_id(llama_stack_conv_id)
conversation_id = normalize_conversation_id(ogx_conv_id)

# Use the conversation in responses.create()
response = await client.responses.create(
input=input_text,
model=model_id,
instructions=system_prompt,
store=True,
conversation=llama_stack_conv_id, # Use OGX format
conversation=ogx_conv_id, # Use OGX format
# ... other parameters
)
```
Expand All @@ -151,7 +151,7 @@ ogx_conv_id = to_ogx_conversation_id(conversation_id)
response = await client.responses.create(
input=input_text,
model=model_id,
conversation=llama_stack_conv_id, # Existing conversation
conversation=ogx_conv_id, # Existing conversation
# ... other parameters
)
```
Expand All @@ -166,7 +166,7 @@ Conversations are stored in **two databases**:
- `openai_conversations`: Stores conversation metadata
- `conversation_items`: Stores individual messages/turns in conversations

**Configuration (in `config/llama_stack_client_config.yaml`):**
**Configuration (in OGX `run.yaml` / library client config):**
```yaml
storage:
stores:
Expand Down
8 changes: 4 additions & 4 deletions docs/testing/e2e_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,10 +142,10 @@ uv run behave tests/e2e/features/health.feature --tags=-skip-in-library-mode
| `E2E_DEPLOYMENT_MODE` | `server` | `server` or `library`. Drives config paths and which scenarios run (e.g. `@skip-in-library-mode`). |
| `E2E_LSC_HOSTNAME` | `localhost` | Host of the Lightspeed Core Stack API. |
| `E2E_LSC_PORT` | `8080` | Port of the Lightspeed Core Stack API. |
| `E2E_LLAMA_HOSTNAME` | `localhost` | Host of the OGX service (server mode). |
| `E2E_LLAMA_PORT` | `8321` | Port of the OGX service. |
| `E2E_LLAMA_STACK_URL` | — | Full base URL for OGX (overrides host/port if set). Used by shield helpers. |
| `E2E_LLAMA_STACK_API_KEY` | `xyzzy` | API key for OGX client (e.g. shield API). |
| `E2E_OGX_HOSTNAME` | `localhost` | Host of the OGX service (server mode). GitHub Actions repo variable: `E2E_LLAMA_HOSTNAME` (compose maps it here). |
| `E2E_OGX_PORT` | `8321` | Port of the OGX service. |
| `E2E_OGX_STACK_URL` | — | Full base URL for OGX (overrides host/port if set). Used by shield helpers. |
| `E2E_OGX_STACK_API_KEY` | `xyzzy` | API key for OGX client (e.g. shield API). |
| `E2E_DEFAULT_MODEL_OVERRIDE` | — | Override default LLM model id (e.g. `gpt-4o-mini`). |
| `E2E_DEFAULT_PROVIDER_OVERRIDE` | — | Override default provider id (e.g. `openai`). |
| `FAISS_VECTOR_STORE_ID` | — | Vector store id for FAISS-related scenarios. |
Expand Down
8 changes: 4 additions & 4 deletions lightspeed-stack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ service:
workers: 1
color_log: true
access_log: true
# llama_stack configuration
# ogx configuration
# When using 'make run', a container is ALWAYS launched at http://localhost:8321 (hardcoded in Makefile).
# This llama_stack section controls where lightspeed-core connects to OGX.
# To use a different port: override with 'make run LLAMA_STACK_PORT=<port>' and update the url below,
# or run ogx manually and don't use 'make run'.
# This ogx section controls where lightspeed-core connects to OGX.
# To use a different port: override with 'make run OGX_PORT=<port>' and update the url below,
# or run OGX manually and don't use 'make run'.
ogx:
use_as_library_client: false
url: http://localhost:8321
Expand Down
8 changes: 4 additions & 4 deletions scripts/ogx-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@
# configuration shape:
# - unified mode: the lightspeed config carries a synthesis input (a
# non-empty inference.providers or vector_store.providers, or a
# llama_stack.config / ogx.config block). The full run.yaml is synthesized
# ogx.config block (deprecated alias: llama_stack.config). The full run.yaml is synthesized
# from it — no external run.yaml mount is needed, and $OGX_CONFIG /
# $LLAMA_STACK_CONFIG is ignored. The shipped default baseline is read from
# $LLAMA_STACK_CONFIG (deprecated) is ignored. The shipped default baseline is read from
# /opt/app-root/data/default_run.yaml.
# - legacy mode: the mounted run.yaml ($OGX_CONFIG, falling back to
# $LLAMA_STACK_CONFIG) is enriched with lightspeed dynamic values
# - legacy mode: the mounted run.yaml ($OGX_CONFIG, falling back to the
# deprecated $LLAMA_STACK_CONFIG) is enriched with lightspeed dynamic values
# (BYOK RAG, Solr/OKP, Azure Entra ID).

set -e
Expand Down
10 changes: 5 additions & 5 deletions src/client/ogx.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,34 +159,34 @@ def _enrich_library_config(self, input_config_path: str) -> str:
"""Enrich OGX config with BYOK RAG and OKP Solr settings."""
try:
with open(input_config_path, encoding="utf-8") as f:
ls_config = yaml.safe_load(f)
ogx_config = yaml.safe_load(f)
except (OSError, yaml.YAMLError) as e:
logger.warning("Failed to read OGX config: %s", e)
return input_config_path

config = configuration.configuration

# Enrichment: BYOK RAG
enrich_byok_rag(ls_config, [s.model_dump() for s in config.rag.byok.stores])
enrich_byok_rag(ogx_config, [s.model_dump() for s in config.rag.byok.stores])

# Enrichment: Solr - enabled when "okp" appears in either inline or tool list
rag_config_for_solr = {
"inline": config.rag.retrieval.inline.sources,
"tool": config.rag.retrieval.tool.sources,
}
enrich_solr(ls_config, rag_config_for_solr, config.rag.okp.model_dump())
enrich_solr(ogx_config, rag_config_for_solr, config.rag.okp.model_dump())

# Enrichment: Azure Entra ID deferred auth
entra_id_config = (
config.azure_entra_id.model_dump() if config.azure_entra_id else None
)
enrich_azure_entra_id_inference(ls_config, entra_id_config)
enrich_azure_entra_id_inference(ogx_config, entra_id_config)

enriched_path = os.path.join(tempfile.gettempdir(), "ogx_enriched_config.yaml")

try:
with open(enriched_path, "w", encoding="utf-8") as f:
yaml.dump(ls_config, f, Dumper=YamlDumper, default_flow_style=False)
yaml.dump(ogx_config, f, Dumper=YamlDumper, default_flow_style=False)
logger.info("Wrote enriched OGX config to %s", enriched_path)
return enriched_path
except OSError as e:
Expand Down
10 changes: 5 additions & 5 deletions src/configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,14 +73,14 @@ def replace_env_vars_preserving_native_override(
ogx_section = config_dict.get("ogx")
if ogx_section is None:
ogx_section = config_dict.get("llama_stack")
ls_config = ogx_section.get("config") if isinstance(ogx_section, dict) else None
if not (isinstance(ls_config, dict) and "native_override" in ls_config):
ogx_config = ogx_section.get("config") if isinstance(ogx_section, dict) else None
if not (isinstance(ogx_config, dict) and "native_override" in ogx_config):
return replace_env_vars(config_dict)

raw_override = ls_config["native_override"]
ls_config["native_override"] = {} # keep secrets out of env resolution
raw_override = ogx_config["native_override"]
ogx_config["native_override"] = {} # keep secrets out of env resolution
resolved = replace_env_vars(config_dict)
ls_config["native_override"] = raw_override # restore source dict if reused
ogx_config["native_override"] = raw_override # restore source dict if reused
resolved_ogx = (resolved.get("ogx") or resolved.get("llama_stack") or {}).get(
"config"
)
Expand Down
Loading
Loading