LCORE-1675: Documentation for conversation compaction

docs/README.md

@@ -29,6 +29,8 @@ See the full documentation at [`../README.md`](../README.md) or browse sub-pages
 
 [OKP guide](https://lightspeed-core.github.io/lightspeed-stack/user_doc/okp_guide.html)
 
+[Conversation compaction](https://lightspeed-core.github.io/lightspeed-stack/user_doc/conversation_compaction.html)
+
 [Authentication and Authorization](https://lightspeed-core.github.io/lightspeed-stack/user_doc/auth.html)
 
 [User data collection](https://lightspeed-core.github.io/lightspeed-stack/user_doc/user_data_collection.html)

docs/design/conversation-compaction/conversation-compaction-spike.md

@@ -146,7 +146,7 @@ The "buffer zone" is the most recent turns kept verbatim (not summarized).
 
 Anthropic's compaction uses token-based thresholds throughout — the buffer is implicit (whatever fits after the compaction block).
 
-**Recommendation**: **Hybrid with degrading guard**. Start with the last 4 turns. If their token count exceeds the available budget, degrade to 3, then 2, then 1, then 0. This handles pathological cases where a few large turns (e.g., with tool results) would overflow the context even after summarizing everything else.
+**Recommendation**: **Hybrid with degrading guard**. Start with the last 4 turns. If their token count exceeds the available budget, degrade to 3, then 2, then 1, then 0. The token estimate only counts message text — non-message items (tool calls, tool results) are not included — so the guard may underestimate turns that carry large tool-result payloads.
 
 ## Decision 10: Concurrency during compaction
 

docs/design/conversation-compaction/conversation-compaction.md

@@ -177,7 +177,7 @@ When triggered, split conversation into:
 - **Summary zone**: Oldest turns that will be summarized.
 - **Buffer zone**: Most recent turns kept verbatim.
 
-Buffer zone uses a degrading guard: start with N turns (default 4), estimate their token count. If they exceed the available budget (context window minus summary minus new query), reduce to N-1 turns and re-estimate. Continue degrading (4→3→2→1→0) until the buffer fits. This handles pathological cases where a few large turns (e.g., with tool results) consume most of the context.
+Buffer zone uses a degrading guard: start with N turns (default 4), estimate their token count. If they exceed the available budget (`context_window * buffer_max_ratio`), reduce to N-1 turns and re-estimate. Continue degrading (4→3→2→1→0) until the buffer fits. The token estimate only counts message text — non-message items (tool calls, tool results) are not included — so the guard may underestimate turns that carry large tool-result payloads.
 
 ## Additive summarization
 

docs/devel_doc/ARCHITECTURE.md

@@ -35,7 +35,7 @@ To keep requests on-topic and protect sensitive data, LCore applies **safety shi
 - **Multi-Provider Support**: Works with multiple LLM providers (Ollama, OpenAI, Watsonx, etc.)
 - **Enterprise Security**: Authentication, authorization (RBAC), and secure credential management
 - **Resource Management**: Token-based quota limits and usage tracking
-- **Conversation Management**: Multi-turn conversations with history and caching
+- **Conversation Management**: Multi-turn conversations with history, caching, and automatic compaction
 - **RAG Integration**: Retrieval-Augmented Generation for context-aware responses
 - **Tool Orchestration**: Model Context Protocol (MCP) server integration
 - **Observability**: Prometheus metrics, structured logging, and health checks
@@ -370,6 +370,91 @@ External A2A requests go through LCore's standard authentication system (K8s, RH
 
 ---
 
+### 2.11 Conversation Compaction (`utils/compaction.py`, `utils/conversation_compaction.py`)
+
+**Purpose:** Automatically summarize older conversation turns when the conversation history approaches the LLM's context window limit, preventing HTTP 413 failures and enabling arbitrarily long conversations.
+
+**Design Philosophy (Option A):** Once compaction triggers, LCore takes ownership of the context sent to the LLM. The `conversation` parameter is dropped from the OGX call (`omit_conversation=True`), and LCore constructs the input explicitly from summaries + recent turns + new query. The full original history remains in OGX for auditing.
+
+**Architecture:**
+
+The compaction system is split into two layers:
+
+1. **Pure Logic Layer** (`utils/compaction.py`) — Side-effect-free functions:
+   - `partition_conversation()` — Splits conversation items into old and recent chunks using a *degrading guard*: starts with the configured `buffer_turns` and shrinks one pair at a time until the recent chunk fits the token budget
+   - `summarize_chunk()` — Single LLM call to produce a `ConversationSummary` from older turns
+   - `recursively_resummarize()` — Folds multiple accumulated summaries into one when they approach the context limit
+
+2. **Runtime Integration Layer** (`utils/conversation_compaction.py`) — Manages side effects:
+   - Per-conversation locking (serializes concurrent requests on the same conversation)
+   - Compaction state loading (cache-preferred with marker fallback)
+   - Marker persistence (`[lightspeed:compaction-summary]` sentinel in conversation items)
+   - `CompactionStartedEvent` emission for streaming progress indicators
+   - `apply_compaction()` (async generator) — Main entry point used by all endpoints
+   - `store_compacted_turn()` — Appends user query + LLM output when in compacted mode
+
+**Data Flow:**
+
+```text
+User Query → Enabled + Context Window Registered?
+                        │
+                  No    │    Yes
+                  ↓     │     ↓
+              Pass-through  Acquire Lock
+                                ↓
+                            Fetch Conversation Items
+                                ↓
+                            Load Compaction State
+                            (cache → marker fallback)
+                                ↓
+                            Estimate Tokens
+                            (instructions + summaries +
+                             recent items + new query)
+                                ↓
+                            Exceeds Threshold?
+                                │
+                          No    │    Yes
+                          ↓     │     ↓
+                      Skip   Partition (old | recent)
+                          │     ↓
+                          │ Summarize Old Chunk (LLM call)
+                          │     ↓
+                          │ Write Marker + Cache Summary
+                          │     ↓
+                          ├─→ Recursive Fold (if needed)
+                                ↓
+                            Build Explicit Input:
+                            [summaries + recent + query]
+                                ↓
+                            Set omit_conversation=True
+                                ↓
+                            Release Lock → Continue to LLM
+```
+
+**Endpoint Integration:**
+
+| Endpoint | Mode | Cache | `context_status` |
+|---|---|---|---|
+| `/v1/query` | Blocking (`apply_compaction_blocking()`) | Yes | Yes (`"full"` / `"summarized"`) |
+| `/v1/streaming_query` | Streaming (`apply_compaction()` generator) | Yes | Yes (in `end` event) |
+| `/v1/responses` | Blocking | Yes | No (OpenAI-compatible, silent) |
+| `/a2a` | Blocking, marker-only (no cache) | No | No (A2A protocol scope) |
+
+**Configuration:**
+
+Compaction is controlled by `CompactionConfiguration` in `lightspeed-stack.yaml`:
+- `enabled` (default: `false`) — Master switch
+- `threshold_ratio` (default: `0.7`) — Fraction of context window that triggers compaction
+- `token_floor` (default: `4096`) — Minimum token count before compaction can fire
+- `buffer_turns` (default: `4`) — Recent turns kept verbatim
+- `buffer_max_ratio` (default: `0.3`) — Max fraction of window for the buffer
+
+Models must have context windows registered via `inference.context_windows` (a map of model ID to token count).
+
+**Concurrency:** A per-conversation lock dictionary serializes concurrent compaction requests on the same conversation. Lock entries are reference-counted and cleaned up when the last waiter exits.
+
+---
+
 ## 3. Request Processing Pipeline
 
 This section illustrates how requests flow through LCore from initial receipt to final response.
@@ -400,11 +485,12 @@ Here's how a real query flows through the system:
 5. **Model Selection** - Use configured default model (e.g., `meta-llama/Llama-3.1-8B-Instruct`)
 6. **Context Building** - Retrieve conversation history, query RAG vector stores for relevant docs, determine available MCP tools
 7. **Shield moderation** - LCore-owned direct-run moderation (and agent capabilities where applicable) using shields configured in LCORE config
-8. **OGX / agent call** - Send request with system prompt, RAG context, and MCP tools
-9. **LLM Processing** - Stack / agent generates response, may invoke MCP tools, returns token counts
-10. **Post-Processing** - Generate conversation summary if new
-11. **Store Results** - Save to Cache DB, User DB, consume quota, update metrics
-12. **Return Response** - Complete LLM response with referenced documents, token usage, and remaining quota
+8. **Conversation compaction** - If enabled and estimated tokens exceed the threshold, summarize older turns and rebuild the context (see [Section 2.11](#211-conversation-compaction-utilscompactionpy-utilsconversation_compactionpy))
+9. **OGX / agent call** - Send request with system prompt, RAG context, and MCP tools
+10. **LLM Processing** - Stack / agent generates response, may invoke MCP tools, returns token counts
+11. **Post-Processing** - Generate conversation summary if new
+12. **Store Results** - Save to Cache DB, User DB, consume quota, update metrics
+13. **Return Response** - Complete LLM response with referenced documents, token usage, and remaining quota
 
 **Key Takeaways:**
 - RAG enhances responses with relevant documentation

docs/devel_doc/openapi.md

@@ -2738,7 +2738,7 @@ user's query to a selected OGX LLM and returning the generated response.
 - mcp_headers: Headers that should be passed to MCP servers.
 
 ### Returns:
-- QueryResponse: Contains the conversation ID and the LLM-generated response.
+- QueryResponse: Contains the conversation ID, the LLM-generated response, and a `context_status` field indicating whether the conversation context is `"full"` or `"summarized"`.
 
 ### Raises:
 - HTTPException:
@@ -3021,7 +3021,7 @@ content type text/event-stream.
 - mcp_headers: Headers that should be passed to MCP servers.
 
 ### Returns:
-- SSE-formatted events for the query lifecycle.
+- SSE-formatted events for the query lifecycle. Includes a `context_status` field (`"full"` or `"summarized"`) in the `end` event payload indicating whether conversation compaction was applied. When compaction is triggered, a `compaction` SSE event is emitted before inference begins.
 
 ### Raises:
 - HTTPException:
@@ -7996,6 +7996,7 @@ Attributes:
     input_tokens: Number of tokens sent to LLM.
     output_tokens: Number of tokens received from LLM.
     available_quotas: Quota available as measured by all configured quota limiters.
+    context_status: Indicates whether the conversation context sent to the LLM is "full" (complete history) or "summarized" (older turns were summarized). Present in QueryResponse and in the end event payload (EndEventData) of the streaming query response; omitted from /v1/responses (OpenAI-compatible) and /a2a responses.
 
 
 | Field | Type | Description |
@@ -8010,6 +8011,7 @@ Attributes:
 | available_quotas | object | Quota available as measured by all configured quota limiters |
 | tool_calls | array | List of tool calls made during response generation |
 | tool_results | array | List of tool results |
+| context_status | string | Indicates whether the conversation context sent to the LLM is `"full"` (complete history) or `"summarized"` (older turns were summarized). Present in QueryResponse and in the `end` event payload (EndEventData) of the streaming query response; omitted from `/v1/responses` (OpenAI-compatible) and `/a2a` responses. |
 
 
 ## QuotaExceededResponse

docs/devel_doc/query_endpoint.md

@@ -145,6 +145,7 @@ The optional `solr` field configures Solr inline RAG behavior:
 | `tool_results` | array[object] | `[]` | Tool call results |
 | `rag_chunks` | array[object] | `[]` | *(Deprecated)* RAG chunks used |
 | `truncated` | boolean | `false` | *(Deprecated)* Always `false` |
+| `context_status` | string | `"full"` | Whether the conversation context is `"full"` (complete history) or `"summarized"` (older turns were summarized via conversation compaction) |
 
 **`referenced_documents` items:**
 
@@ -242,7 +243,7 @@ Emitted when the full response is assembled.
 
 #### 7. `end`
 
-Emitted last on success. Contains metadata.
+Emitted last on success. Contains metadata including `context_status` (`"full"` or `"summarized"`).
 
 ```json
 {
@@ -251,7 +252,8 @@ Emitted last on success. Contains metadata.
     "referenced_documents": [],
     "truncated": null,
     "input_tokens": 11,
-    "output_tokens": 19
+    "output_tokens": 19,
+    "context_status": "full"
   },
   "available_quotas": {"UserQuotaLimiter": 998911}
 }
@@ -327,9 +329,9 @@ Both endpoints share the same pre-processing pipeline:
 11. Prepare Responses API parameters (model, system prompt, tools, MCP headers)
 12. Extract image attachments separately for multimodal input construction
 
-**`/v1/query` then:** applies conversation compaction (blocking), calls the LLM, generates topic summary, consumes tokens, stores results, returns JSON.
+**`/v1/query` then:** applies conversation compaction (blocking), calls the LLM, generates topic summary, consumes tokens, stores results, returns JSON. When compaction is applied, the response includes `context_status: "summarized"`; otherwise `context_status: "full"`.
 
-**`/v1/streaming_query` then:** generates a `request_id`, starts the SSE stream, emits events as the LLM generates tokens, performs post-stream cleanup (topic summary, token consumption, persistence).
+**`/v1/streaming_query` then:** generates a `request_id`, starts the SSE stream, applies compaction if needed (emitting a `compaction` SSE event), emits events as the LLM generates tokens, performs post-stream cleanup (topic summary, token consumption, persistence). The `end` event includes `context_status` indicating whether compaction was applied.
 
 ---
 
@@ -409,7 +411,8 @@ curl -X POST http://localhost:8090/v1/query \
   "tool_calls": [],
   "tool_results": [],
   "rag_chunks": [],
-  "truncated": false
+  "truncated": false,
+  "context_status": "full"
 }
 ```
 
@@ -500,7 +503,7 @@ data: {"event": "token", "data": {"id": 2, "token": " an"}}
 
 data: {"event": "turn_complete", "data": {"id": 50, "token": "Kubernetes is an open-source..."}}
 
-data: {"event": "end", "data": {"referenced_documents": [], "truncated": null, "input_tokens": 11, "output_tokens": 50}, "available_quotas": {"UserQuotaLimiter": 998950}}
+data: {"event": "end", "data": {"referenced_documents": [], "truncated": null, "input_tokens": 11, "output_tokens": 50, "context_status": "full"}, "available_quotas": {"UserQuotaLimiter": 998950}}
 ```
 
 ### Streaming Query Interrupt

docs/index.md

@@ -34,6 +34,8 @@ product questions using backend LLM services, agents, and RAG databases.
 
 [OKP guide](https://lightspeed-core.github.io/lightspeed-stack/user_doc/okp_guide.html)
 
+[Conversation compaction](https://lightspeed-core.github.io/lightspeed-stack/user_doc/conversation_compaction.html)
+
 [Authentication and Authorization](https://lightspeed-core.github.io/lightspeed-stack/user_doc/auth.html)
 
 [User data collection](https://lightspeed-core.github.io/lightspeed-stack/user_doc/user_data_collection.html)

docs/user_doc/config.md

@@ -214,6 +214,51 @@ Attributes:
 | buffer_turns     | integer | Number of recent turns to keep verbatim.                                                                    |
 | buffer_max_ratio | number  | Maximum fraction of context window the buffer zone can occupy, regardless of buffer_turns.                  |
 
+### How to enable conversation compaction
+
+Compaction is disabled by default. To enable it, add a `compaction` section to your `lightspeed-stack.yaml` and set `enabled: true`. You must also register context window sizes for the models you use via the `inference.context_windows` map so the compaction trigger can calculate when older turns should be summarized.
+
+**Minimal configuration:**
+
+```yaml
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini
+  context_windows:
+    openai/gpt-4o-mini: 128000
+
+compaction:
+  enabled: true
+```
+
+**Full configuration with all options:**
+
+```yaml
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini
+  context_windows:
+    openai/gpt-4o-mini: 128000
+    openai/gpt-4o: 128000
+
+compaction:
+  enabled: true
+  threshold_ratio: 0.7       # trigger at 70% of context window (default)
+  token_floor: 4096           # minimum tokens before compaction can fire (default)
+  buffer_turns: 4             # recent turns kept verbatim (default)
+  buffer_max_ratio: 0.3       # buffer may use at most 30% of the window (default)
+```
+
+**Key considerations:**
+
+- `context_windows` is required. Models absent from this map have no registered window and compaction will not trigger for them.
+- `threshold_ratio` controls how aggressively compaction fires. Lower values compact sooner; higher values wait longer (closer to the window limit).
+- `buffer_turns` sets how many recent user/assistant turn pairs are kept in full. A degrading guard automatically reduces this if the buffer itself would exceed `buffer_max_ratio` of the window.
+- `token_floor` prevents compaction from triggering on very short conversations.
+- When compaction is disabled (the default), requests that exceed the context window surface as HTTP 413.
+
+For a comprehensive explanation of the feature, see the [Conversation Compaction Guide](conversation_compaction.md).
+
 
 ## Configuration