Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@ database files (`.hyper`) without any C library dependencies.
>
> As of **1.0.0** the public API is stable and follows [semantic versioning](https://semver.org/):
> breaking changes require a major release, so the frequent churn of the `0.x`
> line is behind us.
> line is behind us. What counts as that public API is per-crate — see the
> [Crate Overview](#crate-overview) for the two crates that scope it
> differently: `hyperdb-api-core` is an internal implementation detail, and
> `hyperdb-mcp` governs its MCP tool surface rather than its Rust library
> target.
>
> Contributors and reviewers should, at a minimum, run an **AI code reviewer**
> over any changes, following the conventions, layering rules, and patterns
Expand Down Expand Up @@ -225,7 +229,7 @@ async fn main() -> Result<()> {
| **[hyperdb-api](hyperdb-api/README.md)** | High-level API — connections, inserters, catalog, Arrow, pooling | crates.io |
| **[hyperdb-api-core](hyperdb-api-core/README.md)** | Internal implementation details (types, protocol, client). Not a public API — depend on `hyperdb-api` instead. | crates.io |
| **[hyperdb-api-salesforce](hyperdb-api-salesforce/README.md)** | Salesforce Data Cloud OAuth authentication | crates.io |
| **[hyperdb-mcp](hyperdb-mcp/README.md)** | MCP server for LLM-driven SQL analytics on `.hyper` files | crates.io |
| **[hyperdb-mcp](hyperdb-mcp/README.md)** | MCP server for LLM-driven SQL analytics on `.hyper` files. Semver covers the MCP tool surface; the Rust library target is not a public API. | crates.io |
| **[sea-query-hyperdb](sea-query-hyperdb/README.md)** | HyperDB dialect backend for sea-query | crates.io |
| **[hyperdb-api-node](hyperdb-api-node/README.md)** | Node.js/TypeScript bindings via napi-rs | npm |
| **[hyperdb-bootstrap](hyperdb-bootstrap/README.md)** | Download the `hyperd` executable from Tableau's release packages | crates.io |
Expand Down
29 changes: 29 additions & 0 deletions hyperdb-mcp/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/).

### Changed

- **The Rust library target is not a supported API surface, and its 21 modules
are now `#[doc(hidden)]`.** `src/lib.rs` already carried a lint `reason`
saying the library "is not a documented API surface", while this crate's
`README.md` promised, without scope, that "the public API is stable and
follows semantic versioning". Against a crate where every module is `pub`,
that sentence promised semver stability on all 21 modules and every item in
them, which is why two incidental internals had to be written up as API
events: `DaemonState` gaining a private field ([#289]) and `state_perms`
becoming new public surface ([#295]). The README now scopes the promise to
the **MCP tool surface** — tool names, their parameters, and their behavior
as reached over the MCP protocol — and states that the library target is
excluded; the root `README.md` crate table says the same, alongside the
equivalent note that already existed for `hyperdb-api-core`.

**Not marked BREAKING, and nothing was privatised.** `pub(crate)` is not
available for any of the 21 modules: Cargo compiles the `hyperdb-mcp`
`[[bin]]`, each file under `tests/`, and `examples/demo.rs` as separate
crates that can only reach library items through the external
`hyperdb_mcp::` path, and every module is used by at least one of them
(`paths` by the binary alone, `stats`, `subscriptions` and `watcher` by one
test file each). So `pub` is load-bearing for compilation, not an API
commitment. `#[doc(hidden)]` removes the modules from published rustdoc and
signals intent; it does not affect name resolution, so no code that compiled
before stops compiling. Narrowing a module to `pub(crate)` later would be
source-breaking and would carry the marker — this does not.

[#289]: https://github.com/tableau/hyper-api-rust/pull/289
[#295]: https://github.com/tableau/hyper-api-rust/pull/295

<!-- The two entries below are retroactive. Both changes shipped in 0.7.3
(commit 44bdf1e, PR #243) and were missing from this file; this crate's
`## [Unreleased]` section has not been rolled over since 0.5.0, so the
Expand Down
9 changes: 8 additions & 1 deletion hyperdb-mcp/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
# hyperdb-mcp

> **Note:** This crate is AI-assisted but human-directed — much of the code was written by AI coding assistants under close review, with the design and engineering trade-offs decided by an experienced developer. As of 1.0.0 the public API is stable and follows [semantic versioning](https://semver.org/), so breaking changes require a major release.
> **Note:** This crate is AI-assisted but human-directed — much of the code was written by
> AI coding assistants under close review, with the design and engineering trade-offs decided
> by an experienced developer. As of 1.0.0 the **MCP tool surface** — tool names, their
> parameters, and their behavior as reached over the MCP protocol — is stable and follows
> [semantic versioning](https://semver.org/), so breaking changes to it require a major
> release. The Rust library target is **not** a supported API surface: it exists only to
> support the `hyperdb-mcp` binary, its tests, and its examples, its modules are
> `#[doc(hidden)]`, and items within it may change in any release.

An MCP (Model Context Protocol) server that turns the Hyper columnar database into an instant SQL analytics engine. Data flows in from other MCP plugins or files, lands in Hyper automatically, and becomes queryable with SQL — no setup, no schema files, no database management.

Expand Down
79 changes: 62 additions & 17 deletions hyperdb-mcp/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,51 +9,96 @@
//! MCP (Model Context Protocol) server that exposes the Hyper columnar database
//! as an instant SQL analytics engine for LLM workflows.
//!
//! # This library target is not a public API
//!
//! `hyperdb-mcp` ships as an MCP server binary. The library target exists to
//! support that binary, its integration tests, and its examples — it is **not**
//! a supported Rust API surface, and no item in it is covered by the crate's
//! semantic-versioning promise. Every module below is therefore
//! `#[doc(hidden)]`.
//!
//! What *is* stable and semver-governed is the **MCP tool surface**: tool
//! names, their parameters, and their behavior as reached over the MCP
//! protocol. That is the contract described in the crate
//! [README](https://github.com/tableau/hyper-api-rust/blob/main/hyperdb-mcp/README.md),
//! and the `get_readme` tool serves the LLM-facing version of it at runtime.
//!
//! The modules stay `pub` rather than `pub(crate)` because Cargo compiles the
//! `hyperdb-mcp` binary, every file under `tests/`, and every example as
//! separate crates, which can only reach library items through the external
//! `hyperdb_mcp::` path. All 21 modules are used by at least one of those, so
//! `pub` is load-bearing for compilation, not an API commitment.
//!
//! # Architecture
//!
//! The crate is layered bottom-up:
//!
//! - [`error`] — Structured error codes with recovery suggestions for LLM self-correction.
//! - [`attach`] — Registry of additional `.hyper` databases attached to the primary
//! - `error` — Structured error codes with recovery suggestions for LLM self-correction.
//! - `attach` — Registry of additional `.hyper` databases attached to the primary
//! workspace for cross-database JOINs and `copy_query`. Replays attachments
//! after a `ConnectionLost` reconnect.
//! - [`stats`] — Performance telemetry (throughput, timing) attached to every response.
//! - [`schema`] — Three-tier schema inference: exact (Arrow/Parquet), structural (JSON),
//! - `stats` — Performance telemetry (throughput, timing) attached to every response.
//! - `schema` — Three-tier schema inference: exact (Arrow/Parquet), structural (JSON),
//! heuristic (CSV). Also handles user-provided schema overrides.
//! - [`engine`] — Manages the `HyperProcess` lifecycle, connection, table CRUD, and
//! - `engine` — Manages the `HyperProcess` lifecycle, connection, table CRUD, and
//! query execution across the local database and optional persistent database.
//! - [`ingest`] — Loads inline JSON (row-by-row INSERT) and CSV (`COPY FROM`) into Hyper.
//! - [`ingest_arrow`] — Loads Parquet and Arrow IPC files via the Arrow crate.
//! - [`inspect`] — Dry-run file inspection powering the `inspect_file` MCP tool.
//! - [`export`] — Writes query results to CSV, Parquet, Arrow IPC, or `.hyper` files.
//! - [`chart`] — Renders SQL query results as PNG/SVG charts via the `plotters` crate.
//! - [`saved_queries`] — Named read-only SQL queries exposed via tools and `hyper://queries/...` resources.
//! - [`subscriptions`] — Per-URI registry of MCP clients that asked for resource-update notifications.
//! - [`table_catalog`] — User-visible catalog of data tables (`_table_catalog`) tracking
//! - `ingest` — Loads inline JSON (row-by-row INSERT) and CSV (`COPY FROM`) into Hyper.
//! - `ingest_arrow` — Loads Parquet and Arrow IPC files via the Arrow crate.
//! - `inspect` — Dry-run file inspection powering the `inspect_file` MCP tool.
//! - `export` — Writes query results to CSV, Parquet, Arrow IPC, or `.hyper` files.
//! - `chart` — Renders SQL query results as PNG/SVG charts via the `plotters` crate.
//! - `saved_queries` — Named read-only SQL queries exposed via tools and `hyper://queries/...` resources.
//! - `subscriptions` — Per-URI registry of MCP clients that asked for resource-update notifications.
//! - `table_catalog` — User-visible catalog of data tables (`_table_catalog`) tracking
//! source, purpose, and load history so workspaces are self-documenting. Disabled by `--bare`.
//! - [`version`] — Compile-time-captured version strings for the MCP crate and the underlying `hyperdb-api`, with a git-hash suffix.
//! - [`readme`] — Static LLM-facing README returned by the `get_readme` tool.
//! - [`watcher`] — Monitors directories for incremental ingest via a `.ready` sentinel protocol.
//! - [`server`] — MCP tool definitions and the `rmcp` server handler that ties everything together.
//! - `version` — Compile-time-captured version strings for the MCP crate and the underlying `hyperdb-api`, with a git-hash suffix.
//! - `readme` — Static LLM-facing README returned by the `get_readme` tool.
//! - `watcher` — Monitors directories for incremental ingest via a `.ready` sentinel protocol.
//! - `server` — MCP tool definitions and the `rmcp` server handler that ties everything together.

// Every module below is `#[doc(hidden)]`: reachable so the binary, the
// integration tests, and the examples compile, but excluded from the published
// rustdoc because the Rust surface is not the crate's supported API. See the
// crate-level docs above.
#[doc(hidden)]
pub mod attach;
#[doc(hidden)]
pub mod chart;
#[doc(hidden)]
pub mod daemon;
#[doc(hidden)]
pub mod diagnostics;
#[doc(hidden)]
pub mod engine;
#[doc(hidden)]
pub mod error;
#[doc(hidden)]
pub mod export;
#[doc(hidden)]
pub mod ingest;
#[doc(hidden)]
pub mod ingest_arrow;
#[doc(hidden)]
pub mod inspect;
#[doc(hidden)]
pub mod lakehouse;
#[doc(hidden)]
pub mod paths;
#[doc(hidden)]
pub mod readme;
#[doc(hidden)]
pub mod saved_queries;
#[doc(hidden)]
pub mod schema;
#[doc(hidden)]
pub mod server;
#[doc(hidden)]
pub mod stats;
#[doc(hidden)]
pub mod subscriptions;
#[doc(hidden)]
pub mod table_catalog;
#[doc(hidden)]
pub mod version;
#[doc(hidden)]
pub mod watcher;
5 changes: 4 additions & 1 deletion hyperdb-mcp/tests/readme_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,10 @@ fn public_docs_database_and_read_only_contract() {
"`status`",
);
let lib_source = LIB_SOURCE.to_lowercase();
let engine_crate_doc = markdown_section(&lib_source, "- [`engine`]", "- [`ingest`]");
// The crate-level architecture bullets name modules in plain code spans, not
// intra-doc links: every module is `#[doc(hidden)]`, and rustdoc leaves a
// link to a hidden item as literal `[engine]` brackets rather than resolving it.
let engine_crate_doc = markdown_section(&lib_source, "- `engine`", "- `ingest`");
let development = DEVELOPMENT.to_lowercase();
let development_prerequisites =
markdown_section(&development, "### prerequisites", "### build");
Expand Down