diff --git a/CHANGELOG.md b/CHANGELOG.md index d669621a..c92bd768 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,14 @@ All notable changes to this project are documented in this file. The format is b ## [Unreleased] +### Fixed + +- `smpy new` apps keep `simple_module_cli` in the workspace root's `dev` group, so a plain `uv sync` no longer uninstalls it and `uv run smpy` no longer falls through to an older global `smpy`. Inside a project, `smpy` now warns when the project pins the framework to another version than its own (except `new`, `update` and `package-update`), and `smpy doctor` run by a CLI outside the project's venv says so instead of "run it inside a project". Existing apps: add `"simple_module_cli=="` to `[dependency-groups] dev` in the root `pyproject.toml` (#929). +- `make dev` reloads when a `locales/*.json` catalog changes: the app template's `dev-api` (and this repository's) pass `--reload-include '*locales/*.json'`, since uvicorn watches only `*.py`. Existing apps: add that flag to the `uvicorn` line of `dev-api` (#930). +- `smpy add` ends with "If `make dev` is running, restart it" (the module 404s until a restart), and suggests `make migration msg="add "` like `create-module`. `create-module`'s next steps and the landing page say `uv run smpy` (#931). +- `smpy new` retries its `uv sync` once with `--refresh` when it fails, so a uv index cache from before a release no longer breaks the first app made with it; if the retry fails too, the next steps say it was already retried and to wait for PyPI before `make install` (#932). +- **Behaviour change:** `make_process_revision_directives` ignores SQLAlchemy's and Alembic's SQLite warnings about the expression indexes the models declare (the hook re-emits them for new tables and names any the database lacks), and SM026 plus the boot-time note now report only expression indexes the SQLite database does not have by name. A migrated fresh app prints neither (#933). + ## [0.0.46] — 2026-11-13 ### Changed diff --git a/Makefile b/Makefile index 66774c43..bdadab76 100644 --- a/Makefile +++ b/Makefile @@ -21,7 +21,7 @@ dev: docker-up gen-pages $(MAKE) -j2 dev-api dev-ui dev-api: - uv run --project host uvicorn host.main:app --reload --port 8000 + uv run --project host uvicorn host.main:app --reload --reload-include '*locales/*.json' --port 8000 dev-ui: npm run dev diff --git a/docs/index.md b/docs/index.md index 8a79913c..8067832d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -77,7 +77,7 @@ make migration msg="add orders tables" # review the generated revision make migrate ``` -That generates `modules/orders/` with a `ModuleMeta`, a SQLModel table, contracts, a service, REST + Inertia endpoints, three React pages, locales, and a smoke test, wires it into the host and creates its table; the dev server picks it up when it reloads. The full walkthrough is in [Your first module](/guide/first-module). +That generates `modules/orders/` with a `ModuleMeta`, a SQLModel table, contracts, a service, REST + Inertia endpoints, three React pages, locales, and a smoke test, wires it into the host and creates its table; restart `make dev` to serve it, since modules are discovered at start-up. The full walkthrough is in [Your first module](/guide/first-module). ## Where to start diff --git a/docs/module-authoring.md b/docs/module-authoring.md index 0aa20946..a0618663 100644 --- a/docs/module-authoring.md +++ b/docs/module-authoring.md @@ -366,8 +366,9 @@ generated file against this checklist before treating it as portable re-emits the expression indexes of every table a revision creates, so an initial schema needs nothing. When the models declare an expression index the database does not have (one added to an existing table), autogenerate on - SQLite logs a banner naming it, and `make doctor` reports `SM026` for every - expression index. Add it to the revision by hand + SQLite logs a banner naming it, and `make doctor` reports it as `SM026` + (SQLAlchemy's and Alembic's own warnings about declared expression indexes + are silenced, since the hook covers them). Add it to the revision by hand (`op.create_index(..., [sa.text("lower(email)")], if_not_exists=True)`) and confirm with the PostgreSQL round trip below; in an app without that make target, point `SM_DATABASE_URL` at a PostgreSQL database, run `make migrate`, diff --git a/docs/reference/diagnostic-codes.md b/docs/reference/diagnostic-codes.md index b40fbf9d..e868bd3f 100644 --- a/docs/reference/diagnostic-codes.md +++ b/docs/reference/diagnostic-codes.md @@ -34,7 +34,7 @@ The framework runs a set of static checks over installed modules at app boot. Th | `SM023` | WARNING | A module's `theme.css` contains an unlayered plain rule (anything but an at-rule or a `:root`-style selector). Unlayered CSS outranks every Tailwind utility. | Move the rule to the module's `styles.css`, which is imported into `layer(components)` so utilities still win. | | `SM024` | WARNING | A unique column, constraint or index on a `MultiTenantMixin` table does not include `tenant_id`, so the first tenant to claim a value locks every other tenant out of it. | Make the key per tenant: add `tenant_id` to it (`Index(..., "tenant_id", "slug", unique=True)`). | | `SM025` | WARNING | `multi_tenant` is on but no module registered `app.state.tenant_resolver`, so only the principal's `tenant_id` claim can bind a tenant — with most auth providers every tenant-scoped query then fails closed. Checked at boot (it needs the built app), in every environment, not by `make doctor`. | Install the `tenants` module, or register your own `async (Request) -> str \| None` resolver on `app.state.tenant_resolver`. | -| `SM026` | INFO | The configured database is SQLite and a module model declares an expression index (e.g. `lower(email)`). SQLite cannot reflect those, so autogenerate and `alembic check` cannot verify them and a clean diff says nothing about them. | Add the index to a revision by hand and verify on PostgreSQL: `make migrations-roundtrip-pg` in this repository, `make migrate` then `alembic check` against a PostgreSQL `SM_DATABASE_URL` in an app. See [Migrations](/module-authoring#reviewing-a-revision-autogenerated-on-sqlite). | +| `SM026` | INFO | The configured database is SQLite and a module model declares an expression index (e.g. `lower(email)`) that the database does not have by name (every one when the database file cannot be read). SQLite cannot reflect those, so autogenerate and `alembic check` cannot verify them and a clean diff says nothing about them. | Add the index to a revision by hand and verify on PostgreSQL: `make migrations-roundtrip-pg` in this repository, `make migrate` then `alembic check` against a PostgreSQL `SM_DATABASE_URL` in an app. See [Migrations](/module-authoring#reviewing-a-revision-autogenerated-on-sqlite). | | `SM027` | WARNING | A key in `host/locales/overrides/.json` matches no catalog key, so the override was skipped. Reported by `make doctor` and at dev boot. | Fix the key: overrides replace an existing key by its full dotted path (e.g. `users.login.aside_heading`). See [i18n](/framework/i18n#overriding-module-copy). | | `SM028` | WARNING | A module's `tasks.py` imports `torch`, `tensorflow`, `ultralytics`, `osgeo`, `rasterio`, `cv2` or `sklearn` at module level, so every process that loads the module's tasks (web, each worker, beat) pays for it and a queue-restricted worker cannot avoid it. | Import the package inside the task function. See [background_tasks](/modules/background_tasks#running-a-worker-for-a-subset-of-modules). | | `SM029` | INFO | A module overrides `db_extensions()` (Postgres extensions such as `ltree`) and the configured database is SQLite, which has none. `ensure_db_extensions` raises `UnsupportedDatabaseError` when migrations run. | Point `SM_DATABASE_URL` at PostgreSQL. See [Postgres extensions and column types](/database/extensions). | diff --git a/docs/superpowers/plans/2026-11-13-issues-929-933.md b/docs/superpowers/plans/2026-11-13-issues-929-933.md new file mode 100644 index 00000000..7f72512f --- /dev/null +++ b/docs/superpowers/plans/2026-11-13-issues-929-933.md @@ -0,0 +1,19 @@ +# Issues #929–#933 implementation plan + +Design: `docs/superpowers/specs/2026-11-13-issues-929-933-design.md`. All tasks +touch disjoint files except the CHANGELOG (last task). Each: test first, then fix, +then `uv run pytest framework/cli` (or the package touched), then commit. + +1. **#929 CLI pin + doctor message + mismatch notice.** `templates/workspace/pyproject.toml.tpl`, + `cli.py` (doctor), new `_project_cli_check.py` (callback warning). Tests in + `framework/cli/tests/`. Done: render test sees the pin; doctor/mismatch tests pass. +2. **#930 locale reload.** App template `Makefile`, root `Makefile`. Test: template + Makefile contains `--reload-include '*locales/*.json'`. +3. **#931 hints.** `_add_install.py` (restart line, short migration name), `cli.py` + `_echo_app_next_steps`. Tests on the echoed text. +4. **#932 refresh retry.** `new.py` install phase. Test with a fake `subprocess.run`. +5. **#933 warnings + SM026 (hard).** `simple_module_db/migrations.py` / + `migration_portability.py` (filters), `migration_state.py` (index names), + `_migration_doctor.py` / `__main__.py` / `_expression_index.py` (SM026 filter), + `simple_module_hosting/migrations.py` (boot log filter). Tests in db/core. +6. CHANGELOG `[Unreleased]`, docs touched by behaviour (diagnostic-codes SM026), full lint + tests. diff --git a/docs/superpowers/specs/2026-11-13-issues-929-933-design.md b/docs/superpowers/specs/2026-11-13-issues-929-933-design.md new file mode 100644 index 00000000..e99668ca --- /dev/null +++ b/docs/superpowers/specs/2026-11-13-issues-929-933-design.md @@ -0,0 +1,103 @@ +# Issues #929–#933: fresh-app DX follow-ups to 0.0.46 + +Five bugs found on a fresh 0.0.46 app (`smpy new` → `create-module` → `smpy add` +→ `make dev`). One branch, one PR, one commit per issue. + +## Goal & scope + +In: the five issues below. Out: #580 (SSR, deferred), and any change to how +modules are discovered at runtime (a restart stays necessary after `smpy add`). + +## #929 — plain `uv sync` uninstalls `simple_module_cli` + +- Add `simple_module_cli=={{FRAMEWORK_VERSION}}` to the workspace root + `[dependency-groups] dev` in `templates/workspace/pyproject.toml.tpl`, next to + `simple_module_test`. `smpy package-update` already rewrites `simple_module_*` + pins in every dependency group, so updates keep it in step. +- `smpy doctor` without `simple_module_core`: when the cwd is inside an app + (`find_workspace_root` finds a project), say the `smpy` that ran is not the + project's (print its version and path) and give the fix (`uv sync --all-packages`, + then `uv run smpy …`), instead of "run it inside a project". +- Version-mismatch notice: every `smpy` invocation inside an app whose + pyproject pins `simple_module_cli==X` (or, failing that, `simple_module_core==X`) + with `X` ≠ the running CLI's version prints one stderr warning naming both + versions and the executable path. Warning only, never blocks. + +## #930 — editing `locales/*.json` doesn't reload under `make dev` + +- App template `Makefile` `dev-api`: add `--reload-include '*locales/*.json'` + (matches module `locales/` and the host's; `Path.match` is right-anchored, so + `package.json` and the rest are not watched). A reload re-runs the dev boot, + which regenerates i18n keys. +- Framework repo's own `make dev`: same flag, for parity. +- Fix the Makefile comment that claimed locales already reloaded. +- Existing apps: CHANGELOG entry gives the one-line Makefile edit. + +## #931 — no restart hint after `smpy add` + +- `smpy add` (after a successful sync): print "If `make dev` is running, + restart it: modules are discovered at start-up, not on reload." Always, not + port-probed: a probe adds a failure mode and a stale-port false negative for + one line of text. +- `create-module` next steps: `uv run smpy add …` (not bare `smpy`); migration + message uses the same slug as `smpy add`'s hint; drop `make install` if `smpy add` + already covers it (it runs `uv sync --all-packages`, gen-pages and sync-js-deps; + if the new npm workspace member still needs `npm install`, print that instead). +- `smpy add`'s migration hint uses the short module name (`add bookmarks`, + `simple_module_` prefix stripped) to match. + +## #932 — `smpy new` hits uv's stale index right after a release + +- In `smpy new`'s install phase, when `uv sync` fails, retry once with + `--refresh`, printing "uv's cached PyPI index may predate this release; + retrying with --refresh". Only the second failure prints the uv output and the + manual-setup warning. +- Rejected: always passing `--refresh-package` for every pinned package — ~20 + extra index fetches on every `smpy new`, for a failure that only happens in the + minutes after a release. +- When the retry also fails (or install is skipped), next steps say + `uv sync --all-packages --refresh` helps if resolution fails on a just-released + version. + +## #933 — library warnings on every `make migration`, SM026 on every doctor + +- `make_process_revision_directives(metadata)` (already called by every app's + `env.py`, so existing apps get it without a template change) installs + `warnings.filterwarnings("ignore", …)` for the SQLAlchemy `SAWarning` + ("Skipped unsupported reflection of expression-based index …") and Alembic's + `UserWarning` ("autogenerate skipping metadata-specified expression-based index + …"), narrowed to the expression-index names the metadata declares. The hook + already re-emits these for new tables and names the ones the database lacks. +- SM026: report only expression indexes whose name the SQLite database does + not have. Doctor already reads the database for SM010/SM011; extend that read + (`read_migration_state`) with SQLite index names and pass them to SM026. No + database reachable → unchanged behaviour (report all). A fresh migrated app + then shows no SM026. The dev-boot path keeps reporting through the existing + `_note_unverifiable_expression_indexes` log, filtered the same way. + +## Decisions made (override any) + +- One PR for all five (they share the CLI and template; previous batches did the same). +- Version-mismatch notice is a warning, not an error. +- `smpy add` restart hint is unconditional. +- `--refresh` on retry, not up front. +- Warning filters live in `simple_module_db`, not the template `env.py`, so existing apps benefit. + +## Testing + +- Unit: template render contains the cli pin; doctor error text inside/outside an + app; mismatch notice (pinned ≠ running, equal, no app); `smpy new` retry path + with a fake runner (fail → refresh → ok; fail twice); `smpy add` output; create-module + next steps; warning filter (emits nothing for declared names, still warns for + undeclared); SM026 with and without existing index names. +- Integration: existing scaffold tests (`smpy new` e2e with local sources) still pass. +- Manual: scaffold an app from the branch, `uv sync`, `uv run which smpy`, + `make migration` output, `smpy doctor`, edit `locales/en.json` under `make dev` + and see a reload. + +## Risks + +- Alembic/SQLAlchemy warning text could change; the filter then stops + matching and the warnings come back (harmless). Tests pin the current text. +- `npm install` may still be needed after `smpy add` for a new npm workspace + member; verified during implementation before dropping `make install`. diff --git a/framework/cli/simple_module_cli/_add_install.py b/framework/cli/simple_module_cli/_add_install.py index 23ba83e3..c25d92f4 100644 --- a/framework/cli/simple_module_cli/_add_install.py +++ b/framework/cli/simple_module_cli/_add_install.py @@ -2,6 +2,7 @@ from __future__ import annotations +import re import subprocess from collections.abc import Callable from pathlib import Path @@ -12,6 +13,8 @@ ExecRunner = Callable[[list[str], Path], int] +_DIST_PREFIX = re.compile(r"^simple[-_]module[-_]", re.IGNORECASE) + _ENTRYPOINT_CHECK = ( "import sys\n" @@ -77,7 +80,13 @@ def post_install( "entry point — it is not a SimpleModule module." ) for dist in models_dists: + # The same message `smpy create-module`'s next steps suggest (GH #931). + slug = _DIST_PREFIX.sub("", dist) typer.echo( f"{dist} ships database models — generate and apply the migration:\n" - f' make migration msg="add {dist}"\n make migrate' + f' make migration msg="add {slug}"\n make migrate' ) + # Only `pyproject.toml` changed, which the dev reloader does not watch; and a + # reload still logs the new module as not importable in that process (GH #915, + # #931), so only a restart serves it. + typer.echo("If `make dev` is running, restart it: modules are discovered at start-up.") diff --git a/framework/cli/simple_module_cli/_project_cli_check.py b/framework/cli/simple_module_cli/_project_cli_check.py new file mode 100644 index 00000000..bd7e08fd --- /dev/null +++ b/framework/cli/simple_module_cli/_project_cli_check.py @@ -0,0 +1,84 @@ +"""Notice when ``smpy`` is not the CLI of the app it runs in (GH #929). + +``uv run smpy`` falls through to whatever ``smpy`` is on ``PATH`` once the +project venv lost ``simple_module_cli`` (a plain ``uv sync`` used to drop it). +That is often an older ``uv tool install``-ed copy, which then scaffolds modules +pinned to its own version inside a newer app, or fails with errors that make no +sense inside the project. Reading the app's pins is cheap and local. +""" + +from __future__ import annotations + +import re +import sys +import tomllib +from collections.abc import Iterator +from pathlib import Path + +import typer + +from simple_module_cli.pins import PLACEHOLDER_VERSION + +_FRAMEWORK_PIN = re.compile(r"^\s*simple[-_]module[-_][a-z0-9_-]+\s*==\s*([^\s;,\[]+)", re.I) + +# Commands that legitimately run a CLI of another version than the app's: +# `new` makes an app, `update`/`package-update` change its pins. +_EXEMPT_COMMANDS = frozenset({"new", "update", "package-update"}) + + +def _requirements(doc: dict) -> Iterator[str]: + project = doc.get("project", {}) + yield from project.get("dependencies", []) + for extra in project.get("optional-dependencies", {}).values(): + yield from extra + for group in doc.get("dependency-groups", {}).values(): + yield from group + + +def _pinned_version(pyproject: Path) -> str | None: + try: + doc = tomllib.loads(pyproject.read_text(encoding="utf-8")) + except (OSError, tomllib.TOMLDecodeError): + return None + for requirement in _requirements(doc): + match = _FRAMEWORK_PIN.match(requirement) if isinstance(requirement, str) else None + if match: + return match.group(1) + return None + + +def app_framework_pin(start: Path) -> tuple[Path, str] | None: + """``(project root, version)`` of the nearest project pinning the framework with ``==``. + + A ``smpy new`` app pins every ``simple_module_*`` package to one version, in + the root ``pyproject.toml`` and (workspace mode) ``host/pyproject.toml``. + Range requirements, such as this repository's own, are not pins. + """ + current = start.resolve() + for candidate in (current, *current.parents): + for pyproject in (candidate / "pyproject.toml", candidate / "host" / "pyproject.toml"): + version = _pinned_version(pyproject) if pyproject.is_file() else None + if version is not None: + return candidate, version + return None + + +def running_cli() -> str: + """Where the running ``smpy`` lives, for messages.""" + return str(Path(sys.argv[0]).resolve()) if sys.argv and sys.argv[0] else sys.executable + + +def warn_if_not_the_apps_cli(command: str | None, current: str, start: Path) -> None: + """One stderr warning when the project pins another framework version than this CLI's.""" + if command in _EXEMPT_COMMANDS or current == PLACEHOLDER_VERSION: + return + found = app_framework_pin(start) + if found is None or found[1] == current: + return + root, pinned = found + typer.echo( + f"WARNING: this project ({root}) pins the framework to {pinned}, but this smpy is " + f"{current} ({running_cli()}).\n" + " Use the project's own CLI: `uv sync --all-packages`, then `uv run smpy ...`.", + err=True, + ) diff --git a/framework/cli/simple_module_cli/cli.py b/framework/cli/simple_module_cli/cli.py index bd4d721c..dabdbcf2 100644 --- a/framework/cli/simple_module_cli/cli.py +++ b/framework/cli/simple_module_cli/cli.py @@ -20,6 +20,11 @@ import typer +from simple_module_cli._project_cli_check import ( + app_framework_pin, + running_cli, + warn_if_not_the_apps_cli, +) from simple_module_cli.add_cmd import add_module, run_add from simple_module_cli.case import to_kebab_case from simple_module_cli.docker_cmd import docker_app @@ -42,6 +47,12 @@ add_completion=False, ) + +@app.callback() +def _check_project_cli(ctx: typer.Context) -> None: + warn_if_not_the_apps_cli(ctx.invoked_subcommand, resolve_framework_version(), Path.cwd()) + + app.command("new")(new_project) app.command("add")(add_module) app.command("update")(update_modules) @@ -60,9 +71,16 @@ def doctor() -> None: from simple_module_core.__main__ import main as run_doctor except ImportError: # The CLI installs on its own (to scaffold); doctor needs a project's framework. - typer.echo( - "ERROR: smpy doctor needs simple_module_core; run it inside a project.", err=True - ) + if app_framework_pin(Path.cwd()) is None: + message = "smpy doctor needs simple_module_core; run it inside a project." + else: + # Inside an app, this is a global smpy outside the app's venv (GH #929). + message = ( + f"this smpy ({running_cli()}) is not installed in this project's venv, " + "so it cannot see the app's modules. Run `uv sync --all-packages`, " + "then `uv run smpy doctor`." + ) + typer.echo(f"ERROR: {message}", err=True) raise typer.Exit(1) from None raise typer.Exit(run_doctor()) @@ -139,7 +157,8 @@ def _app_add_command(target: Path) -> str | None: rel = f"./{rel}" # parse_add_spec reads a bare "a/b" as a PyPI requirement # The steps are printed "from the app root", so name the host relative to it. host_arg = host.resolve().relative_to(root.resolve()).as_posix() - return f"smpy add {rel} --path {host_arg}" + # `uv run`: after `uvx … smpy new`, `smpy` itself is not on PATH (GH #931). + return f"uv run smpy add {rel} --path {host_arg}" def _echo_app_next_steps(add_command: str, slug: str) -> None: @@ -150,7 +169,7 @@ def _echo_app_next_steps(add_command: str, slug: str) -> None: """ typer.echo("\nNext steps (from the app root):") typer.echo(f" {add_command} # adds it as {{ workspace = true }}") - typer.echo(" make install") + typer.echo(" make install # links its pages (npm) and generates its i18n keys") typer.echo(f' make migration msg="add {slug}"') typer.echo(" make migrate") typer.echo(" make test") diff --git a/framework/cli/simple_module_cli/new.py b/framework/cli/simple_module_cli/new.py index 8dc6730e..c463a296 100644 --- a/framework/cli/simple_module_cli/new.py +++ b/framework/cli/simple_module_cli/new.py @@ -192,8 +192,15 @@ def new_project( ) _echo_next_steps(target, manual_setup, resolved) return - if not _run_quietly(cmd, target): + run = _sync_retrying_with_refresh if cmd is uv_sync else _run_quietly + if not run(cmd, target): _echo_next_steps(target, manual_setup, resolved) + if cmd is uv_sync: + typer.echo( + f"`{' '.join(cmd)}` was already retried with --refresh. If uv still cannot " + "find this release's packages, PyPI may not list them yet: wait a few " + "minutes, then run `make install`." + ) return alembic = _alembic_argv(target, host_dir) @@ -211,19 +218,37 @@ def new_project( _echo_next_steps(target, [], resolved) -def _run_quietly(cmd: list[str], cwd: Path) -> bool: - """Run a setup step, showing its output only when it fails (GH #917).""" +def _sync_retrying_with_refresh(cmd: list[str], cwd: Path) -> bool: + """``uv sync``, retried once with ``--refresh`` when it fails (GH #932). + + The pins are this CLI's own version, so they are published; right after a + release uv's cached index may still predate them. + """ + if _run_quietly(cmd, cwd, report=False): + return True + typer.echo("uv's cached PyPI index may predate this release; retrying with --refresh...") + return _run_quietly([*cmd, "--refresh"], cwd) + + +def _run_quietly(cmd: list[str], cwd: Path, *, report: bool = True) -> bool: + """Run a setup step, showing its output only when it fails (GH #917). + + ``report=False`` keeps a failure silent, for a caller that retries. + """ try: result = subprocess.run( cmd, cwd=cwd, check=False, capture_output=True, text=True, errors="replace" ) except OSError as exc: # the executable is missing - typer.echo( - f"WARNING: {' '.join(cmd)} could not run ({exc}); finish setup manually.", err=True - ) + if report: + typer.echo( + f"WARNING: {' '.join(cmd)} could not run ({exc}); finish setup manually.", err=True + ) return False if result.returncode == 0: return True + if not report: + return False typer.echo(f"{result.stdout}{result.stderr}", err=True) typer.echo( f"WARNING: {' '.join(cmd)} failed (exit {result.returncode}); finish setup manually.", diff --git a/framework/cli/simple_module_cli/templates/host/Makefile b/framework/cli/simple_module_cli/templates/host/Makefile index 53585d5b..3349862a 100644 --- a/framework/cli/simple_module_cli/templates/host/Makefile +++ b/framework/cli/simple_module_cli/templates/host/Makefile @@ -18,7 +18,7 @@ dev: gen-pages $(MAKE) -j2 dev-api dev-ui dev-api: - uv run uvicorn main:app --reload --port $(API_PORT) + uv run uvicorn main:app --reload --reload-include '*locales/*.json' --port $(API_PORT) dev-ui: cd client_app && npm run dev diff --git a/framework/cli/simple_module_cli/templates/host/client_app/pages/Landing.tsx b/framework/cli/simple_module_cli/templates/host/client_app/pages/Landing.tsx index 8daf2813..99405dc5 100644 --- a/framework/cli/simple_module_cli/templates/host/client_app/pages/Landing.tsx +++ b/framework/cli/simple_module_cli/templates/host/client_app/pages/Landing.tsx @@ -53,17 +53,17 @@ export default function Landing({ isAuthenticated }: LandingProps) {
  • Scaffold a module:{' '} - smpy create-module orders --dest modules/orders + uv run smpy create-module orders --dest modules/orders
  • Wire it into the host:{' '} - smpy add ../modules/orders --path host + uv run smpy add ../modules/orders --path host {' '} ( - smpy add ./modules/orders + uv run smpy add ./modules/orders {' '} in a flat app)
  • diff --git a/framework/cli/simple_module_cli/templates/workspace/Makefile b/framework/cli/simple_module_cli/templates/workspace/Makefile index e2a850f4..b0a29f8e 100644 --- a/framework/cli/simple_module_cli/templates/workspace/Makefile +++ b/framework/cli/simple_module_cli/templates/workspace/Makefile @@ -18,10 +18,11 @@ dev: gen-pages $(MAKE) -j2 dev-api dev-ui # --reload-dir keeps the reloader watching in-repo module packages under -# modules/* as well as the host, so edits to a module's routes/endpoints/ -# locales hot-reload too (uvicorn otherwise watches only the launch cwd). +# modules/* as well as the host (uvicorn otherwise watches only the launch cwd). +# The reloader watches only *.py unless told otherwise, so --reload-include adds +# the locale catalogs: a reload re-reads them and regenerates the i18n keys. dev-api: - cd host && uv run uvicorn main:app --reload --reload-dir . --reload-dir ../modules --port $(API_PORT) + cd host && uv run uvicorn main:app --reload --reload-dir . --reload-dir ../modules --reload-include '*locales/*.json' --port $(API_PORT) dev-ui: npm run dev diff --git a/framework/cli/simple_module_cli/templates/workspace/pyproject.toml.tpl b/framework/cli/simple_module_cli/templates/workspace/pyproject.toml.tpl index 1c94c395..86b89167 100644 --- a/framework/cli/simple_module_cli/templates/workspace/pyproject.toml.tpl +++ b/framework/cli/simple_module_cli/templates/workspace/pyproject.toml.tpl @@ -38,6 +38,9 @@ dev = [ "{{TY_REQUIREMENT}}", # Provides the build_test_app / fake_event_bus fixtures (pytest11 plugin). "simple_module_test=={{FRAMEWORK_VERSION}}", + # `uv run smpy`: without it here a plain `uv sync` uninstalls the CLI, and + # `uv run smpy` falls through to any older smpy on PATH (GH #929). + "simple_module_cli=={{FRAMEWORK_VERSION}}", ] [tool.pytest.ini_options] diff --git a/framework/cli/tests/test_cli_add_cmd.py b/framework/cli/tests/test_cli_add_cmd.py index 3b3e8d22..ffd774fb 100644 --- a/framework/cli/tests/test_cli_add_cmd.py +++ b/framework/cli/tests/test_cli_add_cmd.py @@ -132,7 +132,18 @@ def test_models_module_prints_migration_reminder( repo = make_git_module_repo([("simple_module_blog", "0.1.0", None, True)]) _, runner = _recorder() run_add(f"git+{repo.as_uri()}", pyproject=host_pyproject, exec_runner=runner) - assert "migration" in capsys.readouterr().out.lower() + out = capsys.readouterr().out + # The same message as create-module's next steps (GH #931). + assert 'make migration msg="add blog"' in out + + +def test_add_says_to_restart_a_running_dev_server( + make_git_module_repo, host_pyproject, capsys +) -> None: + repo = make_git_module_repo([("simple_module_blog", "0.1.0", None, False)]) + _, runner = _recorder() + run_add(f"git+{repo.as_uri()}", pyproject=host_pyproject, exec_runner=runner) + assert "If `make dev` is running, restart it" in capsys.readouterr().out def test_no_sync_writes_but_skips_pipeline(make_git_module_repo, host_pyproject) -> None: diff --git a/framework/cli/tests/test_cli_app_install_flow.py b/framework/cli/tests/test_cli_app_install_flow.py index bed78241..dbc1db8d 100644 --- a/framework/cli/tests/test_cli_app_install_flow.py +++ b/framework/cli/tests/test_cli_app_install_flow.py @@ -48,7 +48,7 @@ def test_create_module_inside_an_app_prints_the_app_flow(tmp_path, monkeypatch) assert result.exit_code == 0, result.output steps = result.output.split("Next steps", 1)[1] - assert "smpy add ../modules/notes --path host" in steps + assert "uv run smpy add ../modules/notes --path host" in steps assert "make install" in steps assert 'make migration msg="add notes"' in steps assert "make test" in steps diff --git a/framework/cli/tests/test_cli_new_quiet_install.py b/framework/cli/tests/test_cli_new_quiet_install.py index 4ba0dca5..41072fdd 100644 --- a/framework/cli/tests/test_cli_new_quiet_install.py +++ b/framework/cli/tests/test_cli_new_quiet_install.py @@ -78,5 +78,51 @@ def missing(cmd, **kwargs): app, ["new", "demo", "--yes", "--db", "sqlite", "--dest", str(tmp_path / "demo")] ) assert result.exit_code == 0, result.output - assert "could not run" in result.output + # Once, although the sync is retried with --refresh (GH #932). + assert result.output.count("could not run") == 1 assert "make install" in result.output.split("Next steps", 1)[1] + + +def _invoke_sync_failures(tmp_path: Path, monkeypatch, failures: int): + """`uv sync` fails ``failures`` times, then succeeds (GH #932).""" + calls: list[list[str]] = [] + remaining = [failures] + + def fake_run(cmd, *, cwd, check, capture_output=False, text=False, errors=None): + del cwd, check, capture_output, text, errors + calls.append(list(cmd)) + failed = cmd[:2] == ["uv", "sync"] and remaining[0] > 0 + if failed: + remaining[0] -= 1 + return subprocess.CompletedProcess( + cmd, 1 if failed else 0, stdout="", stderr="No solution found\n" if failed else "" + ) + + monkeypatch.setattr(new_mod.shutil, "which", lambda name: f"/usr/bin/{name}") + monkeypatch.setattr(new_mod.subprocess, "run", fake_run) + result = CliRunner().invoke( + app, ["new", "demo", "--yes", "--db", "sqlite", "--dest", str(tmp_path / "demo")] + ) + return result, calls + + +def test_a_failed_sync_is_retried_once_with_refresh(tmp_path, monkeypatch) -> None: + result, calls = _invoke_sync_failures(tmp_path, monkeypatch, failures=1) + + syncs = [c for c in calls if c[:2] == ["uv", "sync"]] + assert syncs == [ + ["uv", "sync", "--all-packages"], + ["uv", "sync", "--all-packages", "--refresh"], + ] + assert "retrying with --refresh" in result.output + assert "No solution found" not in result.output + assert "Setup complete" in result.output + + +def test_a_sync_that_fails_after_refresh_shows_the_output(tmp_path, monkeypatch) -> None: + result, calls = _invoke_sync_failures(tmp_path, monkeypatch, failures=2) + + assert len([c for c in calls if c[:2] == ["uv", "sync"]]) == 2 + assert "No solution found" in result.output + assert "Setup complete" not in result.output + assert "already retried with --refresh" in result.output.split("Next steps", 1)[1] diff --git a/framework/cli/tests/test_cli_new_scaffold_layout.py b/framework/cli/tests/test_cli_new_scaffold_layout.py index 8f21a876..7d417f7a 100644 --- a/framework/cli/tests/test_cli_new_scaffold_layout.py +++ b/framework/cli/tests/test_cli_new_scaffold_layout.py @@ -117,6 +117,17 @@ def test_sm_new_dev_api_watches_modules_dir(tmp_path: Path) -> None: assert "../modules" in makefile +def test_sm_new_dev_api_reloads_on_locale_edits(tmp_path: Path) -> None: + """GH #930: the reloader watches only *.py by default, so a locale edit + never reached a running `make dev`.""" + from simple_module_cli.app_project import create_app_project + + for flat in (False, True): + target = tmp_path / ("flat" if flat else "ws") + create_app_project(target, name="demo", db="sqlite", tenancy=False, selected=[], flat=flat) + assert "--reload-include '*locales/*.json'" in (target / "Makefile").read_text() + + def test_sm_new_makefile_has_quality_gate_targets(tmp_path: Path) -> None: """Regression #201: the scaffold Makefile must emit test/lint/doctor targets and use `alembic upgrade heads` (plural) for per-module branch heads.""" diff --git a/framework/cli/tests/test_cli_project_cli_check.py b/framework/cli/tests/test_cli_project_cli_check.py new file mode 100644 index 00000000..29b872de --- /dev/null +++ b/framework/cli/tests/test_cli_project_cli_check.py @@ -0,0 +1,108 @@ +"""`smpy` says when it is not the CLI of the app it runs in (GH #929).""" + +from __future__ import annotations + +import builtins +import tomllib +from pathlib import Path + +import pytest +from simple_module_cli import cli as cli_mod +from simple_module_cli._project_cli_check import app_framework_pin, warn_if_not_the_apps_cli +from simple_module_cli.app_project import create_app_project +from simple_module_cli.cli import app +from simple_module_cli.pins import resolve_framework_version +from typer.testing import CliRunner + +_APP = '[project]\nname = "demo"\nversion = "0.1.0"\ndependencies = []\n' + + +def _app(root: Path, *, root_dev: list[str] | None = None, host: list[str] | None = None) -> Path: + root.mkdir(parents=True) + groups = f"\n[dependency-groups]\ndev = {root_dev!r}\n".replace("'", '"') if root_dev else "" + (root / "pyproject.toml").write_text(_APP + groups, encoding="utf-8") + if host is not None: + (root / "host").mkdir() + deps = repr(host).replace("'", '"') + (root / "host" / "pyproject.toml").write_text( + f'[project]\nname = "demo-host"\nversion = "0.1.0"\ndependencies = {deps}\n', + encoding="utf-8", + ) + return root + + +def test_the_workspace_root_keeps_the_cli_in_its_dev_group(tmp_path: Path) -> None: + target = tmp_path / "demo" + create_app_project(target, name="demo", db="sqlite", tenancy=False, selected=[]) + root = tomllib.loads((target / "pyproject.toml").read_text(encoding="utf-8")) + version = resolve_framework_version() + assert f"simple_module_cli=={version}" in root["dependency-groups"]["dev"] + + +def test_the_pin_is_found_from_a_module_subdirectory(tmp_path: Path) -> None: + root = _app(tmp_path / "demo", host=["simple_module_core==0.0.46"]) + nested = root / "modules" / "notes" + nested.mkdir(parents=True) + assert app_framework_pin(nested) == (root.resolve(), "0.0.46") + + +def test_ranges_are_not_pins(tmp_path: Path) -> None: + _app(tmp_path / "demo", root_dev=["simple_module_test>=0.0.40"]) + assert app_framework_pin(tmp_path / "demo") is None + + +@pytest.mark.parametrize("command", [None, "create-module", "doctor"]) +def test_a_mismatch_warns_once(tmp_path: Path, capsys, command) -> None: + root = _app(tmp_path / "demo", root_dev=["simple_module_cli==0.0.46"]) + warn_if_not_the_apps_cli(command, "0.0.45", root) + err = capsys.readouterr().err + assert err.count("WARNING") == 1 + assert "0.0.46" in err + assert "0.0.45" in err + assert "uv run smpy" in err + + +@pytest.mark.parametrize("command", ["new", "update", "package-update"]) +def test_version_changing_commands_do_not_warn(tmp_path: Path, capsys, command) -> None: + root = _app(tmp_path / "demo", root_dev=["simple_module_cli==0.0.46"]) + warn_if_not_the_apps_cli(command, "0.0.45", root) + assert capsys.readouterr().err == "" + + +def test_a_matching_version_and_no_app_stay_silent(tmp_path: Path, capsys) -> None: + root = _app(tmp_path / "demo", root_dev=["simple_module_cli==0.0.46"]) + warn_if_not_the_apps_cli("doctor", "0.0.46", root) + outside = tmp_path / "elsewhere" + outside.mkdir() + warn_if_not_the_apps_cli("doctor", "0.0.45", outside) + assert capsys.readouterr().err == "" + + +def _without_core(monkeypatch) -> None: + real_import = builtins.__import__ + + def fake_import(name, *args, **kwargs): + if name.startswith("simple_module_core"): + raise ImportError(name) + return real_import(name, *args, **kwargs) + + monkeypatch.setattr(builtins, "__import__", fake_import) + monkeypatch.setattr(cli_mod, "resolve_framework_version", lambda: "0.0.46") + + +def test_doctor_inside_an_app_names_the_wrong_smpy(tmp_path: Path, monkeypatch) -> None: + root = _app(tmp_path / "demo", host=["simple_module_core==0.0.46"]) + monkeypatch.chdir(root) + _without_core(monkeypatch) + result = CliRunner().invoke(app, ["doctor"]) + assert result.exit_code == 1 + assert "not installed in this project's venv" in result.output + assert "run it inside a project" not in result.output + + +def test_doctor_outside_a_project_keeps_the_plain_error(tmp_path: Path, monkeypatch) -> None: + monkeypatch.chdir(tmp_path) + _without_core(monkeypatch) + result = CliRunner().invoke(app, ["doctor"]) + assert result.exit_code == 1 + assert "run it inside a project" in result.output diff --git a/framework/core/simple_module_core/__main__.py b/framework/core/simple_module_core/__main__.py index dc02c9fd..3ff8cb1f 100644 --- a/framework/core/simple_module_core/__main__.py +++ b/framework/core/simple_module_core/__main__.py @@ -33,6 +33,7 @@ run_diagnostics, ) from simple_module_core.diagnostics._dependencies import CODE as MISSING_DEPENDENCY_CODE +from simple_module_core.diagnostics._expression_index import sqlite_file_index_names from simple_module_core.diagnostics._i18n import unknown_overrides_for from simple_module_core.discovery import ( discover_modules, @@ -194,6 +195,7 @@ def main() -> int: i18n_extra_sources=extra, i18n_unknown_overrides=unknown_overrides, database_dialect=_database_dialect(), + existing_index_names=sqlite_file_index_names(_database_url()), ) diagnostics.extend(override_findings) diagnostics.extend(migration_diagnostics(modules, _database_url(), find_env_file().parent)) diff --git a/framework/core/simple_module_core/diagnostics/_expression_index.py b/framework/core/simple_module_core/diagnostics/_expression_index.py index 771699dd..a7dae36e 100644 --- a/framework/core/simple_module_core/diagnostics/_expression_index.py +++ b/framework/core/simple_module_core/diagnostics/_expression_index.py @@ -4,7 +4,8 @@ autogenerate and ``alembic check`` silently skip them there and report a clean diff. That is "unverified", not "clean": on Postgres the same index is either missing from the history or drifted. SM026 says so whenever the configured -database is SQLite and any module model declares such an index. +database is SQLite and a module model declares such an index the database +does not have by name. Duck-typed on SQLAlchemy ``Table``/``Index`` objects (core does not depend on SQLAlchemy). @@ -15,7 +16,9 @@ import importlib import logging import re +import sqlite3 from collections.abc import Iterable +from contextlib import closing from pathlib import Path from typing import TYPE_CHECKING, Any @@ -122,13 +125,48 @@ def expression_index_advice(root: Path | None = None) -> str: ) +def sqlite_file_index_names(database_url: str) -> frozenset[str] | None: + """Every index name in the SQLite file ``database_url`` names, or ``None``. + + SQLite records an expression index's name even though SQLAlchemy cannot + reflect its definition. ``None`` when that cannot be read: another + database, an in-memory or URI-mode path, or a file that is missing (it is + opened read-only, so it is never created) or unreadable. + """ + if not database_url.startswith("sqlite"): + return None + _, sep, rest = database_url.partition(":///") + path = rest.partition("?")[0] + if not sep or not path or path.startswith((":memory:", "file:")) or not Path(path).is_file(): + return None + try: + uri = f"{Path(path).resolve().as_uri()}?mode=ro" + with closing(sqlite3.connect(uri, uri=True)) as conn: + rows = conn.execute("SELECT name FROM sqlite_master WHERE type = 'index'").fetchall() + except sqlite3.Error: + return None + return frozenset(row[0] for row in rows) + + def check_expression_indexes_unverifiable( - tables: Iterable[Any], dialect: str | None, module_name: str + tables: Iterable[Any], + dialect: str | None, + module_name: str, + *, + existing_index_names: frozenset[str] | None = None, ) -> list[Diagnostic]: - """SM026: INFO on SQLite when the models declare indexes it cannot verify.""" + """SM026: INFO on SQLite when the models declare indexes it cannot verify. + + ``existing_index_names`` (from :func:`sqlite_file_index_names`) drops the + indexes the database already has by name, as the revision hook does, so a + migrated app hears only about the ones it lacks (GH #933). ``None`` reports + them all. + """ if dialect != "sqlite": return [] found = find_expression_indexes(tables) + if existing_index_names is not None: + found = [(table, index) for table, index in found if index not in existing_index_names] if not found: return [] names = ", ".join(f"{idx} ({tbl})" for tbl, idx in found) @@ -153,5 +191,6 @@ def check_expression_indexes_unverifiable( "find_expression_indexes", "index_is_expression_based", "module_all_tables", + "sqlite_file_index_names", "unwrap_ordering", ] diff --git a/framework/core/simple_module_core/diagnostics/_runner.py b/framework/core/simple_module_core/diagnostics/_runner.py index 8e363b46..013cc4e5 100644 --- a/framework/core/simple_module_core/diagnostics/_runner.py +++ b/framework/core/simple_module_core/diagnostics/_runner.py @@ -33,6 +33,7 @@ def run_diagnostics( i18n_extra_sources: list[tuple[str, str, Path]] | None = None, database_dialect: str | None = None, i18n_unknown_overrides: dict[str, list[str]] | None = None, + existing_index_names: frozenset[str] | None = None, ) -> list[Diagnostic]: """Convenience function to run all diagnostics. @@ -44,7 +45,8 @@ def run_diagnostics( host/ui locale dirs that aren't owned by a ``ModuleBase``. ``database_dialect`` (``"sqlite"``, ``"postgresql"``...) enables SM026, which flags expression indexes a SQLite database cannot verify, and SM029, which flags - modules declaring ``db_extensions`` on SQLite. + modules declaring ``db_extensions`` on SQLite. ``existing_index_names`` + (``sqlite_file_index_names``) limits SM026 to indexes the database lacks. ``i18n_unknown_overrides`` (``I18nRegistry.unknown_override_keys``) enables SM027. """ diagnostics = ModuleDiagnostics().run(modules) @@ -53,7 +55,10 @@ def run_diagnostics( for mod in modules: diagnostics.extend( check_expression_indexes_unverifiable( - module_all_tables(mod), database_dialect, mod.meta.name + module_all_tables(mod), + database_dialect, + mod.meta.name, + existing_index_names=existing_index_names, ) ) diagnostics.extend(check_db_extensions_need_postgres(mod, database_dialect)) diff --git a/framework/core/tests/test_expression_index_diagnostics.py b/framework/core/tests/test_expression_index_diagnostics.py index 8bfad8f9..a14520b8 100644 --- a/framework/core/tests/test_expression_index_diagnostics.py +++ b/framework/core/tests/test_expression_index_diagnostics.py @@ -120,3 +120,44 @@ def test_text_ordering_and_literal_columns_still_count_as_expressions(): ("t", "ix_t_literal"), ("t", "ix_t_text"), ] + + +def test_sm026_skips_indexes_the_database_already_has(): + """GH #933: a migrated fresh app has the index; SM026 has nothing to say.""" + assert ( + check_expression_indexes_unverifiable( + _tables(), + "sqlite", + "Users", + existing_index_names=frozenset({"ix_users_user_email_lower"}), + ) + == [] + ) + missing = check_expression_indexes_unverifiable( + _tables(), "sqlite", "Users", existing_index_names=frozenset() + ) + assert [d.code for d in missing] == ["SM026"] + + +def test_sqlite_file_index_names_reads_the_file(tmp_path): + from simple_module_core.diagnostics._expression_index import sqlite_file_index_names + + db = tmp_path / "app.db" + engine = sa.create_engine(f"sqlite:///{db}") + with engine.begin() as conn: + _tables()[0].metadata.create_all(conn) + engine.dispose() + + names = sqlite_file_index_names(f"sqlite+aiosqlite:///{db}") + assert names is not None + assert "ix_users_user_email_lower" in names + + +def test_sqlite_file_index_names_is_none_when_it_cannot_tell(tmp_path): + from simple_module_core.diagnostics._expression_index import sqlite_file_index_names + + missing = tmp_path / "missing.db" + assert sqlite_file_index_names(f"sqlite+aiosqlite:///{missing}") is None + assert not missing.exists() # read-only: never creates the file + assert sqlite_file_index_names("sqlite+aiosqlite:///:memory:") is None + assert sqlite_file_index_names("postgresql+asyncpg://u@h/db") is None diff --git a/framework/db/simple_module_db/migration_portability.py b/framework/db/simple_module_db/migration_portability.py index 4d064531..6a947ff0 100644 --- a/framework/db/simple_module_db/migration_portability.py +++ b/framework/db/simple_module_db/migration_portability.py @@ -20,6 +20,8 @@ from __future__ import annotations import logging +import re +import warnings from collections.abc import Iterator from typing import Any @@ -30,6 +32,7 @@ find_expression_indexes, ) from sqlalchemy.dialects import postgresql +from sqlalchemy.exc import SAWarning from sqlalchemy.schema import DefaultClause from sqlalchemy.sql.elements import ColumnElement, False_, TextClause, True_ @@ -120,6 +123,32 @@ def rewrite_boolean_defaults(upgrade_ops: Any) -> None: column.server_default = DefaultClause(replacement) +def silence_expression_index_reflection_warnings(metadata: sa.MetaData) -> None: + """Ignore the library warnings about expression indexes the models declare (GH #933). + + On SQLite, SQLAlchemy warns that it skipped reflecting each expression index + and Alembic that it skipped comparing it, on every autogenerate. For an + index in ``metadata`` the revision hook already covers both cases: it + re-emits the index for a new table and :func:`warn_unverifiable_expression_indexes` + names one the database lacks. Indexes the models do not declare still warn. + """ + names = [index for _, index in find_expression_indexes(metadata.tables.values())] + if not names: + return + alternatives = "|".join(re.escape(name) for name in names) + warnings.filterwarnings( + "ignore", + message=rf"Skipped unsupported reflection of expression-based index ({alternatives})$", + category=SAWarning, + ) + warnings.filterwarnings( + "ignore", + message=rf"autogenerate skipping metadata-specified expression-based index " + rf"'({alternatives})';", + category=UserWarning, + ) + + def warn_unverifiable_expression_indexes( dialect_name: str | None, metadata: sa.MetaData, diff --git a/framework/db/simple_module_db/migrations.py b/framework/db/simple_module_db/migrations.py index abbb615a..c1c5b459 100644 --- a/framework/db/simple_module_db/migrations.py +++ b/framework/db/simple_module_db/migrations.py @@ -36,6 +36,7 @@ from simple_module_db.migration_portability import ( iter_ops_recursive, rewrite_boolean_defaults, + silence_expression_index_reflection_warnings, sqlite_index_names, warn_unverifiable_expression_indexes, ) @@ -207,6 +208,7 @@ def make_process_revision_directives( process_revision_directives=make_process_revision_directives(target_metadata), ) """ + silence_expression_index_reflection_warnings(metadata) expression_indexes: dict[str, list[Index]] = {} for table in metadata.tables.values(): for index in table.indexes: diff --git a/framework/db/tests/test_db_expression_index_library_warnings.py b/framework/db/tests/test_db_expression_index_library_warnings.py new file mode 100644 index 00000000..1d87f704 --- /dev/null +++ b/framework/db/tests/test_db_expression_index_library_warnings.py @@ -0,0 +1,51 @@ +"""SQLAlchemy's and Alembic's SQLite expression-index warnings are quiet (GH #933). + +The revision hook re-emits the models' expression indexes for new tables and +names the ones the database lacks, so the two library warnings about the same +index told a fresh app nothing it could act on, on every `make migration`. +""" + +from __future__ import annotations + +import warnings + +import sqlalchemy as sa +from alembic.autogenerate import produce_migrations +from alembic.runtime.migration import MigrationContext +from simple_module_db.migrations import make_process_revision_directives, render_item + + +def _metadata() -> sa.MetaData: + md = sa.MetaData() + sa.Table( + "people", + md, + sa.Column("id", sa.Integer, primary_key=True), + sa.Column("email", sa.String), + sa.Index("ix_people_email_lower", sa.text("lower(email)"), unique=True), + ) + return md + + +def _autogenerate_warnings(*, undeclared_index: bool = False) -> list[str]: + target = _metadata() + engine = sa.create_engine("sqlite://") + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + hook = make_process_revision_directives(target) + with engine.connect() as conn: + _metadata().create_all(conn) + if undeclared_index: + conn.execute(sa.text("CREATE INDEX ix_people_abs_id ON people (abs(id))")) + ctx = MigrationContext.configure(conn, opts={"render_item": render_item}) + hook(ctx, None, [produce_migrations(ctx, target)]) + return [str(w.message) for w in caught] + + +def test_declared_expression_indexes_raise_no_library_warnings() -> None: + assert not [m for m in _autogenerate_warnings() if "ix_people_email_lower" in m] + + +def test_an_index_the_models_do_not_declare_still_warns() -> None: + messages = _autogenerate_warnings(undeclared_index=True) + assert any("ix_people_abs_id" in m for m in messages) diff --git a/framework/hosting/simple_module_hosting/_dev_boot.py b/framework/hosting/simple_module_hosting/_dev_boot.py index 1d8d05b2..3ada6aec 100644 --- a/framework/hosting/simple_module_hosting/_dev_boot.py +++ b/framework/hosting/simple_module_hosting/_dev_boot.py @@ -15,6 +15,7 @@ from typing import TYPE_CHECKING from simple_module_core.diagnostics import DiagnosticLevel, print_diagnostics, run_diagnostics +from simple_module_core.diagnostics._expression_index import sqlite_file_index_names from simple_module_hosting.i18n_manifest import emit_frontend_types_for_modules @@ -57,6 +58,7 @@ def run_dev_boot( i18n_extra_sources=i18n_extra, i18n_unknown_overrides=i18n_unknown_overrides, database_dialect=settings.database_url.split(":", 1)[0].split("+", 1)[0], + existing_index_names=sqlite_file_index_names(settings.database_url), ) diagnostics = diagnostics_state.rerun() errors = [d for d in diagnostics if d.level == DiagnosticLevel.ERROR] diff --git a/framework/hosting/simple_module_hosting/migrations.py b/framework/hosting/simple_module_hosting/migrations.py index cd1b3e7a..55c09f5c 100644 --- a/framework/hosting/simple_module_hosting/migrations.py +++ b/framework/hosting/simple_module_hosting/migrations.py @@ -251,12 +251,17 @@ def _note_unverifiable_expression_indexes(engine) -> None: from simple_module_core.diagnostics._expression_index import ( expression_index_advice, find_expression_indexes, + sqlite_file_index_names, ) from simple_module_db.base import all_module_bases found = find_expression_indexes( table for base in all_module_bases for table in base.metadata.tables.values() ) + # Only those the database lacks by name, as doctor's SM026 (GH #933). + existing = sqlite_file_index_names(str(engine.url)) + if existing is not None: + found = [(table, index) for table, index in found if index not in existing] except Exception: # pragma: no cover - purely advisory logger.debug("Could not inspect expression indexes", exc_info=True) return diff --git a/framework/hosting/tests/test_check_migrations_expression_note.py b/framework/hosting/tests/test_check_migrations_expression_note.py index bc3e321f..a64d92ca 100644 --- a/framework/hosting/tests/test_check_migrations_expression_note.py +++ b/framework/hosting/tests/test_check_migrations_expression_note.py @@ -9,8 +9,9 @@ from simple_module_hosting.migrations import _note_unverifiable_expression_indexes -def _engine(dialect: str): - return SimpleNamespace(dialect=SimpleNamespace(name=dialect)) +def _engine(dialect: str, url: str = "sqlite://"): + # "sqlite://" is in-memory: no file to read index names from, so every index is reported. + return SimpleNamespace(dialect=SimpleNamespace(name=dialect), url=url) def test_sqlite_logs_that_expression_indexes_are_unverified(caplog): @@ -28,3 +29,18 @@ def test_postgres_stays_quiet(caplog): _note_unverifiable_expression_indexes(_engine("postgresql")) assert caplog.text == "" + + +def test_sqlite_that_has_the_index_stays_quiet(caplog, tmp_path): + """GH #933: a migrated app's database already has it by name.""" + pytest.importorskip("simple_module_users") + import sqlite3 + + db = tmp_path / "app.db" + with sqlite3.connect(db) as conn: + conn.execute("CREATE TABLE users_user (id INTEGER PRIMARY KEY, email TEXT)") + conn.execute("CREATE INDEX ix_users_user_email_lower ON users_user (lower(email))") + with caplog.at_level(logging.INFO, logger="simple_module_hosting.migrations"): + _note_unverifiable_expression_indexes(_engine("sqlite", f"sqlite:///{db}")) + + assert "ix_users_user_email_lower" not in caplog.text diff --git a/skills/simple-module-cli/SKILL.md b/skills/simple-module-cli/SKILL.md index 3a512ae0..f3417b8c 100644 --- a/skills/simple-module-cli/SKILL.md +++ b/skills/simple-module-cli/SKILL.md @@ -97,7 +97,7 @@ smpy create-module orders --dest modules/orders # inside an app: a workspace m smpy create-module orders # writes ./simple_module_orders/ ``` -Inside an app the printed next steps are the app's: `smpy add ../modules/orders --path host` (which syncs the whole workspace), `make install`, `make migration msg="add orders"`, `make migrate`, `make test`, then restart `make dev`. Never run `uv sync` inside the module directory there: it syncs the module alone and uninstalls the rest of the app from the shared venv. Outside an app it prints the standalone flow (`uv sync --extra dev`, `uv run pytest`, `smpy module verify`). +Inside an app the printed next steps are the app's: `uv run smpy add ../modules/orders --path host` (which syncs the whole workspace and itself says to restart a running `make dev`), `make install` (links the module's pages into the npm workspace and generates its i18n keys), `make migration msg="add orders"`, `make migrate`, `make test`, then restart `make dev`. Never run `uv sync` inside the module directory there: it syncs the module alone and uninstalls the rest of the app from the shared venv. Outside an app it prints the standalone flow (`uv sync --extra dev`, `uv run pytest`, `smpy module verify`). The result is a complete package: `pyproject.toml` with the entry point declared, `module.py` with `ModuleBase`/`ModuleMeta` skeleton, `models.py`, `contracts/`, `web/endpoints/{api,views}.py`, `pages/`, `locales/en.json`, `core/{constants,settings,services}.py`, `web/{deps,service}.py`, plus nested `tests///test__*.py` wired up with `simple_module_test` fixtures. @@ -217,6 +217,8 @@ Don't bake `--password` literals into a script you commit; use a secrets store a - **Ran `smpy host gen-pages` from outside the host directory.** Defaults to `./client_app`; pass `--host-dir` from elsewhere. - **Forgot `smpy host sync-js-deps` after `pip install`-ing a module with pages.** Vite resolves module imports against `client_app/node_modules`; the new module's JS deps won't land until you sync. - **Ran `uv sync` inside a module of a `smpy new` app.** That syncs the module's own project and uninstalls the app's other packages from the shared venv. Run `make install` (`uv sync --all-packages`) at the app root instead; a plain `uv sync` at the root also works, since the root depends on the host. +- **`uv run smpy` warns that the app uses another framework version, or `smpy doctor` says this smpy is not installed in the project's venv.** A global `smpy` is answering; run `uv sync --all-packages` so the app's own CLI is installed, then `uv run smpy …`. +- **Editing `locales/*.json` doesn't reload `make dev`** in an app made before 0.0.47: add `--reload-include '*locales/*.json'` to the `uvicorn` line of `dev-api`. - **A module added while `make dev` runs logs "installed but … cannot be imported in this process".** The reload keeps the import path the server started with; restart `make dev`. - **Calling `smpy create-admin` before migrations have run.** The users tables don't exist yet; the command will error. Run `alembic upgrade head` first (or use `smpy new` which does it for you when `--no-install` isn't set). - **Expecting `--with products` (or `keycloak` / `audit_log`) to work on `smpy new`.** Those modules exist in the monorepo but aren't scaffolder-catalog keys, so `smpy new --with products` errors. Either scaffold without them and `pip install simple_module_products` into the host afterward, or use `smpy create-host --with Products` (which doesn't validate against the catalog).