Skip to content

Remove legacy migration-based flow compilation before the queue-routing release #647

Description

@jumski

Summary

Remove the legacy migration-based flow compiler after #650 and #651 are implemented and validated, but before their first stable release.

Worker startup becomes the only supported deployment path for flow definitions. During #650 and #651, leave the legacy path unchanged and do not extend it with queue mode, step routes, aliases, or other deployment metadata.

This is a pre-release cleanup for #653, not a prerequisite for implementing queue identity or private per-step queues.

Why remove the complete path

The legacy workflow has four coupled surfaces:

compileFlow()
  -> ControlPlane
  -> pgflow compile
  -> generated flow-definition migration

Keeping it for new deployment metadata would require a second implementation of queue-mode and route validation. Removing only ControlPlane would break pgflow compile. Remove the complete legacy unit instead.

Scheduling

#650 queue identity
  -> #651 private per-step queues
  -> downstream validation
  -> #647 legacy compiler removal
  -> stable queue-routing release

Do not make #650 or #651 wait for this cleanup. Do not publish the permanent dual-path support matrix either.

Remove

pkgs/cli

  • Delete pgflow compile and its tests.
  • Stop creating supabase/functions/pgflow/ during installation.
  • Remove compile-command output, options, and current documentation.

pkgs/edge-worker

  • Delete ControlPlane, its public export, server code, fixtures, and tests.
  • Remove compilation: false; every flow worker compiles or verifies its definition during startup.
  • Remove compilation.allowDataLoss; remote startup never authorizes destructive recompilation.

pkgs/dsl

  • Delete the public SQL-text compileFlow() helper, export, and tests.
  • Delete TypeScript shape-comparison helpers that have no caller after ControlPlane removal.
  • Keep flow-shape extraction used by worker startup.

Documentation and examples

  • Remove the installed and demo ControlPlane functions.
  • Redirect current manual-compilation, Compile API, and ControlPlane pages to startup compilation or the migration notice.
  • Remove migration-based flow compilation from current examples, package READMEs, architecture guides, navigation, and generated indexes.
  • Keep historical release-news pages intact while marking removed APIs as obsolete in current versions.

Behavior after removal

missing concrete flow  -> compile during worker startup
matching flow          -> verify and start
local mismatch         -> use the existing automatic local recompilation behavior
remote mismatch        -> fail without deleting data; deploy a new concrete slug

A structural or semantic production change uses a new concrete slug:

const greetUserV2 = new Flow({ slug: 'greetUserV2' })

Startup shape verification cannot detect handler-code or TypeScript contract changes. The versioning guide must continue to require a new slug for those changes.

Preserve

  • Preserve current local same-slug hot reload and destructive recompilation behavior.
  • Preserve the existing SQL startup compiler, flow-shape extraction, and concrete-slug advisory lock.
  • Preserve create_flow() and add_step() as database building blocks used by startup compilation.
  • Preserve existing generated migrations as ordinary historical SQL in user repositories.
  • Preserve the production enable-fence guidance from Document the worker-function enable fence for production deployments #654.

Migration requirements

PostgreSQL keeps overloads with different argument lists. The migration must explicitly remove the production data-loss entry point:

DROP FUNCTION pgflow.ensure_flow_compiled(text, jsonb, boolean);

Then create or keep the intended signature without allow_data_loss, and regenerate database types.

The migration notice must tell existing users to:

  1. Remove supabase/functions/pgflow/ and any matching function configuration.
  2. Remove pgflow compile from scripts and CI.
  3. Remove compilation: false and compilation.allowDataLoss from worker configuration.
  4. Make each worker import its complete flow definition.
  5. Keep existing generated migration files as migration history.

Acceptance criteria

  • Persist physical queue identity for flow tasks #650 and Add private per-step queues with typed step workers #651 implementation plus downstream validation finish before this cleanup starts.
  • ControlPlane code, exports, templates, demo functions, tests, and current docs are removed.
  • pgflow compile, its tests, options, installation output, and current docs are removed.
  • Public SQL-text compileFlow() and orphaned TypeScript-only compiler helpers are removed.
  • Worker startup is the only supported owner of flow compilation and verification.
  • compilation: false and compilation.allowDataLoss are rejected by the type system and runtime.
  • The three-argument ensure_flow_compiled(text, jsonb, boolean) overload is absent after migration.
  • Production mismatches fail without deleting data.
  • Existing automatic local recompilation behavior remains covered and unchanged.
  • Current versioning docs require a new concrete slug for structural and undetectable semantic changes.
  • Existing-user migration instructions cover ControlPlane files, configuration, scripts, worker imports, and historical migrations.
  • Release notes identify every removed public surface.
  • The stable Persist physical queue identity for flow tasks #650 + Add private per-step queues with typed step workers #651 release contains no migration-based flow compiler.

Out of scope

  • Worker drain or quiescence redesign.
  • Worker restart-state or worker_functions.enabled changes.
  • Startup-failure or shape-mismatch persistence.
  • Stable aliases.
  • Per-step queue implementation.
  • Custom or shared queues.
  • Changes to flow-shape semantics.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions