feat(repl): analyze each loaded file as a workspace document of its own - #309
devin-ai-integration[bot] wants to merge 32 commits into
Conversation
Files loaded from the command line or by %load were joined into the transcript document, so a root-level import in one file served the others and two files declaring one root package were reported as duplicates. Each loaded file is now a workspace document under its own name, indexed with the others and analyzed on its own, as the editor and the corpus gates analyze it; the typed transcript stays one joined document. A differential test runs every multi-file directory of the fixtures and the OMG corpora through the command line and a workspace and asserts the same diagnostics. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…pt alone Co-Authored-By: jason.han <hanhuijun@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…ing skill Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
A loaded file is analyzed as a document of its own, so an error in it gates that file's deeper checks only. The blocker note on a clean prompt submission now skips diagnostics from loaded files, and a load's from the transcript. Co-Authored-By: jason.han <hanhuijun@gmail.com>
A load shares no document with the rest of the buffer, so nothing blocks it and it neither names nor forgets the error the transcript has already been told of. Co-Authored-By: jason.han <hanhuijun@gmail.com>
… interval A load still names no blocker, but when it leaves the transcript unblocked the recorded note is cleared, so the error is named again should a reload bring it back. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md
… document is left A file reloaded with its enclosure left open is masked and its workspace document removed; with no scoped document left, symbolIndex returned before taking the file's previous declarations back out of the session index, so a qualified lookup kept answering with what the session no longer held. The empty-document path now drops every indexed document, as a reset does, and keeps the standard library. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
Co-Authored-By: jason.han <hanhuijun@gmail.com>
… load order Two files declaring the same root package are ordered as the workspace orders documents, by name, so which declaration a reference reaches does not depend on the order the files were given in. A test pins both orders. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…er-file loading Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
…ocuments Co-Authored-By: jason.han <hanhuijun@gmail.com> # Conflicts: # README.md # docs/project/spec-compliance.md
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if typed, found := s.transcript(); found { | ||
| s.ws.Open(docName, []byte(typed), s.version) | ||
| } else { | ||
| s.ws.Remove(docName) | ||
| } | ||
| live := make(map[string]bool, len(s.snippets)) | ||
| for _, sn := range s.snippets { | ||
| if sn.origin == "" || sn.open { | ||
| continue | ||
| } | ||
| live[sn.origin] = true | ||
| if doc := s.ws.Document(sn.origin); doc == nil || doc.Version != sn.gen { | ||
| s.ws.Open(sn.origin, []byte(sn.src), sn.gen) |
There was a problem hiding this comment.
🟡 Transcript-name file drops declarations
A loaded file named <repl> shares openDocuments' transcript key. The next load or prompt submission overwrites one document, making valid declarations unreachable.
Learn more
The workspace identifies documents by name. The transcript always uses <repl>, while loaded files now use their path directly. A user can create a file literally named <repl> and load it with %load "<repl>". On the next submission, openDocuments opens both sources under the same key, so the second open replaces the first.
Example: Load %load "<repl>", then type package Typed { part def A; }. The loaded file's later open replaces the transcript document, so Typed::A no longer resolves.
Recommended fix: Give loaded documents collision-free internal workspace keys while retaining their original paths for diagnostics and display. Apply the mapping consistently to snippets, symbol DocName values, index entries, lookups, source registration, and document removal.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Same finding as the earlier thread on these lines (#309 (comment)): confirmed, reachable only through a SourceFile.Name that is the literal string <repl> (every loader passes the path the file was read from; standard input is <stdin>), so on the command line it takes a file literally named <repl> in the working directory.
Closing it changes the SourceFile contract rather than the per-file migration, so it stays open for a maintainer decision between (1) documenting <repl> as the transcript's reserved name, (2) refusing a load under that name at LoadPaths/SubmitFiles, or (3) keying loaded documents apart from their names with a display name mapped back everywhere sym.DocName is shown (%explain, the debuggers, file:line:col). Left unresolved until then.
What and why
Files loaded from the command line (
-validate,-satisfy,-check,-run, …) or by%loadwere joined into the transcript document (<repl>, with a synthetic<repl>.kermlfor KerML files) and analyzed as one text. That buffer had semantics of its own: a root-level import in one file served every other file, and two files declaring the same root package were reported as a duplicate. The editor, the corpus gates and the pilot tools analyze the same files throughmodel.Workspace, one document per file, and report the opposite —unresolved reference: Realin the file that does not import it, and no duplicate.Each loaded file is now a workspace document under its own name, indexed together with the others and analyzed on its own; typed submissions still form the one joined transcript document. Concretely, in
internal/repl:Readers of
s.ws.Document(docName)that assumed every declaration lived in<repl>—%calc,%sweep,%analyze,%print, the compound-expression path of%eval,promptScope,symbolsInLoadOrder(%view) — now go throughhasDeclarations()/sessionDocs()/sessionMembers()/rootScopeOf(sym); nothing re-joins the files behind the scenes.%printcan print a declaration of any loaded file rather than only of<repl>/<repl>.kerml.%saveis unchanged: it writes each snippet's own text back.Two user-visible changes for multi-file loads, both the behavior
model.Workspacealready had:%load a.sysml— the text typed at the prompt. Root packages remain reachable through the global namespace as before, and the prompt still evaluates in the last namespace declared, which may be one a loaded file declares.package Aare two root namespaces of that name, not a duplicate; a reference toAresolves to the declaration in the file whose name sorts first — the order the workspace gives documents, whatever order the files were given in on the command line (TestRepeatedRootPackageResolvesByDocumentNameNotLoadOrderpins both orders).The compound-expression path of
%evalparses the expression appended to the transcript with the loaded files masked out (as the transcript document has them), so a loaded file's text can no longer fail the parse of a prompt expression; the expression is still evaluated inpromptScope().This is the first of two PRs: the parallel batch pipeline over these documents (parallel parse, one index and one wildcard expansion per batch, a worker pool with a private
passes.Contextper worker, diagnostics in document order, a workers setting, the-split-planesgenerator mode and the measurements) follows in #312. Merge order: this PR first, then #312.Existing
internal/repltests were examined individually; none asserted the import leak or the duplicate, so none was changed.TestTargetlessPseudoViewSpansLoadedDocumentsandTestLoadFilesSummaryMatchesLoadingEachFilefailed during the migration and were fixed in the code (symbolsInLoadOrdersorting by buffer offset;Result.ownMembersfiltering by the buffer offset), not in the tests.Known behaviour
Two prompt evaluation rules predate this change and are kept as they are (documented under "Known behaviour" in the loading section of
docs/guide/04-repl.md); both are open to change. A prompt expression evaluates in the last namespace declared (promptScope), so when a loaded file declares it, the expression sees that file's root imports although a typed declaration does not —%eval 1.5 as Realresolves after loadingprivate import ScalarValues::*; package A { … }while a typedattribute y : Real;reportsRealunresolved; the alternative is a fallback to the transcript's own root. A qualified command argument (%eval A::y,%print A::y) goes through the single index-basedlookupSymbol, which holds every document's declarations, so with two loadedpackage Ait reaches the secondA's member that a model reference or a compound expression cannot; the alternatives are a resolver-based lookup for the evaluating commands, or rejecting a root that resolves ambiguously. Both were already so ondevelop(the second for a.kerml/.sysmlpair); per-file loading makes them reachable with any two files.Specification basis
KerML 8.2.3.3 and 8.2.4: a root-level
importsurfaces its names in the importing document's own root namespace (docs/project/spec-compliance.md, row "A root-levelimport X::*surfaces its names in the importing document's own root namespace"). KerML 8.2.3.5: two root namespaces of the same name are neither an ambiguity nor a duplicate, and resolution uses the first declaration (docs/project/pilot-xpect.md, "The global namespace…", item 1;TestNameResolutionPassResolvesARepeatedTopLevelNameToTheFirst). No compliance row moves; the command line converges on what those rows already certify for the workspace.How it was verified
New in
internal/repl/filedocs_test.go:TestLoadedFilesDoNotShareRootImports—a.sysmlwithprivate import ScalarValues::*;,b.sysmlusingReal: the command line reportsunresolved reference: Realinb.sysml, and its diagnostics equalmodel.Workspace's over the same files.TestLoadedFilesDeclaringOneRootPackageAreNotDuplicates— twopackage Afiles plus a file referencingA::X(first declaration) andA::Y(second): no duplicate,A::Xresolves, diagnostics equal the workspace's.TestPromptDoesNotSeeALoadedFilesRootImports— after%load a.sysml, typed text still reachesAbut not a bareReal.TestCommandLineLoadMatchesWorkspace— every multi-file directory undertestdata/,examples/and the four OMG corpus roots (examples/sysml-v2-training,examples/pilot-corpora/{kerml-examples,sysml-examples,sysml-validation}) run through the command-line load and throughmodel.Workspaceyields the same diagnostics. All four fail ondevelop.Gates, all run locally:
training_examples_expected.txtis unchanged and empty; no pilot ratchet moved.By hand:
sysml -validate a.sysml b.sysmlon the import case (unresolved reference: Realinb.sysml, exit 2) and on the same-name-package case (clean; the existing note thatAis opened by more than one loaded file still prints), and%loadof several files followed by%print,%view,%evalof a compound expression and%save(each file's text written back verbatim). The REPL's own name lookup (%print A,%view A) still reports a root name two loaded files declare as ambiguous, as it did before; model resolution (A::X) takes the first declaration.Checklist
make testandmake lintpass locallydocs/reference/cli.md"Multiple Files",docs/guide/04-repl.mdloading section)changes/unreleased/per-file-documents.changed.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved