Repository navigation
Open
Description
Problem
Shared sources compiled for a profile, such as expression utils, profile objects and Java getter adapters, are kept loaded in the presentation compiler (ScexPresentationCompiler.compile calls askLoadedTyped(sourceFile, keepLoaded = true, …)). Because of how scala.tools.nsc.interactive.Global works, every completion or getErrors request makes all of them stale. The background compiler then re-parses, re-enters and re-typechecks every shared unit, even though they never change after they are compiled.
How a single request causes it
askLoadedTyped(expressionSource)loads a new source, which callsreloadSourcesand thennewTyperRun()andminRunId = currentRunId. Every loaded shared unit is now out of date.- The expression is typechecked, and
outOfDatetriggers abackgroundCompilepass over all loaded units. That pass runsreset(unit)(clearingsynthetics,contexts, …), thenparseAndEnter, thentypeCheck. askFilesDeleted(expressionSource)at the end of the request does not start a new run, but it callsdemandNewCompilerRun(). If a background pass is running, that throwsFreshRunReqand aborts it partway through, often while a shared unit is half-typechecked.- The next request repeats steps 1–3.
Why it matters
- Wasted work: every request re-typechecks every shared unit of every profile used so far. The cost grows with the number of profiles and with adapter or utils size.
- Races: it keeps creating half-reset and half-typechecked states, and scex's multi-step requests can see them. Two known bugs come from this:
- Fix flaky presentation compiler crash on lazy accessor completion #190: an interrupt run during parsing forced an accessor whose synthetic tree had already been cleared (
no synthetic tree found for bean accessor). This is fixed by not polling during parse/enter, but the churn that exposes it remains. - Completer.getErrors gives quite a lot of false positives #17:
backgroundCompilestarts every pass withreporter.reset(), which wipes errors that scex already collected for the expression. This is still open.
- Fix flaky presentation compiler crash on lazy accessor completion #190: an interrupt run during parsing forced an accessor whose synthetic tree had already been cleared (
- The reports that disabling getter adapters or caches "fixes" these bugs are a side effect of this churn. Those settings change how many shared units are loaded, and therefore how long each pass takes.
Constraint
The presentation compiler currently has no other way to see shared symbols:
- It is created with plain
settings(presentationCompilerSettings).ClassfileReusingScexCompileronly adds the classfile directory to the regular compiler's settings (compilerSettings). - Without
-SCEXclassfile-directory, shared classes are written to an in-memoryVirtualDirectory(sharedClassLoader) that is not on any classpath.
So simply not keeping shared sources loaded would break completion.
Possible directions
- Put shared classfiles on the presentation compiler's classpath and stop loading shared sources. This is the only option that removes the churn completely.
- Needs a classpath entry for the in-memory output, e.g.
VirtualDirectoryClassPath. - The presentation compiler caches package contents, so newly compiled shared classes must become visible: invalidate the classpath entry or rescan the package.
- Annotations that completion relies on (
@Documentation,@ParameterNames) must survive the round-trip through pickles.
- Needs a classpath entry for the in-memory output, e.g.
- Reduce aborts without changing what is loaded. For example, delete expression sources lazily or in batches, or reuse a single expression source. This removes the extra
FreshRunReqfromaskFilesDeleted, but every new expression still needsreloadSources, which starts a new run. Shared units would still be re-typechecked on every request, so this only shortens the race windows. - Make each scex request one step from the presentation compiler's point of view. Do typecheck, scope/type completion and error collection inside a single work item, so a background pass can't run in between. This addresses the races, including Completer.getErrors gives quite a lot of false positives #17, but not the wasted work.
Option 1 looks like the real fix. Option 3 is a complementary hardening step.
🤖 Written by Claude.
Metadata
Metadata
Assignees
Labels
No labels
Activity