Add the VSCode migration case study

[ulfjack]

A reverse-engineered case study of ulfjack/vscode@vscode-with-bazel (7 commits on vscode 1.132.0: 4 custom rules, 8 tool scripts, 44 targets, zero bazel_deps). Re-run from a clean checkout for this document, so every number is reproduced rather than recalled; the handful of claims read off code rather than observed are marked inferred.

The organizing fact is that vscode's build is bespoke — gulp streams, gulp-tsb, esbuild driven from JS — so there is no compile_commands.json, no File API, no reactor. Nothing to extract. The model has to be captured by instrumenting the build as it runs, and the document follows the consequences.

Results. npm reference build green in 8.4 min; bazel build //... 44 targets green; 7722 npm actions captured; 7710 of 7710 common .js files byte-identical, 0 differing; all four NLS metadata files identical, including a 2.3 MB nls.metadata.json.

Findings, worst-first.

1. NLS rewriting assigns a global monotonic counter in file-traversal order (upstream says so in capitals at build/lib/nls.ts:24). Reversing one sort comparator in the ported transform makes 1903 of 7710 .js files differ and the build still reports success. A traversal-order-dependent step has no flags, so an action-graph diff is blind to it by construction — only byte comparison catches it. compilation.ts:45 is a second instance: it computes emitted line endings by reading its own file's bytes.
2. The source tree is a build output. codicon.ttf is gitignored yet lives inside src/, copied there from node_modules by npm postinstall — so glob(["src/**"]) is an undeclared dependency on the original build. Only visible because I ran npm ci --ignore-scripts.
3. node_modules is three problems at once: it is the toolchain (tsc.js, plus a second npm project under build/ for esbuild); it ships a foreign BUILD.bazel + WORKSPACE (cpu-features) that breaks bazel build //...; and it's where the runtime looks, so the Bazel app tree needs the npm tree overlaid. I measured that .bazelignore is the wrong fix (it hides the toolchain) and --deleted_packages is the right one — the same conclusion as Dolphin §8's submodule case, from the opposite direction.
4. Capture-based extraction records the build system's own compiler use: file.ts appears 45 times as a synthetic in-memory filename from nls-analysis.ts. No declared-graph model could contain it.
5. In-process interception on modern Node must happen at module-resolution time. The preload's comments record three failed approaches (post-hoc module.exports mutation, Module._cache swapping, Proxy) that any capture-based JS frontend will hit.
6. node-gyp actions ask host questions: run_nodegyp.mjs falls back to npx --yes node-gyp, which downloads from the npm registry inside a build action (observed), and native-keymap's binding.gyp shells out to pkg-config for x11.
7. Two claims that are wrong as committed: MODULE.bazel says Electron is "pinned to what package.json requests" but pins 42.2.0 against package.json's 42.7.0 (verified by running the fetched binary), and compiles all 9 native modules against that ABI; and both launcher scripts compute the repo root one level too high, so they have never worked as committed.

Scope: the core TS compile, 23 bundles, 9 native modules, a runnable app tree. Extensions, tests, web/REH/CLI and packaging are out, so nothing here claims what those would cost.

The appendix reproduces every number, including the prerequisites that are themselves findings (node 24.18.0 per .nvmrc, npm ci at both the root and build/, node-gyp on PATH, x11/xkbfile/secret/krb5 dev packages).

Rebased onto main after #8, #10, #11, #12 and #14 merged; 126 tests pass. Dropped a sentence that described Ladybird as an incomplete migration, since main's Ladybird case study now records a Bazel-built browser that renders.

[anakanemison]

I think this is a good example of the desirability of modifying the project while migrating the build system. That the old build system’s output is a function of a global file ordering seems limiting of how “idiomatic” any bazel migration could get while preserving action and output equivalence.

I left a comment in the case doc where I thought the report could be less ambiguous.

[anakanemison]

I'm not sure how to interpret what the LLM wrote here from this text alone. It seems like it's describing a problem it encountered during migration, caused by the admittedly weird behavior in the old build system described here. Did it ... solve this problem, and is this leaving a breadcrumb trail saying just that it was a problem that needed solving? ... or did it bypass the problem by running the referred-to non-Bazel step first (either because there was no good solution, or because it didn't try hard enough to find one)?

I should be able to answer this question for myself by reading the commits at ulfjack/vscode@vscode-with-bazel and/or checking that branch out and experimenting with it. But it would be nice if this report didn't require me to do that, to understand it fully.

[ulfjack]

It complains about legacy build systems putting files into the source tree, and then getting picked up by bazel's glob. I believe this got fixed, but I haven't fully reproduced the bazel build on my local machine.