Conversation
…) (bd-ieldbghj)
`Math.source_info` spans the whole `$…$` / `$$…$$` node, but `text` is
not a plain substring of it once a math node spans lines: the reader
folds each soft break of inline math to a literal `\n` and strips the
block-continuation gutter (`> `, list indentation) from the interior
lines of display math. Consumers that parse `text` further (the coming
quarto-math crate, bd-entbg6x3) need to map any byte of it back to the
`.qmd`. This adds that mapping.
quarto-pandoc-types
- `Math.text_source: Option<SourceInfo>`: length == `text.len()`,
`map_offset(i)` lands on the byte `text[i]` came from. `None` means no
mapping finer than `source_info` is known.
pampa reader
- `location::provenance_builder_with_context`: a ProvenanceBuilder
rooted like `node_source_info_with_context` (in_parent under an
embedded re-parse, in_file otherwise).
- inline math: text segments verbatim; a soft break whose bytes are
exactly `\n` verbatim; any other soft break (gutter, CRLF) a
`replacement(range, 1)` so the folded newline maps to the source line
ending.
- display math: first line and every `\n` verbatim; each stripped
continuation gutter a zero-length replacement (deletion). Behavior of
the strip itself is unchanged (bd-q6ed / bd-qpa2 still own it).
JSON wire
- new `textS` sidecar (bare pool ref) after `t`, emitted by both the
Value and the streaming writer, read back by the reader; absent or
null reads as `None`. Documented next to `a`/`targetS` in
claude-notes/designs/raw-json-format.md.
Filters and transforms
- Lua: assigning `.text` clears the mapping; `pandoc.Math()` builds
with `None`.
- crossref_render: appending `\tag{N}` keeps the mapping by
concatenating a zero-width synthesized piece at the node end.
- every other constructor (tests, generators, equation label, math_js,
JSON reader) passes `None`.
Tests
- crates/pampa/tests/integration/math_text_source.rs (15): single- and
multi-line inline math, blockquote and list gutters, display math
plain / quoted / nested-quoted / listed / labeled / inside strong,
several nodes per paragraph, JSON round trip of `textS`, JSON without
the sidecar reads as `None`, and a sweep over every in-tree `.qmd`
with math (21 files, 36 nodes) asserting each text byte maps inside
the node span to the identical source byte unless it is a folded
newline.
- quarto-core crossref_render: `equation_tag_extends_text_source_
instead_of_dropping_it`.
- test_location_health now validates `text_source` alongside
`source_info`.
- `textS` added to every source-key strip list: Rust
`remove_location_fields`; TS `stripSourceInfoFields` (+ unit test),
`pampaOracle`, tiptap spike `canonical`/`pampa`; `textS?: number` on
the TS `Annotated_Inline_Math` type.
Snapshots (1 file updated, reviewed)
- crates/pampa/snapshots/json/math-with-attr.snap: each of the three
Math nodes gains a `textS` ref and the pool gains the matching body
ranges (e.g. 29..37 for `E = mc^2` inside `$E = mc^2$` at 28..38).
No other change.
- ts-packages/annotated-qmd/examples/academic-paper.json regenerated
with the live writer (guard test); only `textS` refs + pool rows
added.
Verified: cargo nextest run --workspace (14,032 passed) and full
cargo xtask verify (all steps passed).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prerequisite for the
quarto-mathcrate (bd-entbg6x3, planclaude-notes/plans/2026-09-21-quarto-math-and-native-docx.md, Phase 0). Strand: bd-ieldbghj.Opened as a draft on purpose: the branch is complete and verified, but we are holding the merge for a while (it will follow #704). Expected rebase conflicts against
mainonce #704 lands are three regenerable artifacts only:Cargo.lock,crates/pampa/snapshots/json/math-with-attr.snap,ts-packages/annotated-qmd/examples/academic-paper.json.Why
Math.source_infospans the whole$…$/$$…$$node, butMath.textis not a plain substring of it once a math node spans lines: the reader folds each soft break of inline math to a literal\nand strips the block-continuation gutter (>, list indentation) from the interior lines of display math. A consumer that parsestextfurther (the comingquarto-mathcrate) needs to map any byte of it back to the.qmd.What
Math.text_source: Option<SourceInfo>, length equal totext.len(),map_offset(i)lands on the bytetext[i]came from.Nonemeans no mapping finer thansource_infois known.location::provenance_builder_with_context(aProvenanceBuilderrooted the waynode_source_info_with_contextroots node spans). Inline math: text segments verbatim, a bare\nsoft break verbatim, any other soft break (gutter, CRLF) a replacement so the folded newline maps to the source line ending. Display math: first line and every\nverbatim, each stripped gutter a zero-length replacement. The strip behavior itself is unchanged (bd-q6ed / bd-qpa2 still own it).textSsidecar (bare pool ref) aftert, emitted by both theValueand the streaming writer, read back by the reader; absent/null reads asNone. Documented next toa/targetSinclaude-notes/designs/raw-json-format.md..textclears the mapping;pandoc.Math()builds withNone;crossref_renderkeeps the mapping when appending\tag{N}by concatenating a zero-width synthesized piece; every other constructor passesNone.textSadded to Rustremove_location_fieldsand the TSstripSourceInfoFields(+ unit test),pampaOracle, tiptap spikecanonical/pampa;textS?: numberon the TSAnnotated_Inline_Mathtype.Tests
crates/pampa/tests/integration/math_text_source.rs(15): single- and multi-line inline math, blockquote and list gutters, display math plain / quoted / nested-quoted / listed / labeled / inside strong, several nodes per paragraph, JSON round trip oftextS, JSON without the sidecar reads asNone, and a sweep over every in-tree.qmdwith math (21 files, 36 nodes) asserting each text byte maps inside the node span to the identical source byte unless it is a folded newline.crossref_render:equation_tag_extends_text_source_instead_of_dropping_it.test_location_healthvalidatestext_sourcealongsidesource_info.Snapshot changes (reviewed)
crates/pampa/snapshots/json/math-with-attr.snap(1 file): each of the threeMathnodes gains atextSref and the pool gains the matching body ranges (e.g.29..37forE = mc^2inside$E = mc^2$at28..38). Nothing else changed.ts-packages/annotated-qmd/examples/academic-paper.jsonregenerated with the livepampa -t jsonbinary (guard test); onlytextSrefs and pool rows added.Verification
cargo nextest run --workspace(14,032 passed) and fullcargo xtask verify(all steps passed) on this branch.🤖 Generated with Claude Code