Support docx, pptx, epub, and typst output via Pandoc writers - #704
Merged
Merged
Conversation
…ic into P1-P8 Freezes the architecture for reusing Pandoc's own writers plus vendored Q1 Lua filters to produce non-HTML formats (docx/pptx first, latex stub), fed from Q2's existing custom-node wire format. Decomposes the epic into eight phases (P1 neutral-core, P2 wire-schema, P3 upstream-crossref, P4 run-machinery, P5 lua-shim, P6 numbering-wiring, P7 format-tail, P8 content-hidden) after a 3-agent Opus review round and several rounds of design decisions.
…utral-core rendering Adds FormatIdentifier::Pptx and a PipelineProfile seam so the pipeline can self-gate transforms that only make sense for HTML (panel-tabset, footnotes, example-embed iframes) when targeting Pandoc-native writers. Splits FootnotesTransform into a neutral B1 (footnotes) and HTML-only B2-B4 (footnotes-resolve) so citation numbering stays shared. Backed by a byte-identity corpus/diff harness proving the HTML render path is unchanged.
…ests Publishes a JSON schema for Q2's custom Pandoc nodes (Proof, CrossrefResolvedRef, Callout, Equation, FloatRefTarget, Theorem, Tabset) as the contract between the Rust writer and the Lua shim, with Rust and TypeScript conformance tests keeping both sides honest. Adds the cite_prefix/cite_mode/label_upper slots and plain_data.order field the epic's numbering feature depends on.
Vendors Q1 pandoc Lua filters at v1.11.3 (with a lint rule pinning the version), builds a QUARTO_FILTER_PARAMS env-block codec with a Windows-safe predicate, and adds PandocWriteStage/render_qmd_to_pandoc to invoke pandoc with the vendored main.lua shim splice. Reserves error-catalog subsystem 18 for pandoc diagnostics and wires unconditional stderr capture (Q-11-1/Q-18-3) so filter failures surface as diagnostics instead of silent pandoc exit codes.
Ports Q2's crossref-numbering patch (assign-numbers / crossref_present()) into the filter tree vendored in P4, with a cross-product matrix test covering the q2/L tier. Most of P3's work was audit and design-decision refinement, already folded into the epic's design-freeze commit; this is the phase's actual code change.
…om P7 After P4 lands, drafts the typst and epub follow-on plans through a five-revision review pass, and splits the format-agnostic Pandoc CLI plumbing out of P7 into its own P7-foundation phase so docx/pptx/epub/typst can all build on shared groundwork.
…rgets Verifies ConditionalContentTransform behaves correctly for non-HTML formats and fixes it to recurse into Inline::Custom slots — previously, conditional content nested inside a custom node was invisible to the transform.
…type routes Implements quarto2-shim.lua's recognizer/sanitizer/slot-collector/route-table skeleton, then Route R (Theorem, Proof, FloatRefTarget, Callout, Tabset) and Route N (CrossrefResolvedRef, Equation) handlers, with per-type golden tests and a Layer-1 contract test. A post-review fix wave closes two crash bugs, a dropped cite_prefix slot, a Callout field-type mismatch, and derives the shim's route table mechanically from route_handlers instead of hand-duplicating it.
Wires crossref category/numbering params through to the Lua filters (custom-passthrough, external-mode matrix, number-parity tests) and calibrates the pandoc oracle-comparison ceiling to pandoc 3.11.
…Pandoc targets Adds Q-18-8 (multi-format render warning) and a project-mode containment gate, and relaxes the CLI format gate so docx/pptx — and later epub/typst — can actually be requested. This is the shared plumbing every later format-specific phase builds on.
… mapping Relaxes the format gate for epub, ports Q1's epub pandoc-defaults and CSS includes, maps epub-chapter-level to pandoc's --split-level, and allow-lists epub-cover-image/metadata/embed-font/CSS keys.
Settles on the golden-test methodology for epub, records an end-to-end verification pass, and a full cargo xtask verify run.
Symptom: q2-preview's title block was entirely missing for ~97/134
dom-parity-opted-in smoke-all fixtures (render.norm.txt had the full
<header class="quarto-title-block"> block; preview.norm.txt had none),
even though PreviewTitleBlock.tsx and its `rendered.has-title-block`
gate have existed for a long time.
Root cause: bf8cd45dc ("P4 Task 9: PandocWriteStage, render_qmd_to_pandoc,
and a real pampa JSON-writer bug fix", plan
claude-notes/plans/2026-09-18-pandoc-hybrid-P4-implementation.md) fixed
pampa's JSON writer so a nested `MetaMap.c` serializes as a genuine JSON
object under `JsonConfig { raw: false }` (crates/pampa/src/writers/json.rs
stream_write_config_value) -- needed so real `pandoc`'s JSON reader, fed
for the first time by the new docx/pptx/epub path, would accept it instead
of pampa's own `{key, key_source, value}`-triple array. That fix correctly
updated the Rust-side writer and its own reader/tests, but never touched
the OTHER consumer of the same non-raw shape: `getMetaPath`
(ts-packages/preview-renderer/src/framework/meta.ts), which still branched
on `Array.isArray(m.c)` for every nested-MetaMap step and silently returned
`undefined` once that stopped matching -- breaking every multi-segment
metadata path (`rendered.has-title-block`, `rendered.navigation.*`, etc.)
uniformly, hence the title block vanishing everywhere at once. No existing
getMetaPath test caught it because its own fixture hard-coded the same
stale array shape.
Fix: read MetaMap.c as a plain object in getMetaPath's nested-step branch,
matching the writer's actual (and now-permanent) non-raw output shape.
Rewrote meta.test.ts's fixture to the same shape; confirmed RED against
the old implementation first (3 failures, all in the nested-path tests),
then GREEN after the fix (23/23).
Verified:
- ts-packages/preview-renderer full suite: 588 passed, 36 skipped
(pre-existing), 0 failed
- hub-client `npm run test:wasm` (the originally-failing suite), both
SMOKE_FILTER=drafts/draft-banner and unfiltered: 24/24 files, 134/134
tests green (previously 23/24, 132/134, with the dom-parity test
reporting "97 parity failure(s)" plus the harness self-check failing)
- hub-client `npm run build:all` (tsc -b + vite build): exit 0
- Not verified in a live browser session (no browser tool available this
session); the wasm-driven harness above exercises the real freshly-built
WASM export + real React registry, which is the closest available
substitute.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… assertion Adds quarto-ooxml-extract, a shared semantic extractor for asserting on real docx/pptx XML, the invocation builder with a pandoc-forwarding allow-list and callout icons, per-format execute defaults, Meta-block coercion for docx/pptx, and a CI-runnable golden-snapshot assertion against 20 real-Quarto fixtures — which caught two real docx/pptx bugs (a multi-id crossref drop and a --output cwd resolution bug).
Closes P8 with an end-to-end smoke fixture proving content-hidden/when-format gating survives a real docx/pptx render, not just the unit-level transform test from the earlier P8 commit.
… brand bridge Wires pandoc's typst writer name, fixes Route N/R for typst's crossref-labeled equations, builds the standalone/wrap invocation builder and format-aware default image extension, wires section-numbering/shift-heading-level-by, bridges ResolvedBrand into the typst filter params, and vendors the typst template plus required packages/fonts (fontawesome, marginalia, octique, showybox, theorion).
Adds the pipeline stage that shells out to typst compile, closing out typst-gather bundling and font discovery so vendored fonts/packages resolve correctly at compile time.
…by follow-on breakage Adds the typst format docs page and a pandoc-bug audit after a full cargo xtask verify pass, folding in two fixes that pass surfaced: build_forwarded_args needed &str not String for to_format, and shift_heading_level_by's call sites needed updating for its new meta parameter, with an explicit-override regression test added.
Contributor
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
The epub writer commit (03471c9) added --math-method=mathml unconditionally, but --math-method was only introduced in pandoc 3.11 (it replaces the deprecated --mathml/--mathjax/--gladtex/--katex/--webtex flags). CI pins PANDOC_VERSION 3.10 (bumped from 3.8.3 by a320a97, earlier in this branch), so pandoc rejects the flag with "Unknown option --math-method", failing e2e_render_epub on every CI leg. Verified against the actual pandoc 3.10 binary locally: it lacks --math-method entirely. Also bumps every other place that records the pandoc pin in lockstep (enforced by the pandoc-pin-agreement lint): PANDOC_PIN, the Pandoc-hybrid dev-setup floor, the vendored-filters README, and the xtask verify preflight, plus the tests whose literals encoded the old 3.10 floor.
gordonwoodhull
force-pushed
the
feature/pandoc-writer-hybrid
branch
from
September 21, 2026 21:05
b2c66be to
630fe83
Compare
test_discover_available_typst_fonts_finds_vendored_font_awesome (and its module docs) assume a real typst binary on PATH, same convention as the existing pandoc assumption — but no "Set up Typst" step was ever added to either workflow, so it fails with "typst must be on PATH to run this test suite" on every CI run. Mirrors the existing pandoc install pattern (curl+tar on Linux, brew on macOS) in both test-suite.yml and ts-test-suite.yml.
…ntract
Four integration-test files (PreviewDocument, RevealDeck,
headroom-header, PreviewTitleBlock) built nested MetaMap fixtures as
an array of {key, value} (or {key, key_source, value}) triples, but
getMetaPath (framework/meta.ts) requires MetaMap.c to be a plain
object keyed by metadata key and explicitly rejects arrays — the same
shape mismatch bd-6zrmjidd already fixed once in the real consumer
path, per that function's own doc comment. Any nested rendered.*
lookup (rendered.navigation.navbar, rendered.reveal.footer, etc.)
silently resolved to undefined, so every chrome-injection assertion
in these files failed against a real null/empty DOM.
Fixes 68 failing tests (mm/metaMap helpers changed to build c as a
plain object; two inline literal duplicates in PreviewDocument's own
file rewritten to use the helper).
cscheid
added a commit
that referenced
this pull request
Sep 21, 2026
… trial-merge findings against the pandoc-hybrid leg (bd-entbg6x3) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
cscheid
added a commit
that referenced
this pull request
Sep 21, 2026
… the problems -> diagnostics mapping (bd-entbg6x3) Phase 1, diagnostics item, of claude-notes/plans/2026-09-21-quarto-math-and-native-docx.md. Subsystem 22 `math` (20 and 21 are taken by PR #704's pandoc and typst). - crates/quarto-error-catalog/error_catalog.json: Q-22-1 .. Q-22-13, one per normalize::ProblemKind — unknown command, unknown environment, mismatched environment, missing argument, unbalanced brace, unbalanced \left/\right, double script, macro expansion limit, unsupported command, ragged rows, ignored \limits, dropped construct, syntax error. - docs/errors/math/Q-22-*.qmd: a page per code (what / why / fix / where it is reported), status stub; docs/_quarto.yml gains the `math` sidebar section after `image`. `cargo xtask lint` enforces both. - quarto_math::diagnostics: code_for / title_for / severity_for / diagnostic_for / diagnostics. Errors stop the conversion (the seam falls back to verbatim TeX); Unsupported, RaggedRows, IgnoredLimits and Dropped are warnings. A diagnostic's location is the problem's span as a SourceInfo::substring of the math text's own SourceInfo (pampa's Math.text_source), so it resolves to the exact .qmd bytes; a span-less problem (the macro engine's synthesized arity error) points at the whole expression. - Tests (tests/integration/diagnostics.rs, 6): every kind has a distinct Q-22 code present in the live catalog with the same title, subsystem and docs_url; severities; code/title/message/hint on a built diagnostic; location arithmetic through a file-rooted SourceInfo; span-less fallback; warning path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Member
|
So cool @gordonwoodhull - Thank you! |
cscheid
pushed a commit
that referenced
this pull request
Sep 23, 2026
…) into feature/bd-entbg6x3-quarto-math Resolution: error_catalog.json and the errors sidebar in docs/_quarto.yml are the union of main's pandoc/typst (Q-20, Q-21) entries and this branch's math (Q-22) entries; Cargo.lock is main's with quarto-math's new packages added by cargo. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
cscheid
pushed a commit
that referenced
this pull request
Sep 23, 2026
…i2zkj) #704's exact surviving-name test landed after this branch was cut. The stage is a deliberate no-op for non-HTML formats (NumberEncoding::Writer) and still removes the quarto-eq-number attribute before pandoc runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
cscheid
pushed a commit
that referenced
this pull request
Sep 23, 2026
A document with html-math-method: mathml rendered --to docx crashed the
Pandoc leg: the stage converted every equation to a RawInline before
pandoc ran, and the vendored crossref/equations.lua died on it
('attempt to concatenate a nil value (field text)', exit 83, Q-20-3).
#704 landed after this branch was cut, so its exact Pandoc stage-list
test only reddened at the merge.
The key is an HTML option that commonly lives in shared metadata of a
project rendering docx or typst too, so it is ignored silently on the
Pandoc leg — like EquationNumberStage's Writer encoding and like Quarto
1, which forwards it to pandoc whose non-HTML writers ignore it.
- applies_to(format, method) gates the stage on is_html_based(); unit test
- math-ml joins PANDOC_STAGE_EXCLUDED; the exact surviving-name list
gains equation-number (a deliberate no-op there)
- end-to-end: a mathml-method document renders to docx with one native
OMML equation numbered (1) and no MathML in the body
- plan: claude-notes/plans/2026-09-23-merge-math-stack.md records the
stack's conflict assessment and merge order
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Quarto can now render
docx,pptx,epub, andtypstdocuments by handing its own document representation to Pandoc's native writers, rather than routing them through Quarto's own HTML-only rendering path. To do this, Quarto feeds its custom AST nodes — crossrefs, callouts, theorems, tabsets, floats — over a documented wire format to a vendored copy of Quarto 1's Lua filter pipeline, which turns them into the constructs each Pandoc writer understands.Trigger a format the same way as any other:
format: docx,format: pptx,format: epub, orformat: typstin front matter or_quarto.yml, or--to <format>on the command line.docx,pptx, andepubshell out to apandocbinary onPATH(or one pointed to byQUARTO_PANDOC);typstshells out the same way to atypstbinary (orQUARTO_TYPST) and compiles the intermediate.typsource to a PDF.What carries over from Quarto 1
@fig-,@tbl-,@eq-, and theorem-family refs resolve and number correctly in every new format. Typst numbers natively through its own compiler rather than through Quarto's numbering pass, so the two always agree.callout-note,.theorem, proofs, etc.) render with their format-appropriate styling.word/media/in docx).reference-doc/templatefor docx/pptx, and custom.typtemplates for typst, resolve the same way other Quarto paths do and fail with a clear diagnostic if the file is missing.brand.ymlcolors and typography apply to typst output the same way they apply to other formats.marginaliapackage, and automatic fetching of any additional Typst Universe package a document's raw Typst content references.epub-chapter-level,epub-cover-image,epub-metadata,epub-embed-font, and an additionalcssstylesheet merged with Quarto's own callout/layout styles.content-hidden/when-formatconditional content is honored for all of these formats, including content nested inside custom nodes.Rendering more than one format
If a document's front matter declares more than one
format:, Quarto still renders only one target per invocation — but it now warns, naming which format it used and which it skipped, instead of silently dropping the others.Known gaps
latex/pdfis not yet supported —--to pdfand--to latexstill refuse to render.Q-20-*codes cover a missingpandocbinary, a version too old, a nonzero pandoc exit (with its stderr/filter traceback attached), and a couple of internal wire-format consistency checks.