Support data URLs in the fetch polyfill - #212
matthargett wants to merge 22 commits into
Conversation
Vendor the ES5 web-streams-polyfill 4.3.0 ponyfill and expose an idempotent Streams initializer that preserves constructors supplied by the selected JavaScript engine. Cover readable, writable, transform, BYOB, error, tee, subclassing, and host-constructor behavior with focused ports from WPT plus Firefox and Chromium regression tests. Validate the implementation on JavaScriptCore under ASan/UBSan and QuickJS Release.
Implement iterable BlobPart conversion, MIME and endings normalization, zero-copy Blob composition and slicing, and lazy 64 KiB byte streams. Delegate File streaming and slicing to its backing Blob while preserving browser class identity.\n\nUse immutable shared segments so nested Blob and slice construction do not duplicate payload bytes. Add focused WPT, WebKit, and Firefox regression coverage for constructor ordering, iterator closure, UTF-8 decoding, BYOB reads, cancellation, lifetime safety, and large nested streams.
Replace the ad hoc buffered fetch response object with standard-shaped Headers and Response classes backed by ReadableStream bodies. Preserve host constructors, normalize and validate header values, enforce single-use bodies, and expose native responses with one transport-to-JavaScript copy. Reuse normalized header views between mutations, compact header storage in place, retain stream chunks only until consumption, and allocate a contiguous body result at most once when required. Add focused WPT, WebKit, Firefox, and Chromium regression coverage plus JavaScriptCore/QuickJS and initialization tests.
Close byte streams for zero-length BufferSource bodies without enqueueing an invalid empty chunk. Preserve the non-null body and bodyUsed semantics, and add focused regression coverage alongside the WPT empty-response cases.
Resolve percent-encoded and base64 data URLs without entering the network transport. Use a validating WHATWG-style forgiving-base64 decoder, strip URL fragments from payloads, and preserve response metadata through the standard Response path. Decode into pre-sized native storage and copy the completed body once into JavaScript-owned memory. Cover representative WPT data URL and forgiving-base64 vectors, plus Chromium whitespace cases.
Feed percent-decoded bytes directly into validating and decoding passes for Base64 data URLs. Allocate the final native payload at its exact size, avoiding the previous percent-decoded staging vector and any vector growth before the single JavaScript ownership copy.
Replace a partial or null host Streams surface as a complete constructor suite so stream products retain compatible instanceof relationships. Preserve a complete suite on repeated initialization.\n\nHarden the native initialization test so N-API failures complete the test promise instead of hanging, and cover partial host replacement plus null handling and cross-constructor compatibility.
Use ReadableByteStreamController.byobRequest views directly and respond with the produced byte count instead of allocating and enqueueing a 64 KiB intermediate chunk. Keep the existing bounded allocation path for default readers.\n\nAdd WPT-derived coverage for small offset BYOB views that cross immutable Blob segment boundaries.
Observe standard ReadableStream consumption paths with weak per-stream state instead of depending on web-streams-polyfill's private _disturbed field. Instrument stream readers and piping once during Fetch initialization, preserve stream identity, and cache the initialized Fetch implementation so repeated initialization does not stack wrappers or replace its internal state. Treat Headers and Response as an implementation pair when either host global is missing or null. Harden native initialization tests against unhandled N-API failures and add WPT-derived coverage for reads, cancellation, piping, and streams disturbed before Response construction.
dac2d8c to
517b298
Compare
…nto the Android test host MSVC caps a single string literal at 16,380 bytes (C2026) and a concatenated one at 65,535; the 71 KB bundle was emitted as two 60,000/11,188-byte raw strings, so every Windows and UWP job failed at "Build Solution". CMake now emits consecutive parts of at most 16,000 bytes, abutting with no separator (the split stays byte-transparent), and a variadic consteval Join reassembles them into the same std::array as before. The desktop UnitTests target links Streams for the Shared.cpp coverage but the Android UnitTestsJNI host did not, so the Android build failed on the include.
b4a248d to
242dccf
Compare
… stub breaks the UWP build)
…nitizer job The ASan/UBSan job failed on a 24-byte "direct leak" allocated inside libjavascriptcoregtk (WTF::BitVector::resizeOutOfLine under JSObjectMake -> JSObject::setPrototypeDirect), reached from napi reference creation. It is JavaScriptCore's own global bookkeeping, never freed at exit, and surfaces for whichever caller happens to trigger the growth -- the Blob stream work already documented the same artifact. Mirrors the existing tsan_suppressions.txt wiring.
… out on iOS simulator / Hermes)
|
Rebased onto This is the first time the branch has had Windows/UWP, JSI, iOS-simulator and Android CI (upstream runs are behind the fork-workflow gate; the fork twin runs the full 24-job matrix), which surfaced a few portability items now fixed on the branch: the embedded Streams bundle split into ≤16 KB literals (MSVC C2026), |
…e globalThis QuickJS guards JS recursion against stack_top - JS_DEFAULT_STACK_SIZE, which is 1 MiB in quickjs-ng -- also the default size of a non-main thread on Android and Windows. On those threads the check sits below the guard page, so deep recursion faults before QuickJS can raise "InternalError: stack overflow": Android_QuickJS died with SIGSEGV (one frame repeated 150+ deep) in the Fetch polyfill's 40k-chunk Response test, while the 8 MiB-stacked desktop QuickJS jobs were fine. Derive the limit from the running thread (pthread_getattr_np / GetCurrentThreadStackLimits) minus a margin for native frames. The Windows 10 Chakra predates ES2020 and has no `globalThis`; every Fetch test failed there with "ReferenceError: 'globalThis' is not defined". Define it on the global object at env attach, as a plain writable configurable property.
…resolver BabylonJS#212 decoded data: URLs inside the fetch polyfill only, so XMLHttpRequest -- the path Babylon.js' asset and texture loaders take -- and every other UrlLib consumer still treated them as network failures (BabylonJS#67). The WHATWG data: URL processor is lifted out of Fetch.cpp into Polyfills/URL (a URLInternal seam, like BlobInternal) and registered with UrlLib::UrlRequest::RegisterSchemeResolver("data", ...) next to blob:, exactly the shape BabylonJS#207 established. A malformed data: URL is a network error per spec, which UrlLib expresses as `handled == false`. fetch keeps its synchronous fast path on the shared decoder. Tests cover XMLHttpRequest text / base64 arraybuffer / malformed, a data: -> Blob -> object URL round trip, and a base64 gzip data: URL piped through DecompressionStream (skipped where the compression polyfill is not present).
…resolver BabylonJS#212 decoded data: URLs inside the fetch polyfill only, so XMLHttpRequest -- the path Babylon.js' asset and texture loaders take -- and every other UrlLib consumer still treated them as network failures (BabylonJS#67). The WHATWG data: URL processor is lifted out of Fetch.cpp into Polyfills/URL (a URLInternal seam, like BlobInternal) and registered with UrlLib::UrlRequest::RegisterSchemeResolver("data", ...) next to blob:, exactly the shape BabylonJS#207 established. A malformed data: URL is a network error per spec, which UrlLib expresses as `handled == false`. fetch keeps its synchronous fast path on the shared decoder. Tests cover XMLHttpRequest text / base64 arraybuffer / malformed, a data: -> Blob -> object URL round trip, and a base64 gzip data: URL piped through DecompressionStream (skipped where the compression polyfill is not present). (cherry picked from commit f30a41c)
|
Following up on #67: the branch now carries the generic path @bkaradzic-microsoft described — the WHATWG Tests: XMLHttpRequest text / base64→ArrayBuffer / malformed→status 0, a |
…allback stack limit by the thread's own size, define globalThis non-enumerable via JsDefineProperty (cherry picked from commit fef8cb9)
(cherry picked from commit 5cd790d)
…o the thread's base (the nominal size under-budgeted the Android runtime thread: depth-128 recursion hit the limit) (cherry picked from commit d1ca32e)
…es the measured-limit approach; emulator-validated depth-128)
Summary
data:URLs locally throughfetchResponsemetadata and body-consumption behaviorDepends on #210.
Memory
Percent decoding reserves once. Base64 URLs stream percent-decoded bytes through validation and decoding passes without materializing an intermediate buffer, then allocate the exact native payload size. The completed body follows the shared fetch response path and is copied once into JavaScript-owned storage.
Coverage
Focused cases are adapted from WPT fetch/data-urls and Chromium's DataURL tests. They cover default and explicit media types, fragment stripping, case-insensitive
base64, ASCII whitespace, missing padding, byte-preserving percent decoding, and invalid alphabet/padding rejection.Validated with 272 JavaScript cases plus 8 native tests on: