Repository navigation
perf(render): coalesce concurrent response cache misses - #592
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
timtofan
marked this pull request as ready for review
November 9, 2026 20:32
deniszh
approved these changes
Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Identical concurrent response-cache misses independently fetch, decode, evaluate and serialize the same render. Large overlapping requests can multiply frontend CPU, memory and backend load precisely when the frontend is under pressure. Sharing that work reduces avoidable load without changing render semantics.
Architecture
The HTTP render handler still parses the form and checks the response cache first. On a miss, a per-App flight table uses exactly
renderForm.cacheKey, the existing encoded response-cache key. The first caller executes the existing backend queues, decoder, evaluator and serializer once. Concurrent callers wait for those immutable result bytes. JSONP wrapping, response UUIDs, error rendering and access-log identity remain per caller. All render parameters already included in the cache key (including maxDataPoints, timezone and image options) remain distinct. No request header currently participates in this cache key or affects evaluation.The shared work has an independent global timeout and retains context values. Cancelling the first caller only ends that caller's wait; remaining callers continue. When every caller leaves, the work is cancelled and removed. Completed errors are removed immediately, so later requests retry. Successful nonempty results are published to the cache before releasing the flight, closing the previous asynchronous cache-write window. Cache writes are now on the completion path and retain the cache client's configured timeout. Explicit noCache requests keep independent execution and the existing cache-population behavior.
This adds no backend protocol or cross-host coordination. Existing backend batch/routing and transport paths remain unchanged.
Before / after
Usage
Enabled directly in the existing render handler for cache-eligible misses, including null or unavailable response caches. No new configuration or dependency. The input-points histogram counts materialized input after replica merging and includes extra evaluator fetches; it is not raw wire volume across replicas.
Validation
make testpassed with the repository's cairo/nativegzip/zlibng race suite and both backend transport build variants.Production validation passed: 61min of normal traffic, 1,589,854 requests, 4,669 coalesced renders, no process restart, and 60/60 byte-identical comparisons across two independent samples.
A natural-traffic sample found three large pairs of identical concurrent renders. Each pair read 22,553 input series (263,779,888 materialized input bytes) through one upstream render call; the follower issued zero upstream calls. No synthetic heavy load was injected.
The existing transport settings and request mixes differed, so these figures are observational. Coalescing preserves independent work for different cache keys; upstream failures can still return 5xx. The instance was restored to its previous release after validation.