Skip to content

feat(metrics): Add explicitly sharded histograms - #311

Open
nox wants to merge 1 commit into
cloudflare:mainfrom
nox:split-histogram
Open

nox wants to merge 1 commit into
cloudflare:mainfrom
nox:split-histogram

Conversation

@nox

@nox nox commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

SplitHistogram collects explicitly registered SplitHistogramShard handles into one standard classic histogram. Each shard records through relaxed atomics without taking a parent-wide observation lock, while the parent retains ownership for collection.

The implementation reuses HistogramSnapshot and the existing histogram encoding path, preserving text and protobuf rendering. Shard placement remains caller-controlled, including the documented option of caller-managed thread-local storage.

Both types are exported from foundations-metrics and the backend-gated foundations facade.

Exemplar support remains deferred because sharing exemplar storage across explicit shards requires additional ownership and conflict semantics.

`SplitHistogram` collects explicitly registered `SplitHistogramShard`
handles into a standard classic histogram while observations use
per-shard relaxed atomics. The parent retains shard ownership, and cloned
shard handles share their underlying counters.

The implementation reuses `HistogramSnapshot` and the existing histogram
encoder, preserving standard text and protobuf rendering. Caller-managed
thread-local storage is documented without adding TLS policy to the
metrics backend.

Both types are available from `foundations-metrics` and its backend-gated
`foundations` facade.

Exemplar support remains deferred because sharing exemplar storage across
explicit shards requires additional ownership and conflict semantics.
@gregor-cf

Copy link
Copy Markdown
Contributor

Why not use the thread_local crate? It already has the ability to aggregate/query the values from each thread. ThreadLocalCounter already uses that f8d656b
With the thread_local create you can also create a new histogram at runtime and it can work with the metrics macros.

@nox

nox commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@gregor-cf I wasn't aware of ThreadLocalCounter, but also I don't think we need the thread_local crate, do we? This performs an additional lookup, poll hooks are hot, and my current SplitHistogram can also work with the metrics macros (though I forgot to implement MetricConstructor<SplitHistogram> for HistogramBuilder) and can be created at runtime, you can even manually store the shards with the thread_local crate too if you want.

@nox

nox commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor Author

Tbh I don't think TLS-backed counters should use the thread_local crate either. We could either align my split histogram with ThreadLocalCounter, or we could also introduce SplitCounter, and reimplement ThreadLocalCounter for compat using SplitCounter internally.

@TheJokr

TheJokr commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

I don't think TLS-backed counters should use the thread_local crate either.

It adds an extra 2 (or 3, if you count the boxing we do for ThreadLocalCounter) pointers to chase. All of those are in cache lines that only get written if a new thread's value is initialized, so should rarely be invalidated. And the rest of your code can execute OOO while the CPU is waiting for the cache. I don't think the overhead is noticeable.

That being said, I'm not against explicitly-sharded metric types, but they are very different from ThreadLocalCounter. The latter is a 1:1 API replacement for a regular Counter, whereas explicit sharding requires the user to figure out what to do with the shards. (For example, stuffing them into static thread locals as in your example.)

.collect(),
});

self.state.shards.write().push(Arc::clone(&state));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we allocate a new shard, can we check if we have an unused shard here? (E.g. checking for Arc::strong_count(shard) == 1 or maintaining a free list.) That way, workloads that churn through threads (or just call .shard() too often) don't derange into an endless list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants