Skip to content

Normalize heapz data in the size class generator to per-process. - #976

Draft
copybara-service[bot] wants to merge 1 commit into
masterfrom
test_981649456
Draft

copybara-service[bot] wants to merge 1 commit into
masterfrom
test_981649456

Conversation

@copybara-service

Copy link
Copy Markdown

Normalize heapz data in the size class generator to per-process.

This gives us a consistent number instead of a total multiplied
by some semi-arbitrary sampling factor; it is especially important
if we want to estimate external fragmentation, where the model
is inherently per-process.

We still store the totals, but without multiplying by the sampling
factor when collecting (we still store it, and use it to compute
the fleet-wide totals), and we store the number of distinct sessions
so that we can divide by them in the generator.

Note that this changes the meaning of the 0.3 factor in the
reuse-distance optimization subtly; it tries to balance memory usage
against allocation rates (in itself two different units; bytes and
seconds/byte), and the two values are now normalized to both be
per-process instead of separately normalized as before. However,
the difference is small enough in practice that it outputs the same
size classes, and we'll want to adjust this factor in a separate
project later, so we keep the same value for now.

We could now output per-process usage columns for each size class,
but we don't change it at this point.

This gives us a consistent number instead of a total multiplied
by some semi-arbitrary sampling factor; it is especially important
if we want to estimate external fragmentation, where the model
is inherently per-process.

We still store the totals, but without multiplying by the sampling
factor when collecting (we still store it, and use it to compute
the fleet-wide totals), and we store the number of distinct sessions
so that we can divide by them in the generator.

Note that this changes the meaning of the 0.3 factor in the
reuse-distance optimization subtly; it tries to balance memory usage
against allocation rates (in itself two different units; bytes and
seconds/byte), and the two values are now normalized to both be
per-process instead of separately normalized as before. However,
the difference is small enough in practice that it outputs the same
size classes, and we'll want to adjust this factor in a separate
project later, so we keep the same value for now.

We could now output per-process usage columns for each size class,
but we don't change it at this point.

PiperOrigin-RevId: 981649456
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant