Skip to content

Release pageheap_lock while HugePageFiller unbacks memory. - #950

Draft
copybara-service[bot] wants to merge 1 commit into
masterfrom
test_982712772
Draft

copybara-service[bot] wants to merge 1 commit into
masterfrom
test_982712772

Conversation

@copybara-service

Copy link
Copy Markdown

Release pageheap_lock while HugePageFiller unbacks memory.

HugePageFiller now releases free pages through unback_without_lock_, as
HugeCache already does, so allocations and deallocations proceed while the OS
is unbacking a range. Since pageheap_lock is dropped for each range, concurrent
operations may observe the filler mid-release:

  • The tracker being released is removed from the filler lists and marked
    BeingSubreleased(). Put() on it defers list maintenance (and the transition
    to empty) to the releasing thread, which handles both after ReleaseFree()
    returns. TryGet() cannot be served from it.
  • Pending release candidates are marked HugePageTreatmentType::kSubrelease,
    which excludes them from another release, from treatment selection, and from
    being freed back to the page heap should a Put() empty them. A tracker
    emptied while marked is parked on fully_freed_trackers_ instead;
    HugePageAwareAllocator drains parked trackers after each release and
    treatment pass, and FetchFullyFreedTracker() skips trackers that are still
    referenced.
  • HugePageUnbackedTrackerTreatment::Restore() re-checks fully_freed() after
    HandleReleaseFree(), which now drops the lock, and its selection skips
    trackers held by an in-flight release.

FreePagesInPartialAllocs() is now a maintained counter rather than a derivation
from list sizes, which would be transiently wrong while a tracker is off the
lists.

Add unit tests and fuzzer support for reentrant Put(), TryGet(), ReleasePages()
and TreatHugepageTrackers() during a release, and a multi-threaded release
stress test.

@copybara-service
copybara-service Bot force-pushed the test_982712772 branch 2 times, most recently from 7132043 to ed0d873 Compare September 17, 2026 00:04
HugePageFiller now releases free pages through unback_without_lock_, as
HugeCache already does, so allocations and deallocations proceed while the OS
is unbacking a range.  Since pageheap_lock is dropped for each range, concurrent
operations may observe the filler mid-release:

* The tracker being released is removed from the filler lists and marked
  BeingSubreleased().  Put() on it defers list maintenance (and the transition
  to empty) to the releasing thread, which handles both after ReleaseFree()
  returns.  TryGet() cannot be served from it.
* Pending release candidates are marked HugePageTreatmentType::kSubrelease,
  which excludes them from another release, from treatment selection, and from
  being freed back to the page heap should a Put() empty them.  A tracker
  emptied while marked is parked on fully_freed_trackers_ instead;
  HugePageAwareAllocator drains parked trackers after each release and
  treatment pass, and FetchFullyFreedTracker() skips trackers that are still
  referenced.
* HugePageUnbackedTrackerTreatment::Restore() re-checks fully_freed() after
  HandleReleaseFree(), which now drops the lock, and its selection skips
  trackers held by an in-flight release.

FreePagesInPartialAllocs() is now a maintained counter rather than a derivation
from list sizes, which would be transiently wrong while a tracker is off the
lists.

Add unit tests and fuzzer support for reentrant Put(), TryGet(), ReleasePages()
and TreatHugepageTrackers() during a release, and a multi-threaded release
stress test.

PiperOrigin-RevId: 982712772
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant