[ckedwards]

Fixes #5991.

Every finished HTTP request to an actor spawned a task, through wait_until, that waits for the in-flight request count to reach zero and then re-arms the sleep timer. Under steady concurrent load the count rarely reaches zero, so these tasks pile up, one per request. Every notification on the shared idle_notify wakes all of them, and each takes the Notify's mutex, sees requests still in flight, and waits again. CPU per call grows with the backlog.

This keeps one such task per actor. An AtomicBool on SleepState is claimed before spawning and released when the waiter returns, just before it resets the sleep timer. A request that finishes in that gap is covered by the reset that follows. If the actor stops while a waiter is pending, the flag stays set, but the actor's next wake starts with fresh sleep state.

Results

rivetkit-napi built from the v3.0.6 tag, without and with this change (both files are unchanged between v3.0.6 and main). One actor whose action inserts a SQLite row, called from the same process; Bun 1.4.3, Linux x86-64, 22 cores:

In flight	Before	After
1	290 calls/s, 0.7 ms CPU per call	451 calls/s, 0.7 ms
4	567 calls/s, 17.6 ms, 10 cores	501 calls/s, 0.7 ms, 0.3 cores
16	122 calls/s, 11.8 ms	395 calls/s, 0.7 ms
64	355 calls/s, 39 ms, 14 cores	528 calls/s, 0.7 ms, 0.4 cores

After the change it held for 60 s at each level: 0.7 ms of CPU per call and 0.3 cores, at 1, 16 and 64 in flight.

Before the change, perf (on a build with symbols) put about three quarters of the CPU in the kernel's futex spin lock. The named frames above it were Mutex::lock_contended in tokio::sync::notify::NotifiedProject::poll_notified, under ActorContext::wait_until(rearm_sleep_after_request).

Testing

cargo fmt, cargo check -p rivetkit-core, and the reproduction below. I didn't add a test: I couldn't find tests for rivetkit-core in this repository, and the bug shows up only as CPU time. Happy to add one wherever you'd like it.

Reproduction (repro.ts)

// Many calls waiting on one actor: throughput and CPU per call, by how many are in flight.
// One actor, one action that inserts a row into its SQLite database; the client in the same process.
//
//   bun install && bun repro.ts              (RivetKit starts its own engine)
import { actor, setup } from 'rivetkit'
import { createClient } from 'rivetkit/client'
import { db } from 'rivetkit/db'

const counter = actor({
  db: db({
    onMigrate: async (database) => {
      await database.execute('CREATE TABLE IF NOT EXISTS hits (id INTEGER PRIMARY KEY, at INTEGER NOT NULL)')
    },
  }),
  actions: {
    hit: async (c) => {
      await c.db.execute('INSERT INTO hits (at) VALUES (?)', Date.now())
      return true
    },
  },
})

const registry = setup({ use: { counter } })
registry.config.noWelcome = true
await registry.startAndWait()
const client = createClient<typeof registry>(registry.parseConfig().endpoint!)
const one = client.counter.getOrCreate(['one'])
await one.hit()

const seconds = Number(process.argv[2] ?? 15)
const levels = (process.argv[3] ?? '1,4,16,64').split(',').map(Number)
console.log(`RivetKit, ${seconds} s at each level; CPU is this whole process (actor and client), user + system`)
for (const inFlight of levels) {
  const times: number[] = []
  const errors = new Map<string, number>()
  const cpu0 = process.cpuUsage()
  const t0 = performance.now()
  await Promise.all(
    Array.from({ length: inFlight }, async () => {
      while (performance.now() - t0 < seconds * 1000) {
        const s = performance.now()
        try {
          await one.hit()
          times.push(performance.now() - s)
        } catch (error) {
          // the engine refusing a write while storage is busy, say: counted, not retried
          const key = String((error as { message?: string }).message ?? error).slice(0, 80)
          errors.set(key, (errors.get(key) ?? 0) + 1)
        }
      }
    }),
  )
  const wall = (performance.now() - t0) / 1000
  const used = process.cpuUsage(cpu0)
  const cpuSeconds = (used.user + used.system) / 1e6
  times.sort((a, b) => a - b)
  const at = (q: number) => times[Math.min(times.length - 1, Math.floor(times.length * q))]!.toFixed(0)
  console.log(
    `${String(inFlight).padStart(3)} in flight: ${(times.length / wall).toFixed(0).padStart(5)} calls/s, ` +
      `p50 ${at(0.5)} ms, p99 ${at(0.99)} ms, ${(cpuSeconds / wall).toFixed(1)} cores, ` +
      `${((1000 * cpuSeconds) / times.length).toFixed(1)} ms CPU per call` +
      (errors.size > 0 ? `; failed: ${[...errors].map(([m, n]) => `${n}× ${m}`).join('; ')}` : ''),
  )
}
process.exit(0)

package.json: { "type": "module", "dependencies": { "rivetkit": "3.0.6" } }, then bun install && bun repro.ts 60 1,4,16,64.

[RivetAgent]

The changes look good; no high-confidence bugs or CLAUDE.md compliance issues found in the supplied commit range.

GitHub authentication was unavailable, so PR status and existing Claude comments could not be verified.

Copy prompt to fix all

Fix all actionable findings in the latest Ironworks code review on PR rivet-dev/rivet#5992. Inspect every referenced location, address each finding without changing unrelated behavior, run the relevant tests, and summarize what changed.