Skip to content

Wait for streaming responses to drain - #369

Draft
tnsardesai wants to merge 1 commit into
mainfrom
hypeship/guest-drain-hold
Draft

Wait for streaming responses to drain#369
tnsardesai wants to merge 1 commit into
mainfrom
hypeship/guest-drain-hold

Conversation

@tnsardesai

@tnsardesai tnsardesai commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

problem

Large streaming handlers can return while response bytes remain unacknowledged in the guest TCP send queue. Releasing the request scale-to-zero hold at that point can suspend the guest before the peer receives the response tail.

change

  • wrap accepted TCP connections so response drains run after net/http writes final response framing, including the terminal chunk for chunked responses
  • retain an additional request-counted scale-to-zero hold from handler completion through response finalization and TCP acknowledgement
  • apply a five-minute write deadline before net/http finalizes the response, then poll Linux TIOCOUTQ every 100 ms within the same overall deadline
  • release on acknowledgement, timeout, write-deadline failure, or socket-state failure, with structured outcome logs
  • preserve current behavior for non-TCP listeners and routes that do not stream files or binary data

TIOCOUTQ support and its unsent-plus-unacknowledged semantics were validated in the current browser runtime.

tests

  • go build ./...
  • go vet ./...
  • go test -race ./lib/scaletozero ./cmd/api/api
  • go test -race $(go list ./... | grep -v '/e2e$')
  • content-length slow-reader, chunked finalization, ceiling, immediate-drain, unsupported-deadline, ioctl-error, and non-TCP fallback coverage
  • slow-reader and chunked/ceiling tests repeated 20 times under the race detector

@tnsardesai
tnsardesai force-pushed the hypeship/guest-drain-hold branch from 679f63d to 831acb0 Compare September 4, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant