fix(ilp): fix a leaked socket and native memory when an HTTP sender fails to start - #78
Conversation
The ILP HTTP senders build their client through this fork's HttpClientFactory, whose constructors took a socket and native buffers with no rollback. A throw past the first acquisition left a half-built client the caller never receives, so close() never ran and everything taken so far leaked, the socket file descriptor included. HttpClient now takes the socket, both buffers and the response parser inside one try and frees what it took in reverse order, attaching a close failure to the primary exception rather than replacing it. ResponseHeaders does the same for the parser its super() call completed. HttpHeaderParser moves the direct UTF-8 sink out of its field initialiser, which runs before the constructor body, so the constructor's own catch can reach it. The three platform subclasses close the completed base client when their poller fails to open. HttpClientWindows also reads the select facade before it takes the FD set. Nothing between that acquisition and the end of the constructor can then throw, which removes a partial-FDSet cleanup branch that only a Windows host could ever have reached. HttpClientConstructorTest covers each rollback point with a close-counting socket, and HttpHeaderParserTest covers the sink. Both inject the fault deterministically and without a host dependency: a negative buffer size makes Unsafe.allocateMemory throw, and a throwing configuration getter fails each platform subclass before it touches platform natives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The client-side port of the HTTP constructor rollback lives in questdb/java-questdb-client#78, which is not merged yet. Pinning the submodule at that branch commit would make this PR depend on an unmerged revision and, if merged first, would permanently fix the pointer at a commit that is not on the client's main branch. Point the submodule back at 2e84db07, the commit this PR already carried and the one main was on when the branch was cut. Once #78 merges, the pointer moves forward to a main commit that contains the fix. Chosen over the current main head, 9ccfadbe: that is 1.3.7-SNAPSHOT and would require bumping questdb.client.version in three parent poms, and it would drag in a version bump and a PEM CA feature unrelated to this PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AbstractLineHttpSender's constructor assigned this.client - either the one createLineSender() handed in or one it built through HttpClientFactory - and then called newRequest(), with neither statement inside a try. newRequest() writes the POST line, the path, the User-Agent header and any auth header into the client's request buffer, and growBuffer() throws once they exceed the maximum. A throw there left a fully built client - a socket plus two native buffers plus the response parser - that no caller ever receives a reference to, so nothing closed it. A Java-heap OutOfMemoryError from the version lookup or the User-Agent concatenation lands in the same window. createLineSender() cannot clean this up. Its three Misc.free(cli) calls all sit before the switch, and the switch builds the sender inside a return expression that no try covers. Both entry shapes leak: an explicit protocol version makes the constructor build the client itself, and an auto-detected one hands in the client the version probe used. The constructor now takes the client, the version and the request inside one try and closes the client on the way out, attaching a close failure to the primary exception rather than replacing it. It frees a handed-in client too: close() already frees the client regardless of where it came from, so the sender owns it from the assignment on either way. LineHttpSenderConstructorTest covers both shapes. A configuration whose maximum request buffer is smaller than the preamble makes newRequest() throw deterministically, with no server and no new production seam. The self-built case asserts against the enclosing leak check; the handed-in case adds a close-counting socket, since the leak check cannot see a file descriptor. Both fail without the fix: the first on 65552 bytes leaked under NATIVE_DEFAULT, the second on the missing socket close. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… probe facadeCloseIsBoundedUnderRepeatedInterruptsDuringSenderCreation failed on windows-msvc-2022-x64 with "sender close left its creation wait before the interrupt storm landed twice; interrupts landed: 6" -- a false negative, not a product defect. Both pools' hasCreationWaiterForTesting() reported the wait region as lock.hasWaiters(creationFinished). An interrupt does not leave the waiter on the condition queue: AQS transfers the node to the lock's sync queue to reacquire before awaitNanos rethrows, so under the test's interrupt storm the probe reads false for a measurable slice of a wait close() never left. A harness replicating close()'s loop measures 5-8% of polls reading false with the 1s deadline honoured exactly. Count the threads inside the wait loop instead: the state no longer flickers, and the rising edge no longer depends on the closer having actually parked. awaitRepeatedInterrupts() also re-reads the interrupt count before failing, since the count it samples precedes the predicate read that fails on it. creationWaitStateDoesNotFlickerUnderInterruptStorm pins the contract: red on the old predicate (flickered after 46 polls, 2 interrupts landed), green now. Full suite green locally: 2766 tests, 0 failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ollback # Conflicts: # core/src/main/java/io/questdb/client/cutlass/http/client/HttpClient.java # core/src/main/java/io/questdb/client/cutlass/http/client/HttpClientLinux.java # core/src/main/java/io/questdb/client/cutlass/http/client/HttpClientOsx.java # core/src/main/java/io/questdb/client/cutlass/http/client/HttpClientWindows.java # core/src/main/java/io/questdb/client/cutlass/line/http/AbstractLineHttpSender.java # core/src/main/java/io/questdb/client/impl/QueryClientPool.java # core/src/main/java/io/questdb/client/impl/SenderPool.java # core/src/test/java/io/questdb/client/test/impl/QuestDBImplCloseLifecycleTest.java
|
Review (level 3): approve. Nothing blocking, and no Moderate or Minor findings. What I checked
Scope
|
[PR Coverage check]😍 pass : 38 / 39 (97.44%) file detail
|
bluestreak01
left a comment
There was a problem hiding this comment.
PR #78 review: HTTP sender and client constructor rollback (level 3)
Verdict: approve with comments. Nothing blocks the merge and the test gate passes. One Moderate item should be fixed before merge: the change removes some released public constructor signatures without keeping the old ones. The rollback fix itself is correct, and I found no issues with it.
Critical
None.
Moderate
M1. Released multi-host sender constructors were changed in place instead of overloaded
- Problem: four constructors from the 1.3.9 release no longer exist. There is no overload with the old parameter list.
- Net impact: any outside code that calls these constructors directly, or subclasses these senders, stops compiling or linking. I found no such callers.
- Evidence:
- Release 1.3.9 (
git show 1.3.9:…) has these 17-argument constructors, ending(…, int currentAddressIndex, Rnd rnd):LineHttpSenderV2andLineHttpSenderV3: public.AbstractLineHttpSenderandLineHttpSenderV1: protected. The classes are public and not final, so subclasses can call them.
- At head 5b450b7,
javap -protectedlists only the 18-argument forms, which addHttpTokenProviderat the end. - A small caller that uses the old
LineHttpSenderV2constructor compiles against base 916e51f (javac exit 0). Against head it fails with "no suitable constructor found".
- Release 1.3.9 (
Where it is:
core/src/main/java/io/questdb/client/cutlass/line/http/AbstractLineHttpSender.java:168-187LineHttpSenderV1.java:82-99LineHttpSenderV2.java:86-105LineHttpSenderV3.java:46-65
module-info.java:52 exports io.questdb.client.cutlass.line.http. The repo's own rules apply here:
ExportedApiCompatibilityTestsays: "Adding an overload is fine … retyping or removing one turns it red". It also explains that a break with no known callers was still restored in the past.createLineSenderkeeps a provider-less overload "so callers compiled against the pre-httpTokenProvider signature keep linking".
This PR goes against that. The test doesn't catch it because it only checks createLineSender and connect, not constructors.
Why Moderate, not Critical: I searched the questdb and questdb-enterprise checkouts and found no callers. They only use the 13-argument single-host LineHttpSenderV2 constructor, which is unchanged. The removed constructors also take internal plumbing types (HttpClient, Rnd, currentAddressIndex). So I can't name anyone who is actually affected.
Suggested fix: add the four 17-argument constructors back as overloads that pass null for the provider. Also add the constructor signatures to ExportedApiCompatibilityTest, for example with a constructor variant of assertSignaturesPresent.
Minor
None.
Coverage map
Test gate: pass. No coverage gaps admitted.
Every behaviour change has a local test that fails without the fix. I ran the new tests on JDK 8 at head (all pass) and at base with the tests adapted to compile there (base 916e51f):
| New test | Result at base |
|---|---|
HttpHeaderParserTest.testConstructorFailureFreesNativeAllocations |
fails: 32 B leaked under NATIVE_DIRECT_UTF8_SINK |
LineHttpSenderConstructorTest.testHandedInClient… |
fails: socket closed once, expected twice |
LineHttpSenderConstructorTest.testProviderClient… |
fails: leak under NATIVE_DEFAULT |
LineHttpSenderConstructorTest.testSelfBuiltClient… |
fails: leak under NATIVE_DEFAULT |
HttpClientConstructorTest.testBaseConstructorFailurePreservesOriginalFailure… |
errors: the socket close failure replaces the original exception |
The wider run (*Http*, *LineSender*, *SenderPool*, *TokenProvider*, Misc*) passed at head: 649 tests, 0 failures. The code also compiles and test-compiles on JDK 25.
No tandem PR is needed. The change only affects what happens when a sender fails to start, which a server can't observe and unit tests here can cover. I searched for tandem PRs on the puzpuzpuz_http_ctor_rollback branch: there are none in questdb or questdb-enterprise.
Summary
- Submodules: no submodule pointer moves.
- Scope of the finding: M1 is an in-diff issue with no breakage outside the diff. I checked every caller of the changed symbols across this repo and the local
questdb/questdb-enterprisecheckouts. - Rollback fix:
- The client a sender is handed is closed exactly once.
createLineSenderhas no cleanup of its own around the constructor switch, so there is no double close. - Passing the token provider through the constructor leaves the request in the same state as the old rebuild. The only difference is one fewer
client.newRequest()call. Misc.freeSuppressingbehaves the same as the oldSenderPoolhelper. It is slightly safer inQwpWebSocketSender, where it can no longer throw on self-suppression.- Nothing changes on the per-row or flush path.
- The client a sender is handed is closed exactly once.
- Findings: 0 Critical, 1 Moderate (M1), 0 Minor.
Before approving, I'd ask the author to add the constructor overloads from M1.
Found while reviewing questdb/questdb#7434, which made the equivalent sender
construction path exception-safe in the core repository. The ILP HTTP sender
uses this repository's
HttpClientFactory, so it needs the same ownershipguarantee here.
Problem
An HTTP sender can fail after acquiring a client but before the sender escapes
to its caller. In particular,
newRequest()can throw while writing the requestpreamble. The same risk exists when the OIDC token provider is installed and the
factory rebuilds the initial request. In both cases, the client, its socket and
its native buffers were left without an owner that could close them.
HttpHeaderParseralso allocated itsDirectUtf8Sinkin a field initializer.Because field initializers run before the constructor body, a later header-buffer
allocation failure could strand the sink outside the constructor's cleanup path.
Finally, if closing a staged socket during
HttpClientconstruction failed, thatcleanup failure could replace the exception that explained why construction
failed.
Changes
native allocations are released if construction fails.
HttpClientconstruction exception and attach any socketclose failure as suppressed.
initial request.
OIDC token provider.
There is no happy-path behavior change. A client supplied to a sender is now
closed if sender construction fails, matching the existing ownership contract on
the successful path, where closing the sender also closes that client.