Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #710 +/- ##
============================================
+ Coverage 73.18% 73.22% +0.03%
- Complexity 3589 3592 +3
============================================
Files 392 392
Lines 16531 16546 +15
Branches 1736 1739 +3
============================================
+ Hits 12099 12116 +17
Misses 3811 3811
+ Partials 621 619 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
MessageDecoder accepted a non-positive maxBufferSize. Since DecodeNext only admits messages while totalAvailable is positive, such a decoder never submits anything and silently stalls the reader instead of failing fast. Construction now rejects it with IllegalArgumentException. ReadPartitionDecoder invoked readyHandler directly from EncodedMessage.decode() and from setError(). An exception thrown by the handler escaped into the decompression executor thread, where it is reported as an uncaught exception and may suppress decoding of the messages that follow. The handler is now called through notifyReady(), which logs the failure after the message has already been marked as ready. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DecodeNext admitted the next message whenever any budget remained, without looking at its size. A message was therefore submitted even when it did not fit, so totalAvailable went negative and the decoder held more uncompressed data than maxMemoryUsageBytes allows. A message is now admitted only when it fits into the remaining budget. A message larger than the whole budget would never fit, so it is still admitted, but only once nothing else retains the buffer, which keeps the overshoot bounded by that single message instead of letting it accumulate. The existing tests asserted the previous behavior through the negative values of getTotalAvailable(), so their expectations are updated accordingly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Myllyenko
force-pushed
the
codex/topic-decoder-safety
branch
from
September 17, 2026 13:41
1bf6a03 to
aa48da9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Guard the ready handler and validate the buffer size
maxBufferSizeproduced a decoder that can never admit a message, becauseDecodeNextonly runs whiletotalAvailableis positive. The reader stalled silently.IllegalArgumentException.readyHandlerwas invoked directly fromEncodedMessage.decode()and fromsetError(). An exception thrown by the handler escaped into the decompression executor thread, where it is reported as an uncaught exception and can suppress decoding of the messages that follow.notifyReady(), which logs the failure after the message has already been marked as ready.This commit changes no existing expectations — all pre-existing tests pass untouched.
2. Make the decompression buffer limit a hard limit
DecodeNextadmitted the next message whenever any budget remained, without looking at its size, sototalAvailablewent negative and the decoder retained more uncompressed data thanmaxMemoryUsageBytesallows.This commit is up for discussion and can be dropped on its own. Four existing tests (
flowControlByBudgetTest,partitionFlowTest,decodeStopTest,decodesOnProvidedExecutorTest) asserted the previous behavior explicitly through negativegetTotalAvailable()values, which suggests the overshoot may be an intentional trade-off rather than an oversight. Their expectations are updated in this commit. The trade-off of the stricter rule: a message larger than the budget now waits until the buffer is fully released, and since one decoder is shared by all partitions of a reader, that wait can be longer when several partitions are active.If the greedy admission is intended, dropping commit 2 leaves commit 1 applicable as is.