Skip to content

Fix timing flakes in libs/apps/logstream tests - #7609

Merged
denik merged 1 commit into
mainfrom
fix-logstream-timing-flakes
Nov 13, 2026
Merged

denik merged 1 commit into
mainfrom
fix-logstream-timing-flakes

Conversation

@denik

@denik denik commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Changes

Fix timing flakes in libs/apps/logstream tests:

  • TestBackoffStrategy_Wait: the upper bounds were 100ms around 50ms/20ms waits; CI runners measured 100-222ms. Keep the lower bound, use a 10s max / 10s initial backoff so a 5s upper bound still tells "waited current, not max" and "returned on cancel" apart.
  • TestLogStreamerRetriesOnDialFailure: the whole retry had a 300ms deadline, 200ms of which is initialReconnectBackoff. Wait for msg1 with require.Eventually (10s) and then cancel, like TestLogStreamerFollowTailWithoutPrefetchEmitsRequestedLines.
  • TestLogStreamerRefreshesTokenAfterAuthClose: read a bytes.Buffer from the test goroutine while the streamer wrote to it (go test -race reports it); use syncBuffer. Same file and same kind of flake, so it is included here.

Tests

go test -race ./libs/apps/logstream/ -count=30 -cpu=1,4 passes (the race above failed it before).

Dashboard

error snippet
error/3 "119.547582ms" is not less than "100ms" (TestBackoffStrategy_Wait/blocks_for_duration, 9 failures in 9 PRs)
error/1 streamer_test.go:166: "" does not contain "msg1" (TestLogStreamerRetriesOnDialFailure)

This pull request and its description were written by Isaac.

TestBackoffStrategy_Wait asserted elapsed < 100ms around 50ms/20ms sleeps;
CI runners observed 100-222ms. Keep the lower bound and use upper bounds
that still distinguish "waited current" from "waited max" / "did not
return on cancel".

TestLogStreamerRetriesOnDialFailure gave the whole retry 300ms, of which
200ms is initialReconnectBackoff. Wait for msg1 with Eventually instead of
a fixed deadline.

TestLogStreamerRefreshesTokenAfterAuthClose read a bytes.Buffer from the
test goroutine while the streamer wrote to it (go test -race); use
syncBuffer like the other tests.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik denik added the ai-auto PRs made by AI loops & factories, label Nov 13, 2026
@github-actions github-actions Bot added the test-only-fix PRs that update failing of flaky tests without changing CLI code. label Nov 13, 2026
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: ed81166

Run: 43461307981

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 7:25
💚​ aws-windows-latest-4core-16gb 1 272 13 5:01
💚​ azure linux-2core-8gb 1 269 15 7:51
💚​ azure-windows-latest-4core-16gb 1 271 13 4:33
💚​ gcp linux-2core-8gb 1 270 15 7:48
💚​ gcp-windows-latest-4core-16gb 1 272 13 5:58
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:13 gcp-windows-latest-4core-16gb TestAccept
3:11 azure-windows-latest-4core-16gb TestAccept
3:00 aws-windows-latest-4core-16gb TestAccept

@denik
denik marked this pull request as ready for review November 13, 2026 16:41
@denik
denik requested review from a team as code owners November 13, 2026 16:41
@denik
denik requested a review from rugpanov November 13, 2026 16:41
@denik
denik enabled auto-merge November 13, 2026 16:41
@denik
denik requested a review from andrewnester November 13, 2026 16:41
@denik
denik added this pull request to the merge queue Nov 13, 2026
Merged via the queue into main with commit b1a8fa4 Nov 13, 2026
28 checks passed
@denik
denik deleted the fix-logstream-timing-flakes branch November 13, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto PRs made by AI loops & factories, test-only-fix PRs that update failing of flaky tests without changing CLI code.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants