Skip to content

Fix quarantined pool checkout worker scheduling - #5119

Open
mdaigle wants to merge 1 commit into
mainfrom
dev/mdaigle/sqlclient-flaky-repairs-a49
Open

mdaigle wants to merge 1 commit into
mainfrom
dev/mdaigle/sqlclient-flaky-repairs-a49

Conversation

@mdaigle

@mdaigle mdaigle commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

🤖 This pull request was created by a fully automated workflow.

Description

Repair Microsoft.Data.SqlClient.ManualTesting.Tests.ConnectionPoolTest.MaxPoolWaitForConnectionTest: four non-Synapse TCP inputs, covering MARS on/off and usePoolV2 true/false.

Demonstrated mechanism: the helper inherits the caller's scheduler while the caller polls synchronously for at most one second. A queued helper need not start within that window. CI recorded Expected: Running / Actual: WaitingToRun for both pool versions. A single-worker scheduler deterministically reproduces that assertion; this models the scheduling dependency, not the original agent's exact load.

Run the blocking checkout on a dedicated LongRunning task using TaskScheduler.Default, with an explicit startup signal. The 30-second startup bound is a failure safeguard, not a delay. Preserve running/completion, physical-connection and pool-identity assertions; release and drain the worker in finally. Add MaxPoolWaitForConnectionTest_SingleWorkerScheduler_ReusesConnection using the same inputs through typed MemberData. No production/API or pipeline changes.

Retain [Trait("category", "flaky")] and add [Trait("category", "probation")]. Ordinary filters exclude the repaired theory, quarantine selects it, and the regression runs ordinarily. Probation begins separately on each upstream branch only when the repair and both traits actually merge there, never on PR creation. Later transitions belong to the independent probation reviewer.

Release routing: the same polling defect exists on releases 6.1, 7.0, 7.1 and 8.0. The complete patch applies cleanly to 8.0, whose original test-file blob matches main; Hotfix 8.0.0 is applied. Routing to 6.1.9, 7.0.5 and 7.1.3 is blocked: the older synchronous signatures/providers make the unchanged regression incompatible, requiring a tailored backport. Those labels are deliberately withheld rather than route an unsafe complete diff. Release binaries were not separately executed.

Issues

#5105 quarantined this same startup assertion. #1465 previously increased its polling duration without removing the scheduling dependency.

Testing

Local Windows SQL Server, TCP/integrated authentication against master. Executed net462/native SNI and net10.0/native and managed SNI (UseManagedSNIOnWindows=true for managed).

$p = 'src\Microsoft.Data.SqlClient\tests\ManualTests\Microsoft.Data.SqlClient.ManualTests.csproj'
$scope = 'FullyQualifiedName~ConnectionPoolTest.MaxPoolWaitForConnectionTest'
$classScope = 'FullyQualifiedName~Microsoft.Data.SqlClient.ManualTesting.Tests.ConnectionPoolTest.'
$quarantine = $scope + '&category=flaky'
$ordinary = $classScope + '&category!=failing&category!=flaky&category!=interactive'
dotnet restore $p -v:q
dotnet test $p --no-restore --filter $scope -v:q
dotnet test $p --no-restore -c Release --filter $scope -v:q
dotnet test $p --no-restore -c Release --filter $classScope -v:q
dotnet test $p --no-build --no-restore -c Release --filter $quarantine -v:q
dotnet test $p --no-build --no-restore -c Release --filter $ordinary -v:q

Results: restore passed; focused Debug and Release runs passed 8 cases per framework/configuration, none skipped. Class run: 18 passed/framework, one Azure-token-only sibling skipped. Quarantine: exactly 4 affected cases passed/framework; ordinary: 12 passed/framework including all 4 regression inputs, with the Azure sibling skipped. --list-tests verified both selections. With original scheduling restored, dotnet test $p --no-restore --filter 'FullyQualifiedName~MaxPoolWaitForConnectionTest_SingleWorkerScheduler_ReusesConnection' -v:q failed all 4 inputs/framework; repaired source passed them.

CI: scoped 193677 completed at the repair SHA. All 48 affected/regression child results passed across six Windows SQL25/Azure SQL configurations, including existing quarantine steps. The build is partially succeeded, not wholly green: unrelated quarantined vector deadlocks and Linux date/time/distributed-transaction failures remain. Linux's result publisher rejected its files, and the mapped artifact download returned no inspectable content; no Linux quarantine case-level pass is claimed. PR build 193679 succeeded; its Azure SQL regression results additionally passed on Linux/net10.0 and Windows/net462/net10.0. Individual execution records.

Guidelines

Please review the contribution guidelines before submitting a pull request:

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 19, 2026 17:10
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Nov 19, 2026
@mdaigle mdaigle added the Hotfix 8.0.0 PRs that must be backported to release/8.0 label Nov 19, 2026
@mdaigle mdaigle added this to the 8.1.0-preview1 milestone Nov 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Missing PR identifiers, base/head SHAs, review history, and trusted immutable supplemental references prevent verification of head freshness, current-head CI, and outstanding feedback resolution.

No open issues 🎉

What changed in this PR

Repairs worker-start synchronization in the connection-pool checkout test, addressing the scheduling dependency behind the quarantined failures without changing production code or APIs.

Changes:

  • Runs the checkout worker on a dedicated LongRunning task with an explicit startup signal and cleanup that releases and drains the worker.
  • Adds a single-worker-scheduler regression covering both MARS settings and both pool implementations through typed MemberData.
  • Retains the flaky category and adds probation while preserving connection-reuse and pool-identity assertions.
File Description
src/​Microsoft.Data.SqlClient/​tests/​ManualTests/​SQL/​ConnectionPoolTest/​ConnectionPoolTest.cs Repairs worker scheduling and synchronization, adds controlled regression coverage, and marks the repaired theory for probation.

Review effort: Balanced

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.56%. Comparing base (17bef7b) to head (ed206ee).
⚠️ Report is 23 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5119      +/-   ##
==========================================
+ Coverage   68.99%   72.56%   +3.56%     
==========================================
  Files         314      315       +1     
  Lines       45857    70982   +25125     
==========================================
+ Hits        31638    51505   +19867     
- Misses      14219    19477    +5258     
Flag Coverage Δ
CI-SqlClient 71.37% <ø> (+2.38%) ⬆️
sqlclient-pr 70.51% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdaigle

mdaigle commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Original failure/control records use source 083570c61341c024edcea80856e72d7911ae8312, Windows 11/Azure SQL, net10.0 managed SNI, Release/Project/Set 3; stage/phase/job/task attempts all 1.

Record Individual results UTC
Failure, build 193066/run 3734166 103318, 103320, child 2 each Nov 18, 01:48:07-01:48:10
Same-source controls, build 193040/run 3733831 102353, 102355, both children each Nov 17, 23:57:49-23:57:51

Build 193677 execution mapping below: x64, Release/Project/Set 3, stage/phase/job/task attempts 1. IDs are ordinary / quarantine parent results; each listed parent has two individually verified Passed children. Run links open the first listed result. UTC bounds are November 19, 2026.

Configuration Run Ordinary IDs Quarantine IDs UTC
Win25 SQL25, net10 managed 3743860 102169, 102174 102628, 102629 17:34:42-17:41:36
Win25 SQL25, net10 native 3743866 102169, 102174 102628, 102629 17:35:49-17:41:34
Win25 SQL25, net462 native 3743928 102103, 102108 102492, 102493 17:36:12-17:44:09
Win11 Azure, net462 native 3744023 102178, 102180 102564, 102566 17:44:34-17:48:34
Win11 Azure, net10 native 3744034 102350, 102352 102806, 102808 17:45:31-17:48:59
Win11 Azure, net10 managed 3744056 102350, 102352 102806, 102808 17:45:41-17:50:03

PR build 193679 used merge SHA 47142a5e0cf7fa0a6de04927b2e0850bcaf73b7f. Azure ordinary-result pairs (two Passed children each), stage/phase/job attempts 1: Linux/net10 3743661:101425,101427, 17:32:46Z; Windows/net10 3743673:101450,101457, 17:33:32-34Z; Windows/net462 3743685:101299,101306, 17:33:48Z.

@mdaigle
mdaigle marked this pull request as ready for review November 19, 2026 18:05
@mdaigle
mdaigle requested a review from a team as a code owner November 19, 2026 18:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Hotfix 8.0.0 PRs that must be backported to release/8.0

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

4 participants