Skip to content

Fix Daily Go Test Parallelizer failures: drop preview cloud-hypervisor runtime, restore daily schedule - #54730

Draft
pelikhan with Copilot wants to merge 2 commits into
mainfrom
copilot/aw-daily-go-test-parallelizer-fix
Draft

Fix Daily Go Test Parallelizer failures: drop preview cloud-hypervisor runtime, restore daily schedule#54730
pelikhan with Copilot wants to merge 2 commits into
mainfrom
copilot/aw-daily-go-test-parallelizer-fix

Conversation

Copilot AI commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

The Daily Go Test Parallelizer workflow was reported at a 42.86% success rate costing ~$274/day for 3 safe items. Investigation shows both symptoms have causes different from those hypothesized in the issue.

Failure root cause

Failures cluster contiguously (runs #63#68, #53#55), not randomly. Every failed run aborts in the agent job's Execute GitHub Copilot CLI step with the same signature (verified on runs 32447678169 and 32404370749):

[WARN] [cloud-hypervisor] stage=guest-connectivity status=failed: Cloud Hypervisor guest connectivity probe failed with exit code 4
[ERROR] Fatal error: Error: Cloud Hypervisor guest connectivity probe failed with exit code 4
##[error]Agent execution exited with code 1

The guest VM's eth0 comes up NO-CARRIER/DOWN and the probe fails with fallback=disabled. This is preview-runtime infrastructure, not a flaky test shard or aggregation error. Failed runs die ~2 min into the agent job and never invoke the model — they consume no agent tokens, so the issue's premise that failures are expensive does not hold, and neither a fail-fast smoke step nor shard-scoped retry would help.

Cost root cause

The schedule was every 2h — 12 runs/day — while the workflow is named "Daily", described as "daily round-robin analysis", and its body references "the next daily run". Spend was driven by successful runs firing 12× more often than intended.

Changes

  • sandbox.agent.runtime: cloud-hypervisor removed — falls back to the default rootless Docker sandbox. Also reduces privilege surface; the repo linter independently flags this runtime as "a privileged KVM preview path… require a human security review".
  • schedule: every 2hschedule: daily — restores intended cadence, ~12× cost reduction. Round-robin advance is unaffected; each run still persists last_file to cache memory.
  • Recompiled daily-go-test-parallelizer.lock.yml.
 on:
-  schedule: every 2h
+  schedule: daily
...
-sandbox:
-  agent:
-    runtime: cloud-hypervisor
 tools:

No retry logic or pre-flight smoke step was added; both were suggested in the issue but neither addresses the actual failure.

Follow-up

~40 other workflows still pin cloud-hypervisor and are exposed to the same intermittent probe failure. Out of scope here, but likely worth its own tracking issue.

…edule

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix low success rate and high-cost failures in daily Go Test Parallelizer Fix Daily Go Test Parallelizer failures: drop preview cloud-hypervisor runtime, restore daily schedule Aug 22, 2026
Copilot AI requested a review from pelikhan August 22, 2026 05:32
@pelikhan pelikhan added the awf label Aug 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR Triage

  • Category: bug
  • Risk: low
  • Priority score: 55/100 (impact 35, urgency 5, quality 15)
  • Recommended action: defer
  • CI status: no_checks

Generated by 🔧 PR Triage Agent · auto · 82.9 AIC · ⌖ 6.65 AIC · ⊞ 8.3K ·

@github-actions

Copy link
Copy Markdown
Contributor

PR Triage

Category: bug | Risk: low | Priority: medium | Score: 68/100
Recommended action: defer

Draft but high-value: fixes 42.86% CI success rate / ~$274/day cost issue on Daily Go Test Parallelizer. Should be undrafted to fast_track.

Automated triage — run 32572524009

Generated by 🔧 PR Triage Agent · auto · 65.4 AIC · ⌖ 6.04 AIC · ⊞ 8.3K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Daily Go Test Parallelizer: 43% success rate, high-cost failures

2 participants