Skip to content

fix(executor): a loop whose condition cannot be evaluated fails instead of ending early as success - #10282

Merged
waleedlatif1 merged 3 commits into
stagingfrom
fix/loop-continuation-fails-loudly
Nov 21, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
fix/loop-continuation-fails-loudly

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Bug. LoopOrchestrator.evaluateWhileCondition returned false whenever the isolated VM failed to evaluate a loop condition, whether the VM answered with an error or the call threw. The loop exited with the iterations it had so far and emitted subflow success. For, ForEach and count-driven Do While loops also evaluated <loop.index> < N in the VM. That meant any sandbox failure could cut any loop short: capacity, owner lease limit, queue timeout, worker crash, or a missing REDIS_URL making getRedisClient() throw.

Impact. The run silently returned wrong results. A ForEach over N items could return 1 result and a While loop 0, and the run still reported success. Downstream blocks treated the truncated array as complete. A typo in a While condition looked like "the condition was false".

Fix.

  • For, ForEach and count-driven Do While loops decide continuation natively (nextIteration < maxIterations), with no VM call. <loop.index> already resolved to scope.iteration, so successful runs behave the same. The synthesized condition and its now-dead helpers are removed.
  • When a While or Do While condition cannot be evaluated, the loop now fails the way the iteration-ceiling and ForEach-resolution failures already do:
    • a "<While|Do While> loop condition could not be evaluated: …" message, projected through projectResolvedSecretDiagnosticError;
    • a loop error log and validationError;
    • a thrown error. Author errors (VM errors without isSystemError, unresolvable references) are marked user. Sandbox failures stay internal.
  • No retry. The VM helpers have no retry pattern, and capacity errors have already waited in the pool's queue.
  • getRedisClient() stays outside the lease try. Moving it inside would quietly tolerate a missing REDIS_URL, against chore(redis): require Redis and remove the no-Redis fallbacks #10272. With this fix, a rejection already fails the loop loudly.

Tests (executor/orchestrators/loop.test.ts):

  • Full DAGExecutor runs:
    • For, ForEach and count-driven Do While loops run all 5 iterations while the VM is at capacity or rejects;
    • a While loop with a syntax-error condition fails the run with a visible loop error and runs 0 iterations;
    • a Do While loop whose VM fails mid-run, with a system error or a rejection, fails instead of ending early.
  • Orchestrator-level tests check user vs internal attribution and that a resolved secret stays out of the persisted error.
  • Every test was red against the pre-fix code, and each guard reverted on its own turns its tests red.
  • The shared @sim/testing isolated-vm mock (from fix(executor): keep the condition-join loop test off the Redis-backed isolate lease #10275) now rejects every call by default instead of evaluating <index> < <count>. evaluateLoopIndexCheck is removed. Any test whose counted loop reached the sandbox would now fail.

…ad of ending early as success

For, ForEach, and count-driven Do While loops decide continuation from their
count instead of evaluating '<loop.index> < N' in the isolated VM, so a sandbox
failure can no longer truncate them. A While or Do While condition that fails to
evaluate (syntax or runtime error, unresolvable reference, sandbox failure) now
fails the loop with a loop error log and a secret-projected message instead of
being read as false. Author errors are attributed to the user; sandbox failures
stay internal.
@vercel

vercel Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 21, 2026 9:20am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 7 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues remain in the reviewed changes.

Summary

Loops now fail when their conditions cannot be evaluated, rather than returning incomplete results as success.

  • Count-driven loops stop at their configured count without asking the sandbox.
  • While and Do While loops fail when their conditions cannot be evaluated.

Reviews (3) · Last reviewed commit: "fix(executor): mark a loop condition fai..." · Reviewed by Greptile

…t helpers

Extract projectLoopErrorMessage for the ForEach-resolution and condition-evaluation
failure paths, drop the count check that the native continuation now covers, and
move loopWorkflow/runLoopWorkflow into a shared test helper.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/executor/orchestrators/loop.ts Resolved
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 754a28e into staging Nov 21, 2026
87 checks passed

This branch was previously deployed

1 inactive deployment
Preview — 889981ad Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant