Skip to content

fix(schedule): bound AI task runs and wait for SOP outcomes - #1080

Open
Robert0622 wants to merge 3 commits into
mainfrom
codex/schedule-timeout-sop
Open

Robert0622 wants to merge 3 commits into
mainfrom
codex/schedule-timeout-sop

Conversation

@Robert0622

@Robert0622 Robert0622 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The v0.4.0 scheduled assistant had no per-execution deadline. SOP dispatch returned immediately, so unattended prompts that needed an outcome could keep querying run details and logs indefinitely.

Changes

  • Add an execution timeout for AI schedules and project subscriptions. Existing jobs default to 30 minutes and can be set to 1–240 minutes. Expiry stops the AI turn and its pending tool calls and records a failed execution with a timeout reason. Dispatched SOPs have their own lifecycle.
  • Add the read-only runs_wait tool. It waits on the exact run ID using a lightweight status query, returns when the run completes or needs human action, and yields after at most 25 seconds per call. Scheduled prompts use it when they need to wait for an SOP outcome.

Validation

  • Ran the original TestScheduledAssistantHasNoDeadline at v0.4.0 to confirm the missing execution deadline.
  • Passed the Go regressions for schedule configuration, timeout cancellation, SOP state and access checks, and the OpenAPI/MCP contract.
  • Passed the scheduled task web tests and all 401 chatbot tests. The chatbot build, go vet, public content check, and Gitleaks scan also passed.
  • Passed the Go suite across non-browser tests and the five separate Chromium tests. The scheduled gateway regression checks tool availability beyond the former five-minute TTL and within the execution deadline.
  • Ran the public Playwright suite against an isolated MySQL installation. 23 passed and 6 were skipped because they require first-run setup or real model credentials.

Live CodeBuddy regression

Real DeepSeek V4.1 Flash was used for the timer prompt and the SOP Agent node on isolated MySQL instances. The Agent wrote a marker artifact, which the following script verified.

Scenario Observed result
v0.4.0, slow SOP, requested one-minute timeout The saved job omitted the timeout. The timer prompt was still running at 65 seconds and used 37 runs_get calls before reporting SOP success at 121.7 seconds.
Initial fix (7382983f), slow SOP, one-minute timeout The timer prompt stopped at 60.1 seconds and recorded an execution timeout. It issued 2 runs_wait calls, including the wait that was cancelled at the deadline. The dispatched SOP later succeeded.
Initial fix (7382983f), short SOP The timer prompt used one runs_wait call and reported the correct success state in 36.8 seconds.

The v0.4.0 run eventually reached the correct outcome in this fixture. The comparison reproduced the omitted deadline and repeated reads.

Follow-up live regression

A fresh timer run exposed another path on 7382983f. The model submitted runs_start as a final reply draft. The schedule recorded success after 15.0 seconds, applied the draft after the CLI exited, and never checked the SOP outcome.

Commit 0c3e528b keeps scheduled SOP dispatch in the active turn. The gateway directs the model to call the dispatch tool, and text recovery rejects a leftover dispatch draft before it can run. The added regression tests cover start, issue dispatch, retry, and the recovered-draft path.

We repeated both timer cases with CodeBuddy on DeepSeek V4.1 Flash and isolated MySQL. Each dispatched one SOP. A real Agent wrote a unique marker in each run, a script checked it, and both the marker and completion artifact were read back from storage.

Timer case on the updated fix Observed result
Short SOP One runs_wait returned succeeded. The timer reported the correct run ID and finished in 40.398 seconds.
Slow SOP, one-minute timeout The test prompted the model to try the terminal draft first. The gateway rejected it, then received one direct start and two runs_wait calls. The deadline cancelled the second wait (HTTP 408) and the timer recorded EXECUTION_TIMEOUT at 60.101 seconds. The independent SOP later succeeded.

Neither timer called runs_get. A later read found the same persisted generation sequence and update timestamp for both completed timer executions.

@Robert0622

Robert0622 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

定时任务超时与 SOP 终态实测

复测通过。 在提交 0c3e528b 上,由调度器自动触发两条提示词任务。定时助手和 SOP 的 Agent 节点都使用 CodeBuddy 的 DeepSeek V4.1 Flash。每个 SOP 都由模型写入一个唯一标记,再由脚本校验。两条运行均只派发一次,已从 MySQL 读回并核对标记和完成产物。

版本与场景 实际结果
v0.4.0,设置 1 分钟超时 保存后丢失超时设置。任务对同一运行调用 runs_get 37 次,在 121.7 秒后才报告 SOP 成功。
初版修复 7382983f,模型把启动 SOP 放在最终回复草稿里 定时任务在 15.0 秒就记为成功,此时 SOP 仍在执行,会话已经结束。
当前提交,短 SOP 一次 runs_wait 收到 succeeded,38.046 秒后定时任务报告正确的运行 ID 和终态。
当前提交,长 SOP,超时设为 1 分钟 网关拒绝最终回复里的启动草稿,模型随后直接启动一次 SOP。第二次 runs_wait 在截止时被取消(HTTP 408)。定时任务在 60.156 秒记录 EXECUTION_TIMEOUT,已派发的 SOP 随后独立成功。

以上两条当前提交的运行均未调用 runs_get。前一轮独立复测也观察到短任务在 40.398 秒正确结束、长任务在 60.101 秒超时,超时后再次读取该会话,事件序号和更新时间保持稳定。此次验证覆盖了等待和超时行为。线上磁盘写满的具体原因还需事发时的数据库记录确认。

本轮执行记录(两条均为定时触发)

定时任务执行记录,长任务按时超时,短任务报告正确的 SOP 运行 ID 和成功状态

已保存的 1 分钟超时和 Flash 模型配置

长任务编辑页中的 CodeBuddy、DeepSeek V4.1 Flash 和 1 分钟超时

草稿回退、单次启动,以及超时取消等待的网关记录

网关拒绝结束会话的草稿,随后记录一次启动和两次等待,最后一次返回 HTTP 408

定时会话超时后,独立 SOP 的最终结果

长 SOP 最终成功,Flash 写入和脚本校验两个节点均已完成

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant