[rig-tasks] Add 10 rig samples — 2026-08-27 - #496
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /codebase-design — requesting changes on correctness issues found in 5 of 10 samples.
📋 Key Findings
Bugs / Correctness
- 472 —
conflictRiskAgentscansgit status --shortfor conflict markers (<<<<<<<,>>>>>>>), but those markers only appear in file contents during an active merge conflict, never ingit statusoutput.conflictCountwill always be 0. - 477 —
resolvedPathoutput is typeds.string(required) but the tool returnsnullfor builtins and failures; schema mismatch will cause repair loops or coerced"null"strings. - 479 —
npm audit --json | head -5000can truncate JSON mid-object;|| echofallback applies only on non-zero exit, not on truncation, so a large audit silently returns garbage.
Design Issues
- 477 —
p.readOptional('package.json')reads the project root, not the resolved dependency's ownpackage.json; metadata will always be the host project's. - 475 — Schema content is inlined into the prompt via
p.readInputand then re-passed as a raw string to the tool for a secondJSON.parse; uses.pathin the tool to read it once.
Minor / Style
- 471 —
stat()is called outside the try/catch that guardsreadFile; inconsistent error handling. - 478 — Uses MD5 when
sha256is identical API effort and is the more correct choice for a checksum sample. - 476 — Coordinator relays data between subagents via prose instructions; structured agent input chaining is more idiomatic in rig.
- 480 — Conditional-type regex has documented false-positive risk; worth a comment in the sample.
Positive Highlights
- ✅ Good variety of patterns across the 10 samples (multi-agent coordination,
defineTool,p.glob,steering()+repair()). - ✅ All samples use
model: 'small'consistently per project style. - ✅
p.readOptionalfallback pattern in 474 and 477 is clean.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 49.2 AIC · ⌖ 4.97 AIC · ⊞ 6.3K
Comment /matt to run again
|
|
||
| // Agent role: resolve an npm module path and extract its package.json metadata. | ||
| const nodeModulePathResolver = agent({ | ||
| model: "small", |
There was a problem hiding this comment.
[/codebase-design] p.readOptional("package.json", "{}") always reads the project root package.json, not the resolved module's own package.json. Metadata returned (name, version, main, types) will reflect the host project, not the dependency.
💡 Suggested fix
Use require.resolve(moduleName + '/package.json') directly in the resolveModulePath tool and return the parsed fields there, eliminating the misleading p.readOptional call.
|
|
||
| const computeChecksum = defineTool("computeChecksum", { | ||
| description: "Compute the MD5 checksum and byte size of a file.", | ||
| parameters: s.object({ filePath: s.path }), |
There was a problem hiding this comment.
[/codebase-design] MD5 is not collision-resistant and is deprecated for integrity use cases. Since this is a sample demonstrating node:crypto, it's better to use sha256 — same API, more correct for any real checksum use.
💡 One-line fix
const checksum = createHash('sha256').update(buf).digest('hex');Samples teach patterns; teaching MD5 normalises a weak algorithm when sha256 costs nothing extra.
| description: "Read a git hook file and determine its shebang line and executability.", | ||
| parameters: s.object({ hookPath: s.path }), | ||
| handler: async ({ hookPath }) => { | ||
| const { readFile, stat } = await import("node:fs/promises"); |
There was a problem hiding this comment.
[/codebase-design] stat(hookPath) will throw if the hook file doesn't exist (e.g. a path constructed from ls output with trailing whitespace or encoding issues). The error bubbles out of the defineTool handler uncaught, while the inner readFile error is silently swallowed. These two error-handling levels are inconsistent.
💡 Suggested fix
Wrap stat in the same try/catch as readFile, or guard both at the top level:
try {
const info = await stat(hookPath);
// ...
} catch {
return { isExecutable: false, shebang: undefined };
}| // Agent role: orchestrate changelog generation by running collector, classifier, then formatter. | ||
| const changelogCoordinator = agent({ | ||
| model: "small", | ||
| agents: { commitCollector, commitClassifier, changelogFormatter }, |
There was a problem hiding this comment.
[/codebase-design] The coordinator passes data between subagents via prompt instructions ("pass the commits array to commitClassifier"), but commitClassifier and changelogFormatter declare typed input schemas. The coordinator should use the input: field of those agents rather than relying on the LLM to faithfully relay the data through prose instructions — that's exactly what the structured input/output pipeline is for.
💡 Pattern note
The rig docs call this chaining via the coordinator's agent calls. Each subagent call passes structured output as the next agent's input, reducing hallucination risk. The current instruction prose "pass the commits array" works but is fragile for larger payloads.
|
|
||
| // Agent role: run npm audit and count vulnerabilities. | ||
| const dependencyAuditor = agent({ | ||
| model: "small", |
There was a problem hiding this comment.
[/codebase-design] The npm audit --json output is piped through head -5000 before being parsed. If the JSON is truncated mid-object, JSON.parse will fail and the fallback echo string won't apply (it only applies when the command itself fails, not when the output is truncated). A truncated audit on large projects will produce silent wrong results.
💡 Suggested fix
Remove the head -5000 pipe, or capture the exit code and use || echo '{...}' on the full npm audit command rather than after piping:
npm audit --json 2>/dev/null || echo '{"metadata":{"vulnerabilities":{"total":0,"critical":0}}}'| output: s.object({ | ||
| resolvedPath: s.string, | ||
| packageName: s.string, | ||
| version: s.string, |
There was a problem hiding this comment.
[/codebase-design] resolvedPath is typed as s.string (non-optional) in the output schema, but the tool returns resolvedPath: null for built-in modules or resolution failures. The agent will attempt to fill a required string field with null, likely triggering a repair loop or returning an invalid "null" string.
💡 Suggested fix
Change the output schema to reflect the nullable reality:
resolvedPath: s.optional(s.string),And update the instructions to handle the built-in case gracefully.
| description: "Recursively count JSON Schema properties at each depth level.", | ||
| parameters: s.object({ schemaJson: s.string }), | ||
| handler: async ({ schemaJson }) => { | ||
| const schema = JSON.parse(schemaJson); |
There was a problem hiding this comment.
[/codebase-design] The agent's instructions use p.readInput("schemaFile") to inline the file content into the prompt, but then passes it again to countPropertiesAtLevel as a raw JSON string that gets JSON.parsed in the tool handler. For large schemas this doubles the token cost — the full schema appears in both the prompt and the tool call. A defineTool that accepts a filePath: s.path and reads the file itself would be cleaner and avoid the double-serialization.
💡 Suggested alternative
const countPropertiesAtLevel = defineTool('countPropertiesAtLevel', {
parameters: s.object({ filePath: s.path }),
handler: async ({ filePath }) => {
const { readFile } = await import('node:fs/promises');
const schema = JSON.parse(await readFile(filePath, 'utf-8'));
// ... same walk logic
}
});Then the instruction can just say: call countPropertiesAtLevel with the schemaFile path.
| // Agent role: check for conflicts in the working tree. | ||
| const conflictRiskAgent = agent({ | ||
| model: "small", | ||
| instructions: p`Analyze ${p.bash("git status --short 2>/dev/null || echo ''")} and count lines containing conflict markers (<<<<<<, >>>>>>). Return conflictCount and hasConflicts.`, |
There was a problem hiding this comment.
[/codebase-design] git status --short does not show conflict markers (<<<<<<<, >>>>>>>); those only appear in file contents after a failed merge. Counting conflict-marker lines in git status --short output will always return 0. To detect conflict markers, grep the working tree instead:
grep -rl '<<<<<<<' . --include='*.ts' --include='*.js' 2>/dev/null | wc -l| parameters: s.object({ filePath: s.path }), | ||
| handler: async ({ filePath }) => { | ||
| const { readFile } = await import("node:fs/promises"); | ||
| const content = await readFile(filePath, "utf-8"); |
There was a problem hiding this comment.
[/codebase-design] The regex /\w+\s+extends\s+[^?]+?\s*[^:]+:/g will match value-level if (x extends ...) patterns in .js files and also plain interface Foo extends Bar declarations (which have no ?). Since the glob is src/**/*.ts, interface extends will be filtered out by the ? requirement — but the regex is still ambiguous and will match type narrowing in function bodies.
💡 Tighter pattern
Anchor to type-level context with a tighter pattern, e.g.:
/type\s+\w[^=]*=\s*[^?]+\?/gOr document the known false-positive rate in the sample so readers know the limitation.
Summary
Added 10 new rig sample files to
skills/rig/samples/.defineTool+node:fs/promisesstats.enumnode:child_processexecSyncp.readOptional+steering()+repair()defineToolhandlernode:modulecreateRequirenode:cryptos.enumhealth statusp.glob+defineToolTypecheck failures
None — all 10 samples passed typecheck.
Tasks run
defineToolwithnode:fs/promisesstat and shebang detections.enumrecommendationnode:child_processexecSync indefineTool, annotated/lightweight classificationp.readOptionaltsconfig +p.bashgrep +defineToolalias mappingdefineToolhandler,s.record(s.int)depth mapnode:modulecreateRequire,p.readOptionalfallbacknode:crypto+node:fs/promises,steering()+repair()addonss.enum("healthy","warning","critical")p.globfan-out,defineToolregex-based conditional type detection