feat(client): Agent Skills — a distinguishable outcome for integrity failure (Gap 2 / LA-2) - #58
Open
XieX wants to merge 1 commit into
Open
feat(client): Agent Skills — a distinguishable outcome for integrity failure (Gap 2 / LA-2)#58XieX wants to merge 1 commit into
XieX wants to merge 1 commit into
Conversation
…failure ``get_skill`` returns ``None`` for four unrelated outcomes: no such skill, the store raised, the requested version is not the one held, and content that failed hash verification. A caller cannot fail closed on suspected tampering while tolerating a merely-absent skill, so no automated customer-side response is possible — finding LA-2 of the Agent Skills security design review. The information already existed internally, as prose in ``Resolution.error``. This gives it a token: ``Resolution`` grows a typed ``reason``, set explicitly at every construction site and declared without a default so a sixth outcome added later has to choose which public token it maps to. ``get_skill_result`` maps that straight through to a frozen ``SkillOutcome`` (``skill``, ``reason``, ``detail``). Deriving the public reason by matching the error string is the fragility LA-2 is about, so the mapping is readable in one table. ``get_skill`` is untouched — its ``None``-for-every-failure contract is documented in its docstring and in the README, and a test now pins that all four failures still collapse to ``None`` and still never raise. Nothing new is emitted: Gap 1's integrity record already fired inside verification before ``resolve_from_store`` returned, and a test asserts one failed retrieval still produces exactly one record and one signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gap 2 (finding LA-2): a distinguishable outcome for integrity failure versus absence
This PR adds public API and needs review as such. Three new exported names:
get_skill_result,SkillOutcome,SkillOutcomeReason.The defect
get_skillreturnsNonefor four distinct outcomes — no such skill, the store raised, therequested version is not the one held, and content that failed hash verification. A customer
therefore cannot fail closed on suspected tampering while tolerating a merely-absent skill,
so no automated customer-side response is possible. LA-1 (shipped in #51) gave the
operator a structured log record; this gives the application the same distinction.
The information already existed internally.
Resolutioninskills_core.pydistinguishedall four cases — but only as prose in its
errorstring, whichget_skilldiscarded with.skill.The API
Chosen over a typed exception, an opt-in strict mode, and an init-time callback.
detailisthe existing human-readable reason string — already safe to surface (skill key and failure
mode only, never content, never a filesystem path), and a test pins that it stays that way.
get_skill_resultraisesRuntimeErroronly when no store is configured, with the samemessage as
get_skill.get_skilldoes not changeIts contract —
Nonefor every failure, never raises for one — is documented in itsdocstring and in the README, and every existing caller treats
Noneas "no skill". A testdrives all four failures through both accessors: the reason is distinguishable and the
collapsed form still collapses.
Not matched on the error string
Resolutiongrows a typedreason, set explicitly at every construction site. Deriving thepublic reason by pattern-matching
Resolution.erroris exactly the fragility LA-2 is about,so the mapping is a table a reviewer can read:
resolve_from_storeoutcomereasonunavailable=True)store_unavailablerawis not a dictabsentverify_raw_skillreturnedNoneintegrity_failureskill.version != wanted_versionwrong_versionokreasonis declared first and without a default, so a contributor adding a sixthinternal outcome has to decide which public token it maps to rather than inheriting one. That
made the two
Resolutionsites inskills_fs.pyexplicit as well — both are the"could not retrieve" path, both already
unavailable=True, both now sayreason="store_unavailable".Resolution.unavailableis unchanged. It stays load-bearing on the prune path — only araising store suppresses pruning, because deleting managed files after a failed lookup would
turn an outage into data loss — and
store_unavailablestays distinct fromabsentfor thesame reason.
Three things deliberately not built
ld.skills.integrity_failurerecord alreadyfired inside verification before
resolve_from_storereturned. A test asserts one failedretrieval still produces exactly one log record and exactly one signal.
IntegrityReasonCodeis not threaded intoSkillOutcome.verify_raw_skillreturnsNoneand does not surface which token fired; plumbing it upwould change that function's return type for a detail the operator already gets from the
log record. The five-token public reason is the actionable surface.
get_skillsandall_skillskeep omitting failed entries andkeep logging the run-level WARN count. A
get_skills_result/all_skills_resultwoulddouble the accessor surface for a case nobody has asked for — possible follow-up if a
customer needs per-key reasons from a batch.
Tests
packages/client/tests/test_skills.py, one new class plus three additions to existingexport/immutability tests:
reason, whetherskillis populated, non-emptydetailstore_unavailableis distinct fromabsent: a raising store and an empty storeokoutcome carries a detail (swept, so a fifth failure path with no message iscaught by a test whose name says what it is about)
get_skillstill returnsNone, and still never raises, for all four failuresget_skill_resultraisesRuntimeErrorwith no store, asserted equal toget_skill's messageget_skill_resultdetailnever carries the skill contentSkillOutcomeis frozen;SkillOutcomeReason's tokens and the three new exports are pinnedDocs
packages/client/README.md—get_skill_resultin the Agent Skills API table, afail-closed example under the Gap 1 observability material (exit on
integrity_failure,tolerate
absent), the reason table, and an explicit statement thatget_skillisunchanged and the two accessors differ only in what they report.
packages/client/agents.md— the five-token vocabulary, theResolution→ reason mapping,and the instruction that a sixth internal outcome must choose a public token rather than
defaulting to
absent.Lockstep with TypeScript
The type name, the accessor name, and the five reason tokens are fixed across both SDKs; a
sibling PR implements the identical shape in
typescript/. Tokens are alphabetical,matching how
IntegrityReasonCodewas written on both sides in Gap 1.Placement
Based on
split/skills-self-healing(#57), which is based on #54. A new PR at the top of thestack so it disturbs none of the existing reviews: Gap 2 spans
types.py,skills.py,skills_core.pyand__init__.py, which #50–#53 own, and #52/#53/#54 are approved while#50/#51 have live reviewer conversation.
Checks
uv run pytest(1481 passed, 11 skipped),ruff check,ruff format --checkall clean.uv run mypy .fails repo-wide on a pre-existing duplicate-conftesterror before checkinganything; scoped to the files this PR touches, mypy reports only two pre-existing
unused-ignorewarnings that are present on the base commit too.🤖 Generated with Claude Code
Note
Overview
Adds
get_skill_resultand frozenSkillOutcome/SkillOutcomeReasonso applications can tell integrity failure apart from a missing skill, store outage, or version mismatch—without changingget_skill(Nonefor all failures, same path, no extra telemetry).Internal
Resolutionnow carries a requiredreasonmapped explicitly at eachresolve_from_storesite (includingskills_fsstore-unavailable paths);get_skill_resultexposesskill,reason, and safe-to-logdetail.README and
agents.mddocument fail-closed handling; tests cover all five reasons, immutability, exports, no double integrity signals, and unchangedget_skillbehavior.Reviewed by Cursor Bugbot for commit eda4ae1. Bugbot is set up for automated code reviews on this repo. Configure here.