ADR: raw microdata in Chronicle is identity, not content - #222
Open
MaxGhenis wants to merge 11 commits into
Open
ADR: raw microdata in Chronicle is identity, not content#222MaxGhenis wants to merge 11 commits into
MaxGhenis wants to merge 11 commits into
Conversation
Register every raw microdata release consumers build from (publisher, vintage, sha256, licence, access class) as a source artifact; archive bytes only for redistributable public-use files; never parse microdata into rows or facts and never hold licensed bytes. Narrows the storage and governance non-goals accordingly and points README, architecture, and AGENTS at the ADR. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 2, 2026
…icrodata identity ADR Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Require kind: microdata_release; define content as rows, cells, or facts; say licensed or restricted bytes everywhere; narrow the fact prohibition to facts computed directly from raw microdata; reference the country-aware key convention; name the pending fail-closed path (chronicle#221); extend the ledger-boundary judge contract and the package docstring to match. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…GENTS, the judge contract, and the package docstring Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ody, canonical prohibition vocabulary, hash attestation semantics, scoped harness plan Byte custody now requires access public AND a licence on the redistributable allowlist; the prohibition reads identically in the ADR, governance doc, judge contract, README, architecture table, AGENTS, and package docstring (records, rows, columns, row values, cells; facts derived from raw microdata by Chronicle or a consumer; licensed or restricted bytes) and the governance test pins the judge wording; hash_source and verified_at define what a registration attests; the storage matrix says what exists today; the harness plan is scoped to publisher aggregates; the key convention states the unmapped-publisher fallback. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ification, artifact-bound licence evidence, provenance lineage as the enforcement path, one shared boundary contract Every new or modified manifest declares kind (frozen grandfather list for the legacy set); byte custody needs licence_evidence binding the artifact, not just an allowlisted licence name; the derived-fact prohibition names its enforcement (asserting principal + root_artifacts, chronicle#238) and is a review obligation until then; attestation records who attested and with what evidence; README and the package docstring are prospective until #221 lands; public microdata bytes are staged untracked; the facts-only ADR and README boundary carry the microdata-grain exception; the three negative clauses live once in chronicle/boundary.py and are asserted verbatim in the judge contract and the governance doc. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ho asserted the value, align byte-custody wording, drop the phantom command, rewrap
…sequences bullet, and the architecture table; rewrap
…nning test asserts the clause count
MaxGhenis
added a commit
that referenced
this pull request
Sep 4, 2026
Adds the microdata-release example to the storage doc's Object Key Conventions (public releases keep the ordinary content-addressed key; licensed and restricted ones have no key at all), a Hash-Only Registrations section to the harness doc, a register-artifact entry to the README artifact command block, and a data-sources note recording why BE-SILC 2023 cannot be registered yet. The governance, architecture, AGENTS, and non-goal amendments are left to the ADR branch (PR #222), which already makes them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 4, 2026
The ADR on PR #222 says every manifest declares its kind, that bytes are archived only under an allowlisted licence with artifact-bound evidence, and that each registration names who asserts its checksum. None of that was in the registration vocabulary: - manifest_kind() no longer defaults a manifest with content to publisher_table. Manifests that predate the rule are frozen byte for byte in chronicle/grandfathered_manifests.py; any other kindless manifest is a ManifestKindError. - chronicle/licences.py is the allowlist of redistributable terms (a U.S. Government work, OGL v3, CC0, CC BY 4.0), each with its evidence URL, and the validator for a licence_evidence block binding one artifact to one term. - hash_source is the closed set chronicle_fetch / consumer_attested / consumer_pin, each with its attester fields; register_hash_only_artifact requires one and refuses a consumer_pin with a verified_at. - Filenames are bare names compared case-folded (bare_filename, filename_key), 2023 and '2023' are one vintage key (resolve_vintage_key), and validate_manifest_files reports duplicate keys, non-canonical names and filename collisions across a whole manifest. - register_hash_only_artifact refuses to reclassify an entry that is public or records an R2 object, resolves the vintage key the manifest already uses, and validates the manifest before touching it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 4, 2026
Adds the microdata-release example to the storage doc's Object Key Conventions (public releases keep the ordinary content-addressed key; licensed and restricted ones have no key at all), a Hash-Only Registrations section to the harness doc, a register-artifact entry to the README artifact command block, and a data-sources note recording why BE-SILC 2023 cannot be registered yet. The governance, architecture, AGENTS, and non-goal amendments are left to the ADR branch (PR #222), which already makes them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 4, 2026
The ADR on PR #222 says every manifest declares its kind, that bytes are archived only under an allowlisted licence with artifact-bound evidence, and that each registration names who asserts its checksum. None of that was in the registration vocabulary: - manifest_kind() no longer defaults a manifest with content to publisher_table. Manifests that predate the rule are frozen byte for byte in chronicle/grandfathered_manifests.py; any other kindless manifest is a ManifestKindError. - chronicle/licences.py is the allowlist of redistributable terms (a U.S. Government work, OGL v3, CC0, CC BY 4.0), each with its evidence URL, and the validator for a licence_evidence block binding one artifact to one term. - hash_source is the closed set chronicle_fetch / consumer_attested / consumer_pin, each with its attester fields; register_hash_only_artifact requires one and refuses a consumer_pin with a verified_at. - Filenames are bare names compared case-folded (bare_filename, filename_key), 2023 and '2023' are one vintage key (resolve_vintage_key), and validate_manifest_files reports duplicate keys, non-canonical names and filename collisions across a whole manifest. - register_hash_only_artifact refuses to reclassify an entry that is public or records an R2 object, resolves the vintage key the manifest already uses, and validates the manifest before touching it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 5, 2026
Adds the microdata-release example to the storage doc's Object Key Conventions (public releases keep the ordinary content-addressed key; licensed and restricted ones have no key at all), a Hash-Only Registrations section to the harness doc, a register-artifact entry to the README artifact command block, and a data-sources note recording why BE-SILC 2023 cannot be registered yet. The governance, architecture, AGENTS, and non-goal amendments are left to the ADR branch (PR #222), which already makes them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 5, 2026
The ADR on PR #222 says every manifest declares its kind, that bytes are archived only under an allowlisted licence with artifact-bound evidence, and that each registration names who asserts its checksum. None of that was in the registration vocabulary: - manifest_kind() no longer defaults a manifest with content to publisher_table. Manifests that predate the rule are frozen byte for byte in chronicle/grandfathered_manifests.py; any other kindless manifest is a ManifestKindError. - chronicle/licences.py is the allowlist of redistributable terms (a U.S. Government work, OGL v3, CC0, CC BY 4.0), each with its evidence URL, and the validator for a licence_evidence block binding one artifact to one term. - hash_source is the closed set chronicle_fetch / consumer_attested / consumer_pin, each with its attester fields; register_hash_only_artifact requires one and refuses a consumer_pin with a verified_at. - Filenames are bare names compared case-folded (bare_filename, filename_key), 2023 and '2023' are one vintage key (resolve_vintage_key), and validate_manifest_files reports duplicate keys, non-canonical names and filename collisions across a whole manifest. - register_hash_only_artifact refuses to reclassify an entry that is public or records an R2 object, resolves the vintage key the manifest already uses, and validates the manifest before touching it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
docs/adr-chronicle-raw-microdata-identity.mdand amends the boundary docs to match. The store stays facts-only. What changes is narrow: Chronicle registers every raw microdata release its consumers build from (publisher, access route, vintage, SHA-256, licence, access class) as a source artifact, archives bytes only for redistributable public-use files, and never parses microdata into rows or facts or holds licensed bytes.Why: raw pins in Microcosm are scattered across four mechanisms with no shared registry, licence record, or witnessed fetch time; publishers revise and withdraw files (IRS PUF, 2026); the node graph's roots deserve declared identity. Content stays out for the reasons in the ADR (registry and journal scale, licence terms, Thesis resolution substrate). The 2026-06-30 removal of microdata surfaces (#68) stands.
Docs touched: README boundary paragraph,
docs/architecture.mdownership row,docs/storage-architecture.md(raw-archive paragraph, ownership matrix row, Non-Goals),docs/chronicle-governance.md(may / must-not lists, judge rule),AGENTS.md.Implementation is tracked in #221 (manifest
licence/accessfields, hash-only registration, first registrations) and PolicyEngine/microcosm#848 (consumer-side references and fail-closed gate).Chronicle Governance
This PR changes the documented source-data boundary (docs only, no code, no source packages, no schemas). None of the three agent roles' allowed paths cover docs; requesting a
ledger-boundaryjudge read from the reviewer. Deterministic checks:tests/test_chronicle_boundaries.py,tests/test_chronicle_governance.pypass.Tests
uv run pytest -q tests/test_chronicle_boundaries.py tests/test_chronicle_governance.py— 7 passed.🤖 Generated with Claude Code