Skip to content

fix(node): ANS-104 codec and recovery policy types (#26 split 2/4) - #524

Merged
beardthelion merged 16 commits into
mainfrom
fix/issue-26-split-2-arweave-transport
Nov 5, 2026
Merged

beardthelion merged 16 commits into
mainfrom
fix/issue-26-split-2-arweave-transport

Conversation

@beardthelion

@beardthelion beardthelion commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds the ANS-104 data item codec and the three-outcome recovery policy types for #26 (split 2/4). Continues #385 by @Gravirei. The 13 commits are carried over unchanged and keep their authorship; main was merged in as a separate commit.

Motivation & context

Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split the work into four narrow PRs. This is Split PR 2.

The P1 finding the reviewer assigned to this split: the legacy anchor_item_present mapped BAD_REQUEST and GONE to Ok(false), and the recovery code interpreted false as permission to reclaim the row and pay for another upload. Neither 400 nor 410 proves a previously paid item was never accepted: 410 can describe an artifact that existed but is no longer served, and 400 can be produced by gateway, proxy, or routing failure. The concrete failure sequence the reviewer named: bundler accepts the item, the response or terminal DB write is lost, recovery probes the persisted item_id, the gateway returns 400 or 410, and the node pays for a second immutable item for the same transition.

Fix: model the probe result as three outcomes, and authorize a re-upload only on a trustworthy, protocol-defined absence.

Round-5 direction (reviewer 2): land here only the standard-compatible ANS-104 primitives and the pure three-outcome policy types. The gateway probe, the verify_anchor path, and the public verify endpoint are deferred until the uploader, retrieval proof, and recovery consumer can be proven as one vertical slice against a real provider contract (same-network upload/read pair, redirect-safe client, envelope-supplying read API, provider-documented absence evidence).

Refs #26

Kind of change

  • Bug fix
  • Feature
  • Security fix
  • Docs
  • Tests / CI
  • Refactor (no behavior change)
  • Breaking or protocol change (issue required first)

What changed

Crate touched: gitlawb-node (plus golden-vector scripts under scripts/).

  • New module ans104: ANS-104 data item (de)serialization, deep-hash matching arbundles@0.10.x getSignatureData (8-element fold, verified byte-exact against the JS implementation), Ed25519 sign/verify (strict), full two-byte signature-type validation, raw-tag-payload preservation across parse/hash/encode. unit tests including a real arbundles-signed Ed25519 golden vector (verified without re-signing), multi-block and size-prefixed tag blocks, and a non-canonical-sigtype rejection test.
  • New module arweave_v2 (policy types only): the ProbeOutcome model the reviewer demanded — Present / DefinitivelyAbsent / Indeterminate — plus the permits_reupload rule, with a unit test. No gateway client, no endpoint.
  • Golden-vector scripts (scripts/ans104_golden_ed25519.mjs, output doc) capturing the interop fixture from the pinned arbundles dependency.

Deferred to the vertical slice (explicitly NOT in this PR): the gateway probe (probe_anchor_item), the verify_anchor path, GET /api/v1/arweave/anchors/verify/{item_id}, the irys_tx_id lookup method and index migration, and the gateway/rate-limit config knobs. There is therefore no provider response table here: DefinitivelyAbsent is authorized only by whatever evidence the future provider contract defines, and every ambiguous response stays retryable.

How a reviewer can verify

cargo test -p gitlawb-node --bin gitlawb-node ans104
cargo test -p gitlawb-node --bin gitlawb-node arweave_v2
cargo fmt --all -- --check
cargo clippy -p gitlawb-node --all-targets -- -D warnings

Full workspace suite runs on CI.

crates/gitlawb-node/src/ans104.rs::tests:

  • dataitem_matches_arbundles_golden_vector — real arbundles-signed Ed25519 item: shape pin, id pin, deep-hash pin against getSignatureData, verify without re-signing against an independently pinned expected key, byte-exact binary round-trip.
  • signed_binary_parse_and_verify_multi_block_and_negative_count — non-canonical tag encodings hash the original payload bytes and verify without re-signing.
  • decode_tags_accepts_negative_block_count_with_size — legal size-prefixed Avro form parses.
  • from_binary_rejects_non_canonical_signature_type_high_byte — wire 02 01 (258) is rejected, never aliased onto Ed25519 type 2.
  • binary_round_trip — signature slot, canonical owner, and payload survive to_binary/from_binary with the signature intact (no re-sign after parse).
  • sign_then_verify_round_trips, flipped_signature_byte_fails_verify, wrong_expected_pubkey_fails_verify, mutated_data_fails_verify, wire_shape_round_trip, deep_hash_is_stable, deep_hash_empty_tags_is_distinct_from_one_tag, data_item_id_is_base64url_of_sha256_of_signature — round-trip, tamper, and derivation pins.
  • external_reference_vectors (3 tests) — SHA-384 deep-hash primitives against the reference suite.

crates/gitlawb-node/src/arweave_v2.rs::tests:

  • only_definitively_absent_authorizes_reupload — the policy: collapsing any ambiguous outcome to absent spends a second immutable upload for the same transition.

Each test names the invariant it pins. Reverting the covered line turns the assertion red.

Before you request review

  • Scope is one logical change; no unrelated churn
  • cargo test --workspace passes locally (the ans104 filter passes on the merged tree; the full suite runs on CI)
  • New behavior is covered by tests (required for fixes)
  • cargo fmt --all and cargo clippy --workspace --all-targets -- -D warnings are clean
  • Commit titles use Conventional Commits (feat(...), fix(...), docs(...))
  • Docs / .env.example updated if behavior or config changed (N/A: no config or behavior change)
  • Checked existing PRs so this isn't a duplicate

Protocol & signing impact

  • Touches DID / did:key, Ed25519 / RFC 9421 signatures, UCAN, ref certs, or P2P wire formats
  • Discussed in an issue before implementation
  • Backward-compatible with existing nodes and previously signed history

Notes for reviewers

ProbeOutcome::permits_reupload is the policy the reviewer demanded: only DefinitivelyAbsent authorizes a paid re-upload. Indeterminate keeps the outbox non-terminal; the next probe (or a future retry) gets another chance to give a trustworthy answer. The exact evidence that establishes DefinitivelyAbsent is defined by the provider contract in the vertical slice that introduces the probe — never by a mock-only body shape.

Why this is its own PR

PR 2 owns ANS-104 serialization/verification and recovery outcome classification. The bundler upload itself (the handler that builds a DataItem and POSTs to the bundler), the provider-backed probe, and the public verification endpoint live in later slices. The ANS-104 module is fully tested and ready for the bundler upload to call into. PR 1's recovery drain consumes ProbeOutcome::permits_reupload; the probe that produces the outcome arrives with the provider contract.

Overlap with open PRs

Safety to land standalone

  • It compiles, runs, and passes its focused tests by itself. No sibling PR required.
  • No migration shipped, and no released migration edited. The arweave_anchors table is untouched.
  • No endpoint, route, config knob, or rate-limiter state added. The public API surface is unchanged.
  • Purely additive: two new modules plus golden-vector scripts. Nothing existing is modified except main.rs/server.rs module wiring and a comment.

Summary by CodeRabbit

  • New Features
    • Added support for creating, parsing, and encoding ANS-104 data items, including Ed25519 signing and signature verification.
    • Added item ID and hash generation, plus validation of item structure and signatures.
    • Preserved original tag encoding when reading items, supporting byte-for-byte round-tripping of compatible inputs.
    • Added a re-upload decision rule that permits re-upload only when an item is definitively absent.
  • Tests
    • Added checks against reference ANS-104 items for IDs, hashes, signatures, and binary encoding.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

crate:node gitlawb-node — the serving node and REST API kind:bug Defect fix — wrong or unsafe behavior subsystem:storage Blob/object store, Arweave, IPFS, archives

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants