Skip to content

fix(5207): merge ADR/RFC/REQ reference-id concepts across files - #5348

Open
Deepjyoti-Sarmah wants to merge 2 commits into
Graphify-Labs:v8from
Deepjyoti-Sarmah:fix/5207-reference-concept-identity
Open

Deepjyoti-Sarmah wants to merge 2 commits into
Graphify-Labs:v8from
Deepjyoti-Sarmah:fix/5207-reference-concept-identity

Conversation

@Deepjyoti-Sarmah

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #5207 for reference ids. ADR-0007 cited from several files now ends up as one concept node, not one per file.

Each file's mention gets its own path-prefixed id, as the extraction spec requires. deduplicate_entities should then merge them by label, and two things stopped it:

Case (3 files, concept) v8 Cause
ADR-0007 ×3 3 nodes label entropy 2.41, under the 2.5 gate meant for generic words like "API"
ADR 7 / ADR-0007 / ADR-007 3 nodes they normalize to adr 7 and adr 0007, so pass 1 never groups them
RFC-12, REQ-42, packages/auth ×3 1 node already fine

Lowering the gate to 2.0 merges the first row, which confirms the gate is the cause. Across 1–999 the gate rejects ADR-1…ADR-9 and ADR-0001…ADR-0011 (36 padded labels), the first and most-cited records.

The change, in pass 1:

-key = _norm(label)
+key = _reference_key(node) or _norm(label)     # concept + whole label ADR|RFC|REQ <n>  ->  "adr-7"
 ...
-and _entropy(label) >= _ENTROPY_THRESHOLD
+and (_entropy(label) >= _ENTROPY_THRESHOLD or _reference_key(n) is not None)

The key ignores separator, case and zero padding. The prefix list is fixed, so other short labels keep the gate.

Options I tried

  • A, canonical key in dedup (this PR). Deterministic and testable; ids and the spec stay as they are.
  • B, spec tells the LLM to mint adr_0007. Depends on model output, can't be pinned in CI, and would reach document nodes, which 0.9.108 blocks on purpose.
  • C, lower the entropy gate. Brings back merges of short generic labels.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Tests or CI
  • Refactor
  • Security fix

Verification & Invariants

Invariant: concept nodes whose whole label names the same ADR/RFC/REQ number merge into one node across files, whatever the spelling; nothing else merges that did not before.

  • Read the CONTRIBUTING.md guide.
  • Reproduced the issue and identified the invariant.
  • Made the smallest fix necessary.
  • Added a regression test (if bug fix) or isolated boundary test.
  • Kept the PR description synchronized with the final implementation.
  • Documented any limitations / unsupported cases explicitly.

Merge Danger

Door: two-way. One helper and two lines in graphify/dedup.py. No id, spec, schema or cache change; reverting the commit restores 0.9.112.

Blast Radius: small. Only concept nodes labelled exactly ADR|RFC|REQ <number> merge where they did not. document and rationale keep the cross-file block (#3094, 0.9.108), and the incremental protected_ids rules are unchanged.

Limitations

  • document/rationale mentions of an ADR stay one per file, by design.
  • Package aliases (@acme/auth vs packages/auth) still split; that needs a workspace map.
  • Ids keep their path prefix; only the merge changes. Other id schemes (PROJ-123) are not covered.

How was this tested?

tests/test_dedup_reference_ids.py (12):

merge to one:  ADR-0007 x3 | ADR 7/ADR-0007/ADR-007 | RFC-1/rfc 1/RFC1 | REQ-42/REQ 42/req-0042
               edges rewired to the survivor
stay apart:    ADR-7/ADR-8/ADR-0007 -> 2 | ADR-7/RFC-7/REQ-7 -> 3 | XYZ-0007 x3 -> 3
               document x3, rationale x3 -> 3 | "ADR-0007 context" -> not an id
incremental:   two protected + one new -> 2

On 0.9.112 340268a7: 7 fail, 5 pass (the guards). On this branch: 12 pass.

Full suite, same venv:

failed passed
0.9.112 76 11515
this branch 76 11527

The 76 are the same tests on both trees: subprocess python -m graphify tests run the shared venv's editable install, which points at another checkout.

deduplicate_entities on 30,000 synthetic concept nodes, 3 runs each: 11.4–12.1 s before, 11.6–11.7 s after. Output 23,113 → 22,675 nodes; the 438 are ADR mentions that now merge.

ruff check graphify/dedup.py tests/test_dedup_reference_ids.py -> All checks passed

Graphify-specific checklist

  • I updated generated skill artifacts (uv run python -m tools.skillgen --bless) when changing their source fragments. (N/A: no skill change.)
  • I confirmed that AST/structural extraction remains deterministic (no ambient state dependencies like ENV variables).
  • I reviewed changes for security implications (no unsafe interpolation into shell/Python). (N/A: label matching only.)
  • I confirmed no API keys or local-only graph data are included.
  • (If applicable) I disclosed AI authorship in my commit messages. (No trailer in the commit messages.)

ADR-0007 cited from several files stayed one node per file: its label fell
under the entropy gate, and 'ADR 7' / 'ADR-0007' normalized to different keys.
Concept nodes whose whole label is a known reference id now share a key that
ignores spelling and zero padding, and skip the entropy gate. document and
rationale nodes keep the cross-file block.
@github-actions github-actions Bot added the contributor Pull request from a returning community contributor label Nov 19, 2026
@github-actions

Copy link
Copy Markdown

Thanks for the pull request, @Deepjyoti-Sarmah. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.11 and 3.13, and that the change keeps extraction deterministic.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify code review View in Graphify

Reviewed · 852a6aa8dcd7

Only static checks ran on this change. They found no blocking issues.

Code graph and static checks · 1025 functions depend on this change

Impact & health

Impact — 1025 functions depend on the 102 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: deduplicate_entities() — 128 callers, 34 callees

Formal checks — 1025 functions this change can affect weren't formally checked in this review. Formal checks are informational only.

Gate & verification

✅ PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 471 function(s) in the blast radius were not formally verified this run

Test selection

41 of 594 test file(s) selected (7%) via static blast radius.

  • tests/test_build.py — impact
  • tests/test_build_located_semantic_identity.py — impact
  • tests/test_build_merge_dedup_scope.py — impact
  • tests/test_build_merge_ghost_fold_4955.py — impact
  • tests/test_build_merge_hyperedges_and_prune.py — impact
  • tests/test_build_merge_shrink_guard.py — impact
  • tests/test_c_include_resolution.py — impact
  • tests/test_carried_hyperedge_remap.py — impact
  • tests/test_chunking.py — impact
  • tests/test_corrupt_graph_json.py — impact
  • tests/test_cpp_forward_declarations.py — impact
  • tests/test_cross_extension_reexport_self_cycle.py — impact
  • tests/test_dedup.py — impact
  • tests/test_dedup_prefix_derived_words.py — impact
  • tests/test_dedup_reference_ids.py — impact, changed-test
  • tests/test_dedup_remaps_hyperedges.py — impact
  • tests/test_dedup_shrink_refuses_force_write.py — impact
  • tests/test_dedup_survivor_richness.py — impact
  • tests/test_ext_tagged_id_clash.py — impact
  • tests/test_failed_ast_replacement.py — impact
  • tests/test_file_id_collision_4854.py — impact
  • tests/test_global_graph.py — impact
  • tests/test_go_qualified_resolution.py — impact
  • tests/test_issue_3472_source_file_collision.py — impact
  • tests/test_js_member_collisions.py — impact
  • tests/test_linked_worktree_reconcile.py — impact
  • tests/test_merge_sweeps_reextract_orphans.py — impact
  • tests/test_no_dedup_flag.py — impact
  • tests/test_non_string_node_ids.py — impact
  • tests/test_prune_source_aliases.py — impact
  • tests/test_prune_sweeps_orphans.py — impact
  • tests/test_python_underscore_resolution.py — impact
  • tests/test_r_extractor.py — impact
  • tests/test_semantic_origin_stamp.py — impact
  • tests/test_shrink_guard_dedup_merge_4940.py — impact
  • tests/test_shrink_guard_replaced_sources.py — impact
  • tests/test_swift_import_resolution.py — impact
  • tests/test_symlinked_root_source_file.py — impact
  • tests/test_unverified_semantic_shrink.py — impact
  • tests/test_validate.py — impact
  • tests/test_watch.py — impact

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Every flagged function

  • deduplicate_entities() in graphify/dedup.py:1150 · 128 callers, 34 callees: fans out to 34 callees (efferent coupling); 128 callers depend on it (afferent coupling)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor Pull request from a returning community contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Path-prefixed IDs split one shared concept into N entities

1 participant