Skip to content

chore: catch doubled words in pre-commit - #404

Merged
thodson-usgs merged 2 commits into
DOI-USGS:mainfrom
thodson-usgs:chore/doubled-word-check
Sep 1, 2026
Merged

chore: catch doubled words in pre-commit#404
thodson-usgs merged 2 commits into
DOI-USGS:mainfrom
thodson-usgs:chore/doubled-word-check

Conversation

@thodson-usgs

Copy link
Copy Markdown
Collaborator

Why

A mechanical rename on another branch this week left "a label label" in a
docstring — \bsource\b matched inside the phrase "a source label" and
rewrote only the first word.

Everything green-lit it. Tests passed, mypy --strict passed, ruff passed, and
an AST comparison proving the change was behaviour-neutral passed — because
that check strips docstrings before comparing
. It is silent by construction
about exactly the text a rename most easily breaks. Only reading the diff by eye
found it.

A doubled word is the mechanical signature of that mistake, and it costs
nothing to check.

Measured before adding

A gate that mostly cries wolf is worse than none, so:

result
tracked text files scanned 168
false positives 0
real defects found on first run 1

Restricting the word to [A-Za-z]+ is what buys the zero. \w+ also matches
RDB column types such as 10n 10n in the fixture captures.

What its first run found

A typo in a user-facing demo notebook, shipped and read past:

"Filters on the the associated monitoring location's point location"

The only other hit was a genuinely clumsy test docstring — "Which variable that
is is platform-specific"
— which is correct English but poor prose. Reworded
rather than excluded, so the hook needs no exclusion list beyond
tests/data/, which the existing prose hooks already skip because it holds
byte-exact API captures.

Line-scoped on purpose

--multiline was tried and rejected. It catches a doubled word split across a
wrapped line, but this repo's prose and code produce that shape legitimately all
the time — a Markdown heading followed by its own first word, return df above
df, import sys above sys:

prose (.md/.rst):  4 cross-line reports, 0 real
code  (.py):      15 cross-line reports, 0 real

19 reports, none of them defects. Line-scoped catches the shape a mechanical
rename actually produces, which is the one that got past every other gate.

It can fail

A gate never seen to fail is not evidence of anything. Seeding a doubled word
makes it exit 1 and name the location:

doubled word.............................................................Failed
- hook id: doubled-word
- exit code: 1
dataretrieval/nwis.py:1267:# seeded the the slip

The hook was run against that seed before being kept.

Scope

.pre-commit-config.yaml, one notebook typo, one reworded docstring.
Independent of #400, #402 and #403. 1139 tests pass; every hook green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD

A mechanical rename left "a label label" in a docstring on a branch this week.
Tests, `mypy --strict`, ruff and an AST comparison all passed over it: the AST
check strips docstrings before comparing, so it is silent by construction about
the text a rename most easily breaks. Only reading the diff found it.

A doubled word is the mechanical signature of that mistake -- a regex that
rewrites one word of a phrase leaves its neighbour standing -- and it is
cheap to check.

Measured before adding, since a gate that mostly cries wolf is worse than none:
across 168 tracked text files the pattern reports **zero** false positives.
Restricting the word to `[A-Za-z]+` is what buys that; `\w+` also matches RDB
column types like `10n 10n`.

Its first real run found a typo in a user-facing demo notebook -- "Filters on
the the associated monitoring location" -- which had been shipped and read past.
The one other hit was a genuinely clumsy sentence in a test docstring ("Which
variable that is is platform-specific"), reworded rather than excluded, so the
hook needs no exclusion list beyond `tests/data/`, which the other prose hooks
already skip because it holds byte-exact API captures.

Line-scoped on purpose. Matching across newlines too was tried and rejected: it
finds nothing real and reports a Markdown heading followed by its own first
word, or `return df` above `df` -- 19 such reports against 0 real ones.

Confirmed it can fail. Seeding a doubled word makes the hook exit 1 and name
the file and line; the check was run against that seed before being kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD
Comment thread .pre-commit-config.yaml Outdated
@thodson-usgs
thodson-usgs merged commit 84a2279 into DOI-USGS:main Sep 1, 2026
11 checks passed
@thodson-usgs
thodson-usgs deleted the chore/doubled-word-check branch September 2, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant