From 976cd33318ff61f5932a499141dfdf3e7c74cc6d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 27 Aug 2026 17:49:54 -0700 Subject: [PATCH 1/3] fix(post_rules,rules,differential): a maiden marker makes the lone name word the family MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A maiden name announces a FORMER surname, which only means anything beside a current one -- so where the clause has taken its name and one name word is left, that word is the family name. `parse("Smith née Jones")` reported given 'Smith' with an empty family; it now reports family 'Smith', maiden 'Jones'. Keyed on the maiden NAME rather than on a marker, which is what the code does and what M1's caller-configured pair needs: `Smith (Jones)` under `maiden_delimiters` carries no marker anywhere and reads family 'Smith' all the same. M2's take, M3's marker-led clause and M1's declared pair announce the same thing, so the rule covers all three. Written as H1's sibling in post_rules and placed under it, so the interaction is decidable by reading: where H1 fired there is no GIVEN left, and where it declined on a given-name title the `titles` guard keeps this rule off the same word ("Sir John née Jones" is unchanged). Sibling except in what it counts -- this guard counts GIVEN tokens where H1 counts nothing -- so a joined name word declines here where H1 fires. That gap is recorded, not closed: widening moves no corpus name and is a behaviour change of its own. Two carve-outs, both read off the tags classify already recorded rather than a predicate of this rule's own, because the rule changes what POSITION decided and must not reach what a word IS (mechanisms.md#TWO-LAYER-ASSIGN): `vocab:bound-given` ("abd née Jones") and `initial` ("J. née Jones Smith V"), each a corpus name and each caught alone by its row under mutation. Two rules recorded, not one. O4 is silent at exactly one name word -- first and last are the same word -- so the shipped reading had no rule. O5 records it as the convention it is: a guess fixed in advance so the same input reads the same way, not a determination, with H1, N3 and M4 named as the rules that decide instead and each one's carve-out named with it. M4 is the exception this commit implements. The code comments cite both verbatim, which is why the rules and the code land together. The differential lands here too, so the gates are green at this commit rather than at the next one. Nine corpus names change reading; eight arrive UNEXPLAINED at all three baselines and are classified with field sets that are exactly what each name moves, splitting where the diffs do: - 1.4.0: seven move {given, middle, family, maiden}; 'John née Jones Smith Ma' moves those plus `suffix`. - 2.0.0/2.1.0: six move {given, family} alone, the marker consumption being the baseline's own reading already. - Two carry compound diffs and get rules naming both causes. The #424 rules are renamed to fix(#424/#445) rather than widened, since one rule explains a name's whole diff; #412's rule keeps its other name at its own three fields, so nothing pre-excuses a `given` loss there. The ninth name, 'Smith (née Jones)', is the one the gates do NOT report: its family agrees with 1.4.0 again, so that ledger's rule is NARROWED to {nickname, maiden} -- the shrink its own revisit note asked to be re-measured for, in the opposite direction from the one it feared. The four `# revisit when #445 lands` markers are corrected rather than deleted: each predicted the fix would leave the diff a subset of the fields its rule already declared and be absorbed in silence. This rule MOVES the word rather than adding one, so `given` empties as `family` fills and the gate goes loud. The notes named the right rule, the right field and very nearly the right set, and saying so is worth more than a clean deletion. Also here: the corpus-wide maiden-clause property in test_parser.py, where the one-name-word class now asserts the flip instead of no movement; the clause-free corpus counts, recounted by pasting the one-liner (the first two were already one low); and two stale `_CORPUS_FLOORS` comments, 175 and 97 where the files hold 234 and 98. Co-Authored-By: Claude Opus 5 --- docs/design/rules.md | 93 ++++++++++++-- nameparser/_pipeline/_assign.py | 8 ++ nameparser/_pipeline/_post_rules.py | 49 +++++++- tests/v2/cases.py | 125 +++++++++++++++---- tests/v2/test_ledger_guards.py | 62 ++++++--- tests/v2/test_parser.py | 49 +++++++- tools/differential/compare.py | 4 +- tools/differential/corpus_rules.jsonl | 5 + tools/differential/expected_since_1.4.0.toml | 123 +++++++++++++----- tools/differential/expected_since_2.0.0.toml | 88 +++++++++++-- tools/differential/expected_since_2.1.0.toml | 89 +++++++++++-- 11 files changed, 589 insertions(+), 106 deletions(-) diff --git a/docs/design/rules.md b/docs/design/rules.md index f2ee4df6..e8fc8a31 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -531,18 +531,19 @@ N3. Rationale: a person set down as a nickname plus one name word is Accepted: a marker-led clause is a maiden clause and not a nickname one (M3), so this rule does not reach a name written that way, and the one name word keeps the reading the bare - spelling gives it rather than becoming the family name. The - empty family that leaves is the bare spelling's, and #445 tracks - whether it is right — 1.4.0 read the bare spelling as first - Smith, middle née, last Jones, emptying nothing, and the same - empty family has been repaired twice already, at H1 (#410) and at - P5's reserve (#411). - "Smith (née Jones)" → given="Smith" - history: decisions.md#N3 · interacts: H1, M3 · implemented: nameparser/_pipeline/_assign.py + spelling gives it. Since #445 that reading is the family name + all the same, M4 reaching both spellings from the other side. + The two rules agree on THIS name and not in general: M4 counts + name words alone, so where a suffix stands beside them it fires + and this rule declines. The Accepted line above reads a bare + nickname-plus-word-plus-suffix name as given with no family, and + the same name carrying a maiden clause reads family instead. + "Smith (née Jones)" → family="Smith" + history: decisions.md#N3 · interacts: H1, M3, M4 · implemented: nameparser/_pipeline/_assign.py ## Maiden names (M) -Background: a maiden name is written beside the current name, set off by a marker or by enclosure. Markers are attested across French née/né and the unaccented nee English writing uses for both, German geb./geborene, Dutch geboren, Czech/Slovak rozená (the abbreviation roz. shipped through 2.1 and was removed in 2.2 -- it collides with the English diminutive Roz, and a caller who needs it adds it to their own Lexicon), Scandinavian født/fødd/född, Russian урожд. and its full participles урождённая/урождённый (the participles in both the ё and е spellings, which case normalization does not fold), Japanese 旧姓, and Polish z domu — both grammatical genders where attested. Every one of these is a marker wherever it stands, the unaccented nee included: an enclosure holding a word past it reads as the maiden name and the marker is dropped, so a configured pair reading "(Nee Jones)" gives maiden Jones and not Nee Jones. A marker need not be one word. z domu is two, and a phrase marker is recognized only whole and only where its words stand together: neither of its words is a marker standing alone, and neither is the pair once a bracketed clause or a comma divides them. That is what makes shipping it safe where shipping its words would not be — z is an ordinary Polish preposition, and a name that merely contains one keeps its family name. Japanese more often writes the marker with a fullwidth colon (旧姓：佐藤), which is no separator, so marker and name arrive as a single word. Which enclosures mean "maiden" rather than "nickname" is a caller convention, so the maiden reading of a delimiter pair is opt-in — except where the clause announces itself. A clause of two words or more led by a marker word has said which convention it means, and reads as the maiden name inside a nickname pair as well (M3) — unless its content is suffix-shaped, which S1 takes ahead of both. A lone marker has said nothing, and neither has one the colon spelling above glues to the name. +Background: a maiden name is written beside the current name, set off by a marker or by enclosure. Markers are attested across French née/né and the unaccented nee English writing uses for both, German geb./geborene, Dutch geboren, Czech/Slovak rozená (the abbreviation roz. shipped through 2.1 and was removed in 2.2 -- it collides with the English diminutive Roz, and a caller who needs it adds it to their own Lexicon), Scandinavian født/fødd/född, Russian урожд. and its full participles урождённая/урождённый (the participles in both the ё and е spellings, which case normalization does not fold), Japanese 旧姓, and Polish z domu — both grammatical genders where attested. Every one of these is a marker wherever it stands, the unaccented nee included: an enclosure holding a word past it reads as the maiden name and the marker is dropped, so a configured pair reading "(Nee Jones)" gives maiden Jones and not Nee Jones. A marker need not be one word. z domu is two, and a phrase marker is recognized only whole and only where its words stand together: neither of its words is a marker standing alone, and neither is the pair once a bracketed clause or a comma divides them. That is what makes shipping it safe where shipping its words would not be — z is an ordinary Polish preposition, and a name that merely contains one keeps its family name. Japanese more often writes the marker with a fullwidth colon (旧姓：佐藤), which is no separator, so marker and name arrive as a single word. Which enclosures mean "maiden" rather than "nickname" is a caller convention, so the maiden reading of a delimiter pair is opt-in — except where the clause announces itself. A clause of two words or more led by a marker word has said which convention it means, and reads as the maiden name inside a nickname pair as well (M3) — unless its content is suffix-shaped, which S1 takes ahead of both. A lone marker has said nothing, and neither has one the colon spelling above glues to the name. A marker also says something about the name standing beside it: it announces a surname the bearer no longer uses, and that is only worth writing where there is a current one to tell it apart from (M4). M1. Rationale: an enclosure the caller has declared to mean maiden holds the former family name; a recognized marker word inside it @@ -567,7 +568,7 @@ M1. Rationale: an enclosure the caller has declared to mean maiden "Jane Smith (Nee) (Jones)" maiden-parens → maiden="Nee Jones" "Andrew Perkins (MBA)" maiden-parens → suffix="MBA" · boundary "Maria Kowalska (z domu)" maiden-parens → maiden="z domu" · boundary - history: decisions.md#M1 · interacts: S1, M2, M3 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_group.py + history: decisions.md#M1 · interacts: S1, M2, M3, M4 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_group.py M2. Rationale: a maiden marker announces that what follows it is the former family name; the marker is an announcement, not a name. @@ -662,6 +663,44 @@ M3. Rationale: an enclosure says nothing about whether it means "Jane Smith (née V)" → maiden="V" history: decisions.md#M3 · interacts: M1, M2, S1, N1 · implemented: nameparser/_pipeline/_extract.py +M4. Rationale: a maiden name is a FORMER family name, and a former + one only means something beside a current one — nobody announces + the surname they used to carry where there is no surname beside + it to tell it apart from. So where a maiden name has been read + out of a name and a single name word is left standing, that word + is the surname the bearer uses now; writing the clause at all is + what rules out the given-name reading O5's convention would + otherwise leave it with. How the maiden name was written does + not enter into it — a marker taking the words after it (M2), a + marker-led clause (M3), and a clause in a pair the caller + declared to mean maiden (M1) announce the same thing, and the + last of those carries no marker at all. + A maiden name standing beside exactly one name word makes that + word the family name, whatever suffix or nickname stands beside + it — an annotation beside the name is no part of the name, which + is how H1 reads the same shape. Two kinds of word are beyond its + reach, for one reason: the rule changes what POSITION would have + decided, so it cannot overrule what a word already IS + (mechanisms.md#TWO-LAYER-ASSIGN). A word the vocabulary has + claimed as a given name keeps that reading, and so does a word + written as an initial, which is nobody's family name. A name + carrying a TITLE is H1's rather than this rule's, H1's + given-name-title carve-out included, which keeps the word a + given name. A nickname holds nothing off: where N3 has already + named the family this rule finds nothing left to move, and where + N3 declined — its count does not set a suffix aside — this rule + names it. Where the maiden name stood does not matter, only what + is left beside it: a marker inside the name takes the rest of it + (M2), and one name word left that way is one name word. + "Smith née Jones" → family="Smith" + "Smith (née Jones)" → family="Smith" + "Smith (Jones)" maiden-parens → family="Smith" + "Jane née Jones Smith" → family="Jane" + "Smith née Jones PhD" → family="Smith" + "abd née Jones" → given="abd" · boundary + "J. née Jones Smith V" → given="J." · boundary + interacts: M1, M2, M3, H1, N3, O5 · implemented: nameparser/_pipeline/_post_rules.py + ## Commas & structure (C) Background: a comma in a name signals one of two conventions — the listing form "Family, Given" or trailing credentials "Name, PhD" — and which is meant can only be judged from what stands after the first comma. Recognizing a credential run is by nature a vocabulary judgment, so this is the one structural decision that consults the suffix word lists. Which characters COUNT as the comma is part of the rule: the Arabic comma (U+060C) and the fullwidth comma (U+FF0C) both signal the listing form, while the ideographic comma (U+3001) is not a name-structure comma at all (#265). @@ -810,6 +849,40 @@ O4. Rationale: what no vocabulary claims can only be read by where exceptions from; its boundaries are the other rules. implemented: nameparser/_pipeline/_assign.py +O5. Rationale: O4 reads a name by comparing where its words stand, + and a name of exactly one name word gives it nothing to compare + — the first name word is also the last, so the rule that decides + every longer name is silent here. Nothing in such an input says + whether the word is a given name or a family name: both readings + fit it equally well, and the library still has to report one. + A name of one name word that nothing else has decided reads that + word as the given name under the default given-first order, and + as the family name under a declared family-first one. That is a + convention rather than a determination — the same input has to + read the same way every time, so one of two equally consistent + readings is fixed in advance — and it is not evidence about the + word. Every rule that DOES decide such a name outranks it, and + each of them carves out a case it declines to decide — where the + convention is what is left. A title makes the word the family + name (H1), except a given-name title, which addresses by given + name; a nickname beside it does the same (N3), except where a + suffix stands beside them too, which its count does not set + aside; and a maiden name beside it does the same (M4), except + where the vocabulary or the word's own shape has claimed the + word already. What the library should SAY about a reading it + merely fixed — whether the convention is worth reporting as an + ambiguity — is open, and #449 holds the measurement that makes + it a design question rather than an implementation one. + "Smith" → given="Smith" + "Garcia" family-first → family="Garcia" + "Sir John" → given="John" + "'Smitty' Jones Jr." → given="Jones" + "abd née Jones" → given="abd" + "Mr. Johnson" → family="Johnson" · boundary + "'Smitty' Jones" → family="Jones" · boundary + "Smith née Jones" → family="Smith" · boundary + interacts: O4, H1, N3, M4 · implemented: nameparser/_pipeline/_assign.py + ## Scripts & writing systems (W) Background: script-conditional behavior is permitted exactly where the writing system itself — not statistics about it — settles the convention; a language can never be inferred from Latin-script text, because transliteration destroys the signal. The facts this section builds on: Chinese and Japanese both write the family name first in native script, so the script settles the order without knowing the language. Hangul is written by exactly one language and Korean family names are a small closed census set. Han text does not identify its language — a Chinese surname list would divide Japanese 高橋一郎 as 高 + 橋一郎 — which is why Han division is opt-in and there is no Korean pack to opt into. Hiragana never transcribes a foreign name (transcriptions are katakana alone), so kanji-plus-kana is a Japanese name in Japanese order, while wholly-katakana is predominantly a transcribed foreign name already in given-first order. Real Chinese text is unspaced (毛泽东); the spaced 毛 泽东 is an artifact. A fuller narrative lives in docs/usage.rst's East Asian section. diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 96f5a8ff..167c2b0c 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -146,6 +146,14 @@ def _name_positions(order: tuple[Role, Role, Role], family, middles..., given. One piece takes order[0]'s role; two pieces take order[0] and the other primary.""" first, second = order[0], order[1] + # rules.md#O5: "a name of one name word that nothing else has + # decided reads that word as the given name under the default + # given-first order, and as the family name under a declared + # family-first one" -- a convention, not a determination: O4 has + # no positions to compare at one word, so this line is where the + # library picks one of two equally consistent readings and picks + # it the same way every time. The rules that DO decide such a + # name (H1, N3, M4) run after this and retag. if count == 1: return [first] if first is Role.GIVEN: # GIVEN_FIRST diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 6c1436be..c0c18d79 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -8,8 +8,8 @@ Reads: Policy.patronymic_rules, Policy.middle_as_family; Lexicon.given_name_titles. -Implements rules H1, P1, O1, O2 and O3 of docs/design/rules.md; each -is cited at its code below, and H1/P1/O1/O2's history lives in +Implements rules H1, M4, P1, O1, O2 and O3 of docs/design/rules.md; +each is cited at its code below, and H1/P1/O1/O2's history lives in docs/design/decisions.md. """ from __future__ import annotations @@ -40,6 +40,11 @@ _NAME_ROLES = (Role.GIVEN, Role.MIDDLE, Role.FAMILY) +#: M4's two carve-outs, as the tags classify recorded them: a bound +#: given-name word is vocabulary claiming the word as a given name, +#: and `initial` is the shape claim. Neither is a predicate M4 owns. +_NEVER_FLIPPED = frozenset({"vocab:bound-given", "initial"}) + def _idx(tokens: list[WorkToken], role: Role) -> list[int]: return [i for i, t in enumerate(tokens) if t.role is role] @@ -194,6 +199,46 @@ def post_rules(state: ParseState) -> ParseState: middles = _idx(tokens, Role.MIDDLE) families = _idx(tokens, Role.FAMILY) + # rules.md#M4: "a maiden name standing beside exactly one name + # word makes that word the family name, whatever suffix or + # nickname stands beside it" (#445) -- the MAIDEN role, not a + # marker: M1's configured pair produces one with no marker + # anywhere ("Smith (Jones)" under maiden_delimiters), and the + # rationale carries over unchanged, so the rule is keyed on the + # maiden name as the statement is. + # + # H1's sibling, and placed under it so the interaction the rule + # states is decidable by reading: where H1 fired, `givens` is + # empty here and this cannot fire; where H1 declined because the + # title addresses by given name, `titles` is what keeps this rule + # off the same word. A titled name is H1's at both outcomes. + # + # Sibling EXCEPT in what it counts, and that is a known gap + # rather than a boundary: this guard counts GIVEN tokens where + # H1 counts nothing (it tests which roles are unoccupied), so a + # name word another rule joined counts as several here and the + # rule declines -- `Dr. Dean of Chemistry` reads family, while + # `Dean of Chemistry née Jones` keeps given 'Dean of Chemistry'. + # rules.md#P3 says a joined part is one name word wherever + # another rule counts them, so the two disagree; widening moves + # no corpus name and is not this change's to make + # (decisions.md#M4). + # rules.md#M4: "a word the vocabulary has claimed as a given name + # keeps that reading, and so does a word written as an initial" + # -- read off the tags classify already recorded rather than a + # predicate of this rule's own, because this rule changes what + # POSITION decided and must not reach what a word IS + # (mechanisms.md#TWO-LAYER-ASSIGN). + if (not titles and len(givens) == 1 and not middles and not families + and any(t.role is Role.MAIDEN for t in tokens) + and not (_NEVER_FLIPPED & tokens[givens[0]].tags)): + _retag(tokens, givens[0], Role.FAMILY) + # recomputed for H1's reason, stated at H1: a stale index list + # is the bug shape #359 fixed + givens = _idx(tokens, Role.GIVEN) + middles = _idx(tokens, Role.MIDDLE) + families = _idx(tokens, Role.FAMILY) + # rules.md#P1: "a never-given particle standing alone where the # given name would go — or opening the name — marks the name as # surname-only: the particle run and the name words it attaches diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 7c2334f1..86af6544 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -1197,10 +1197,18 @@ def __post_init__(self) -> None: {"given": "abd", "maiden": "Jones"}, classification="fix(#411)", notes="the shortest form of the same decision: the very next " - "piece is the marker. Empty family is M2's ordinary " - "one-name-word behaviour, not a leftover of the join " - "-- 'Jane née Jones' reads the same way and always " - "has"), + "piece is the marker. It is also M4's boundary on the " + "vocabulary side, and a corpus name rather than a " + "constructed one: 'abd' is a bound given-name word, so " + "the vocabulary layer has claimed it as a GIVEN name " + "and M4 -- which changes only the positional default " + "-- does not reach it (mechanisms.md#TWO-LAYER-ASSIGN). " + "Dropping that carve-out reads family 'abd'. The empty " + "family is what M4 leaves standing here, not a " + "leftover of the join, and not M2's ordinary " + "one-name-word behaviour either -- since #445 'Smith " + "née Jones' reads family 'Smith', and this row is why " + "'abd née Jones' does not"), Case("bound_given_reserve_arabic_script", "عبد Berg née Jones", {"given": "عبد", "family": "Berg", "maiden": "Jones"}, @@ -1229,11 +1237,15 @@ def __post_init__(self) -> None: "connective_join_never_reaches_a_taken_marker"), Case("maiden_marker_ahead_of_a_conjunction", "Jane née and Jones Smith", - {"given": "Jane", "maiden": "and Jones Smith"}, - classification="fix(#412)", + {"family": "Jane", "maiden": "and Jones Smith"}, + classification="fix(#445)", notes="M2's greedy reading, with a connective among the " "words taken: the same as 'Jane née Jones Smith' with " - "'and' inside it. Until #412 closed, P3's join ran " + "'and' inside it -- which is why M4 reads it the same " + "way too, the take leaving one name word either way " + "(the family was empty here until #445; 1.4.0 read " + "first 'Jane' / middle 'née and Jones' / last 'Smith', " + "measured 2026-08-27). Until #412 closed, P3's join ran " "first and produced a marker-HEADED piece 'née and " "Jones' that the lone-piece test could not see, so the " "name read middle 'née and Jones', family 'Smith'. " @@ -1519,30 +1531,91 @@ def __post_init__(self) -> None: "Parity: 1.4.0 and 2.1.0 both read nickname 'Jones née'"), Case("marker_led_clause_with_one_name_word", "Smith (née Jones)", - {"given": "Smith", "maiden": "Jones"}, - classification="fix(#335)", + {"family": "Smith", "maiden": "Jones"}, + classification="fix(#445)", notes="N3's shape meeting M3, and the row exists because the " "two rules disagree about what a clause is. N3 reads a " "name that is only a nickname plus one name word as " "'that word is the family name' -- but a marker-led " "clause is not a nickname clause, so N3 never sees this " - "one and 'Smith' keeps the given-name reading the bare " - "spelling gives it. Recorded as an N3 Accepted line. " - "The empty family is the bare path's, not this " - "change's: 'Smith née Jones' reads given 'Smith', " - "family '' on 2.1.0 too, and 1.4.0 read first 'Smith' / " - "middle 'née' / last 'Jones' -- the marker as a " - "middle name. The parallel evidence at 1.4.0 is not " - "the fix(#410) maiden-flavor rule, which is keyed on " - "the TITLED 'Dr. Smith née Jones', but fix(#274), " - "which claims the bare one-word names 'Janey née " - "Jones' and 'abd née Jones' and declares `family` for " - "exactly the reason this row records. The given side " - "has agreed since 1.4.0, " - "the emptying is 2.x's and is tracked separately as the " - "#410 analogue. What #335 moves is only the bracketed " - "spelling: 1.4.0, 2.0.0 and 2.1.0 all read it family " - "'Smith', nickname 'née Jones' (measured 2026-08-26)"), + "one, and until #445 the word kept the given-name " + "reading the bare spelling gives it. M4 now reaches " + "both spellings from the other side: the marker " + "announces a FORMER surname, so the one name word left " + "beside it is the current one. The bracketed spelling " + "RESTORES 1.4.0's family, which read family 'Smith', " + "nickname 'née Jones' (measured 2026-08-26) -- the " + "clause reading maiden rather than nickname is #335's " + "half and the only thing left that differs. Was " + "given 'Smith' on 1.4.0's bare spelling (first " + "'Smith' / middle 'née' / last 'Jones'), on 2.0.0, on " + "2.1.0 and here until this rule"), + Case("maiden_marker_makes_the_lone_name_word_the_family", + "Smith née Jones", + {"family": "Smith", "maiden": "Jones"}, + classification="fix(#445)", + notes="the rule, in its bare spelling: a maiden marker marks " + "a surname the bearer no longer uses, so it only means " + "anything beside one they do. With exactly one name " + "word left after the take, that word is the current " + "surname, and the positional convention O5 would " + "otherwise apply (a lone name word is read given) is " + "the thing M4 overrides. Read given 'Smith', family '' " + "on 2.0.0 and 2.1.0; 1.4.0 had no maiden support and " + "read first 'Smith' / middle 'née' / last 'Jones' " + "(measured 2026-08-27), so this is a new reading and " + "not a restoration -- the bracketed sibling " + "marker_led_clause_with_one_name_word is the " + "restoration"), + Case("maiden_marker_interior_makes_the_lone_name_word_the_family", + "Jane née Jones Smith", + {"family": "Jane", "maiden": "Jones Smith"}, + classification="fix(#445)", + notes="the marker standing INSIDE the name, where M2's take " + "runs to the end and swallows the rest -- so what is " + "left is again one name word, and M4 counts what is " + "left rather than where the marker stood. The widest " + "half of the rule and the row that pins it: a guard " + "that asked for the marker to be trailing would leave " + "this one given 'Jane'. A new reading, and the one " + "furthest from 1.4.0, which read first 'Jane' / middle " + "'née Jones' / last 'Smith' (measured 2026-08-27) -- " + "the real surname there is 'Smith', which 2.x reads as " + "part of the maiden name (M2's greedy take, unchanged " + "here)"), + Case("maiden_marker_lone_name_word_with_suffix", + "Smith née Jones PhD", + {"family": "Smith", "suffix": "PhD", "maiden": "Jones"}, + classification="fix(#445)", + notes="an annotation is not a name word, which is what #410 " + "established for H1 and M4 inherits: the credential " + "stands beside the name and does not make it any " + "longer, so the count that decides this reading is " + "one either way. A guard written over roles generally " + "rather than the three name roles reads this name as " + "two words and declines. 1.4.0 read first 'Smith' / " + "middle 'née' / last 'Jones' / suffix 'PhD' (measured " + "2026-08-27)"), + Case("maiden_marked_lone_initial_stays_given", + "J. née Jones Smith V", + {"given": "J.", "maiden": "Jones Smith V"}, + classification="fix(#274)", + notes="M4's boundary on the shape side, and a corpus name " + "rather than a constructed one: an initial is not a " + "family name, so the word the vocabulary layer has " + "already claimed as a shape keeps its reading " + "(mechanisms.md#TWO-LAYER-ASSIGN -- M4 changes the " + "POSITIONAL default and must not reach a word another " + "layer has claimed). Dropping the carve-out reads " + "family 'J.'. Unchanged by #445, and the maiden value " + "is M2's: the trailing 'V' is S2's suffix reading only " + "where a name word precedes it, and an initial does " + "not count, so the numeral stays maiden text (M2 " + "carries the same input as a boundary example). The " + "fix classification is #274's marker consumption, " + "which is what makes this differ from 1.4.0 (first " + "'J.' / middle 'née Jones' / last 'Smith' / suffix " + "'V', measured 2026-08-27)"), Case("marker_led_clause_in_a_quote_pair", 'Jane Smith "née Jones"', {"given": "Jane", "family": "Smith", "maiden": "Jones"}, diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 5cfe5cf3..d2cb5d85 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -940,6 +940,26 @@ class _LatinCopy(NamedTuple): # begins, and the alternation is over ANCHORS, not over words -- # there is no vocabulary here to drift from. frozenset({"^", ",\\s*"}), + # fix(#445)'s movers, one corpus name per alternative -- a list of + # names, not a copy of any wordlist, so there is no vocabulary for + # it to drift from. Two sets because the two field shapes split + # differently: the 1.4 ledger's seven names all move four fields, + # while at 2.0.0/2.1.0 six of them move two and the other two are + # compounds with rules of their own. + # + # Spelled out rather than written as the shape -- one word, then a + # marker -- because that shape also matches rules.md#M4's two + # carve-outs, 'abd née Jones' and 'J. née Jones Smith V'. A rule + # covering those two while declaring `given` would stand ready to + # explain the very given -> family move they exist to prevent, + # which is the absorption these rosters exist to catch. + frozenset({"Janey n[ée]e Jones", "Jane n[ée]e Jones J\\. V", + "Jane n[ée]e Jones Smith", "Jane n[ée]e and Jones Smith", + "John n[ée]e Jones Smith V", "Smith n[ée]e Jones", + "Smith n[ée]e Jones PhD"}), + frozenset({"Janey n[ée]e Jones", "Jane n[ée]e Jones J\\. V", + "Jane n[ée]e Jones Smith", "John n[ée]e Jones Smith Ma", + "Smith n[ée]e Jones", "Smith n[ée]e Jones PhD"}), }) def _unjustified_reach(name_regex: str, members: set[str]) -> list[str]: @@ -1320,7 +1340,7 @@ def _claim(rule: dict) -> _Claim: _CORPUS_CLAIMS: dict[str, dict[str, _Claim]] = { "expected_since_1.4.0.toml": { "fix(#335) a marker-led clause leaves the one name word its bare reading": - _Claim(1, ('family', 'given', 'maiden', 'nickname'), "c09cc7dba88b"), + _Claim(1, ('maiden', 'nickname'), "c09cc7dba88b"), "fix(#434) a multi-word maiden marker takes the maiden name": _Claim(1, ('family', 'maiden', 'middle'), "c428798fc6ef"), "fix(#434) a multi-word marker leads a bracketed clause to the maiden name": @@ -1336,7 +1356,7 @@ def _claim(rule: dict) -> _Claim: "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots": _Claim(108, ('family', 'given', 'middle'), "9a814f70c2dc"), "fix(#274) maiden markers consumed": - _Claim(29, ('family', 'maiden', 'middle'), "c0981c1c6557"), + _Claim(31, ('family', 'maiden', 'middle'), "67e2280be79d"), "fix(cjk-maiden-marker) maiden marker consumed, compounding with the CJK order flip": _Claim(5, ('family', 'given', 'maiden', 'middle'), "bc0e10dd7ec8"), "fix(#379) a tussenvoegsel after a family comma attaches to the family": @@ -1364,7 +1384,7 @@ def _claim(rule: dict) -> _Claim: "fix(comma-precomma-family) pre-comma run reads as family, not given": _Claim(279, ('family', 'given'), "28a62b622a48"), "fix(suffix-routing) two-token name with unambiguous trailing suffix stays suffix": - _Claim(1085, ('family', 'given', 'suffix'), "0df8e4a51a54"), + _Claim(1090, ('family', 'given', 'suffix'), "89e0b6d7f4c8"), "fix(suffix-delimiter-rendering) no-space delimiter core token kept whole": _Claim(0, ('suffix',), "e3b0c44298fc"), "ambiguous-surname-acronym data change: parenthesized (MA)/(DO) now stays nickname": @@ -1421,8 +1441,8 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('given', 'middle'), "2010cc79a34d"), "fix(#424) the particle chain stops before the trailing numeral": _Claim(1, ('family', 'suffix'), "2c99162bc9cf"), - "fix(#424) accepted: the maiden walk keeps a bare acronym": - _Claim(1, ('family', 'maiden', 'middle', 'suffix'), "f2c6cd2e3001"), + "fix(#424/#445) accepted: the maiden walk keeps a bare acronym, and the lone name word is the family": + _Claim(1, ('family', 'given', 'maiden', 'middle', 'suffix'), "f2c6cd2e3001"), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle, the P4 example": _Claim(1, ('family', 'given'), "42b69cf1b320"), "fix(#424) accepted: the maiden walk keeps the numeral an initial before the marker vetoes": @@ -1435,6 +1455,8 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'middle', 'suffix'), "a564b97f7162"), "fix(#360) ste moved into the never-given particles with mc": _Claim(1, ('family', 'given'), "e62caedec864"), + "fix(#445) a maiden marker makes the lone name word the family": + _Claim(7, ('family', 'given', 'maiden', 'middle'), "3de9ef12b4a8"), }, "expected_since_2.0.0.toml": { "fix(#335) a marker-led clause leaves the one name word its bare reading": @@ -1539,8 +1561,12 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('_ambiguities', 'family', 'middle', 'suffix'), "a564b97f7162"), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle, the P4 example": _Claim(1, ('_ambiguities', 'family', 'given'), "42b69cf1b320"), - "fix(#424) the maiden walk stops before the trailing numeral": - _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "cbe5bdd97317"), + "fix(#424/#445) the maiden walk stops before the trailing numeral, and the lone name word is the family": + _Claim(1, ('_ambiguities', 'family', 'given', 'maiden', 'suffix'), "cbe5bdd97317"), + "fix(#445) a maiden marker makes the lone name word the family": + _Claim(6, ('family', 'given'), "f521c94c79fc"), + "fix(#445) the lone name word beside a marker a connective join no longer absorbs": + _Claim(1, ('family', 'given', 'maiden', 'middle'), "52544a41dd62"), "fix(#424) a marker followed only by the numeral is just a word": _Claim(1, ('_ambiguities', 'family', 'maiden', 'middle', 'suffix'), "aaf53040b071"), "fix(#369) the bound given-name join takes a particle-and-bound word, so no fork is reported": @@ -1637,8 +1663,12 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('_ambiguities', 'family', 'middle', 'suffix'), "a564b97f7162"), "fix(#424) an unlisted abbreviation is as transparent as a listed title to the leading particle, the P4 example": _Claim(1, ('_ambiguities', 'family', 'given'), "42b69cf1b320"), - "fix(#424) the maiden walk stops before the trailing numeral": - _Claim(1, ('_ambiguities', 'maiden', 'suffix'), "cbe5bdd97317"), + "fix(#424/#445) the maiden walk stops before the trailing numeral, and the lone name word is the family": + _Claim(1, ('_ambiguities', 'family', 'given', 'maiden', 'suffix'), "cbe5bdd97317"), + "fix(#445) a maiden marker makes the lone name word the family": + _Claim(6, ('family', 'given'), "f521c94c79fc"), + "fix(#445) the lone name word beside a marker a connective join no longer absorbs": + _Claim(1, ('family', 'given', 'maiden', 'middle'), "52544a41dd62"), "fix(#424) a marker followed only by the numeral is just a word": _Claim(1, ('_ambiguities', 'family', 'maiden', 'middle', 'suffix'), "aaf53040b071"), "fix(#369) the bound given-name join takes a particle-and-bound word, so no fork is reported": @@ -1910,11 +1940,15 @@ class _Excluded(NamedTuple): # 51 -> 54 as rules.md gained the bracketed Polish examples # (#434): 'Maria Kowalska (z domu Nowak)', 'Maria Kowalska # (z domu)', and the boundary 'Anna z (domu) Nowak' M2 gained - # when the clause-straddling defect was fixed. Growth in the - # corpus, not in the exclusion -- its regex is untouched -- and - # `absorbed_by` stayed empty, so no rule reaches the protected - # shape. - _Excluded(54, "e2924f45c9d8", ()), + # when the clause-straddling defect was fixed. 54 -> 55 for + # M4's markerless clause example, 'Smith (Jones)' (#445), + # which the exclusion costs nothing: under the default facade + # the pair is a nickname one, so the name reads family + # 'Smith', nickname 'Jones' exactly as 1.4.0 read it and has + # no diff to silence. Growth in the corpus, not in the + # exclusion -- its regex is untouched -- and `absorbed_by` + # stayed empty, so no rule reaches the protected shape. + _Excluded(55, "7ad8ff289eb2", ()), } diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index f9ba35a8..c01b7753 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -1378,8 +1378,12 @@ def test_the_clause_free_corpus_is_not_empty() -> None: already -- the delimiter strip, then the move from word membership to the marker predicate, which a phrase entry made necessary -- and the one-liner in that filter's comment recounts - every stage: 642 names before either, 632 after the strip, 629 - once the predicate decides. So the floor is what says a future + every stage: 645 names before either, 635 after the strip, 631 + once the predicate decides. Recounted 2026-08-27 by pasting it: + the four rules-corpus rows #445 added move the last stage by two + (the other two carry a marker), and the first two figures were + already one low before that -- a corpus row landed without the + recount the comment asks for. So the floor is what says a future widening emptied it. Deliberately far below today's count: this asks whether the filter still selects a corpus, not what the corpus holds.""" @@ -1413,15 +1417,52 @@ def test_a_maiden_clause_changes_nothing_else(name: str) -> None: Jane née Smith' given 'Jane' -- and no longer moves, so the assertion now covers every name that reaches the marker with something to stand behind. + + One class of name moves ON PURPOSE, and is asserted MOVING rather + than skipped: rules.md#M4 makes the clause decide a name that + holds exactly one name word, because a marker announces a former + surname and only means anything beside a current one. Fourteen + of these names are in that class ('Smith', 'Smith Jr.', "'Smitty' + Jones Jr.", 'John V', 'de' ...), and the flip they assert is + exactly given -> family with every other field standing still -- + which is the whole of what #445 changed, checked over the corpus + rather than at the six rows cases.py carries. + + The `flips` predicate below is a second reading of M4's guard, and + that is a maintenance cost taken deliberately rather than the + silent-drift hazard it resembles: it fails LOUDLY in both + directions -- narrow M4 and the flip assertion fails, widen it and + the stands-still assertion does. What it buys is the carve-out + witness. Review offered the cheaper form, asserting only that + (given, family) is either unchanged or moved wholesale, with no + predicate at all; under that form a name whose word the vocabulary + claims would be free to flip, and dropping the `vocab:bound-given` + carve-out would stop failing here on 'abdul'. The corpus is the + only place that name is asked. """ base = parse(name) if not (base.given or base.middle or base.family or base.title or base.suffix): pytest.skip("nothing before the marker at all: M2 leaves it a word") + # M4's guard, read off the base parse: one GIVEN token, no other + # name word, no title (a titled name is H1's), and neither + # carve-out tag. Tokens rather than fields because the rule counts + # tokens -- a joined 'de la Vega' in `given` is one field and + # three of them. + givens = [t for t in base.tokens if t.role is Role.GIVEN] + flips = (len(givens) == 1 and not base.middle and not base.family + and not base.title + and not ({"initial", "vocab:bound-given"} & givens[0].tags)) with_clause = parse(name + " née Jones") assert with_clause.maiden == "Jones" - for field in ("title", "given", "middle", "family", "suffix", - "nickname"): + moved = ("title", "middle", "suffix", "nickname") if flips else ( + "title", "given", "middle", "family", "suffix", "nickname") + if flips: + assert (with_clause.family, with_clause.given) == (base.given, ""), ( + f"{name!r}: one name word beside a maiden clause is the " + f"family name (rules.md#M4), but it reads family " + f"{with_clause.family!r} / given {with_clause.given!r}") + for field in moved: assert getattr(with_clause, field) == getattr(base, field), ( f"{name!r}: {field} reads {getattr(base, field)!r} alone and " f"{getattr(with_clause, field)!r} with a maiden clause") diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 440a470b..f065906b 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -356,9 +356,9 @@ def _is_latin_only(name: str) -> bool: #: decision when a corpus is added, the way the Script tables do. _CORPUS_FLOORS = { "corpus.jsonl": 480, # 486 today, from v1's banks at a pinned ref - "corpus_cjk.jsonl": 95, # 97 today, generated from the case table + "corpus_cjk.jsonl": 95, # 98 today, generated from the case table "corpus_issues.jsonl": 370, # 381 today, harvested and append-only - "corpus_rules.jsonl": 150, # 175 today, generated from rules.md + "corpus_rules.jsonl": 150, # 234 today, generated from rules.md } diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 655bbdb3..f7543f31 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -36,6 +36,7 @@ "Freiherr von Berg MA" "Freiherr von Richthofen V" "Gal·la Serra" +"Garcia" "Garcia Juan Carlos" "Hans „Erster“ und “Zweiter” Müller" "Hassan Mohamad Ali" @@ -139,9 +140,13 @@ "Sir John" "Sir abdul van der Berg" "Sir de Mesnil" +"Smith" +"Smith (Jones)" "Smith (née Jones)" "Smith Jr." "Smith Jr., Mr." +"Smith née Jones" +"Smith née Jones PhD" "Smith, Abd" "Smith, Dr." "Smith, Dr. Jr." diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 7916b1da..dfe86fab 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -132,7 +132,10 @@ issue = "fix(#274) maiden markers consumed" # do carry marker vocabulary, so the presence guard is satisfied. # Hence the two context tests, which say what this rule has always # meant: a marker with a name word BEFORE it and one AFTER it. Reach -# 16 -> 11, and all seven names it explains are inside the eleven. +# 16 -> 11 when they were added, and every name it explained then was +# inside the eleven -- figures from that measurement rather than from +# today's corpus, which has grown since. Today it explains 12 +# (measured 2026-08-27, after #445). # # Four claimed names still do not move, recorded rather than excluded: # 'Jane Smith, nee Jones' (post-comma, M2's remaining Accepted limit), @@ -142,20 +145,31 @@ issue = "fix(#274) maiden markers consumed" # connective membership -- a second alternation this rule cannot # spend. # -# revisit when #445 lands, and this is the half of that shape the -# bracketed fix(#335) rules do NOT cover. This rule declares `family`, -# and eight names it claims read family '' because the marker takes -# every word after it -- the bare spelling of exactly what #445 asks -# about. If #445 lands as "keep the family", their 1.4.0 diff stays a -# subset of {maiden, middle, family} and this rule keeps explaining -# them with the gate green, which is the silent absorption the -# four-field split exists to prevent, on the same field and the same -# baseline. The checklist to re-measure, all one name word before the -# marker: 'Janey née Jones' and 'abd née Jones' (the exact N3-parallel -# shape, one word after the marker too), plus 'Jane née Jones Smith', -# 'Jane née and Jones Smith', 'John née Jones Smith V', -# 'John née Jones Smith Ma', 'J. née Jones Smith V' and -# 'Jane née Jones J. V'. +# #445 landed 2026-08-27, and the prediction this note used to carry +# was wrong in a way worth keeping on the record. It reasoned that a +# "keep the family" fix would leave these diffs a subset of {maiden, +# middle, family}, so this rule would go on explaining them and would +# absorb the change in silence. The rule #445 actually chose MOVES +# the one name word rather than adding one: `given` empties as +# `family` fills, so every name that moved left this rule's fields +# and arrived UNEXPLAINED -- loud, at all three baselines. The note +# earned its keep all the same, and that is why it is corrected here +# rather than deleted: it named the right rule, the right field and +# the right eight names, and re-measuring its checklist is what +# showed its own reasoning wrong. +# +# Six of the eight moved: five by the fix(#445) rule at the end of +# this file, and 'John née Jones Smith Ma' by the fix(#424/#445) +# acronym rule above it, its diff carrying the suffix v1 read as +# well. The other two still belong here, and they are +# rules.md#M4's own carve-outs: 'abd née Jones', whose word the +# bound-given vocabulary has claimed, and 'J. née Jones Smith V', +# whose word is an initial. Their diffs are unchanged to the field, +# which is what makes them a boundary rather than an oversight -- if +# either ever gains a `given` move it lands UNEXPLAINED, since this +# rule does not declare that field. The two other fix(#445) names, +# 'Smith née Jones' and 'Smith née Jones PhD', are corpus rows M4's +# own example lines added. name_regex = "(?i)(? maiden 'Jones Smith Ma': @@ -1084,8 +1098,19 @@ issue = "fix(#424) accepted: the maiden walk keeps a bare acronym" # it needs includes the words the marker removes). The fix(#274) rule # cannot carry it: its fields omit the suffix v1 read. A rules.md # Accepted example, first witnessed here. +# +# Renamed for #445 (2026-08-27) rather than widened in place, because +# the diff now has two causes and one rule has to explain the whole +# of it. What the walk leaves is a single name word, and rules.md#M4 +# makes that word the family: given 'John' -> family 'John', so +# `given` joins the four fields above. Attributing that to #424 alone +# would have been false, and leaving this rule at four fields would +# have made it explain nothing while the name went UNEXPLAINED -- +# which is exactly how the gate reported it before this edit. The +# name is #445's only mover whose diff is not the shape the fix(#445) +# rule below carries, the suffix v1 read being the difference. name_regex = "(?i)^john\\s+n[ée]e\\s+jones\\s+smith\\s+ma$" -fields = ["middle", "family", "suffix", "maiden"] +fields = ["given", "middle", "family", "suffix", "maiden"] [[change]] issue = "fix(#360) ste moved into the never-given particles with mc" @@ -1509,20 +1534,56 @@ issue = "fix(#335) a marker-led clause leaves the one name word its bare reading # N3 reads a name that is only a nickname plus one name word as "that # word is the family name", and a marker-led clause is not a nickname # clause, so N3 no longer reaches this shape: the word keeps the -# reading the bare "Smith née Jones" gives it, which is `given`. +# reading the bare "Smith née Jones" gives it. # # Its own rule rather than a sixth alternative in the fix(#335) rule -# above, for the reason that rule states: this diff moves FOUR fields -# where those move two, and one rule holding both would have to -# declare the union, pre-excusing a given/family regression on the -# five. The empty `family` is not this change's -- the bare spelling -# reads that way on 2.0.0 and 2.1.0 too -- but it is inside this -# diff, so the rule declares it. -# -# revisit when #445 lands: this rule declares `family`, and `family` -# is exactly the field #445 would change. Fixing it there moves this -# diff, and this rule would absorb the change in silence at all three -# baselines -- the same failure the four-field split exists to avoid, -# displaced onto a different field. Re-measure this rule then. +# above, for the reason that rule states: one rule holding both would +# have to declare the union of two different diffs and would +# pre-excuse whatever else landed inside it. +# +# The revisit note this block carried was answered on 2026-08-27, and +# the answer is the opposite of what it feared. It expected #445 to +# widen this diff and be absorbed here; #445 SHRANK it. The bare +# reading the word keeps is now `family` (rules.md#M4), and 1.4.0 +# read this name family 'Smith', nickname 'née Jones' -- so `given` +# and `family` agree with the baseline again and only the clause +# changes hands. The declaration is narrowed to the two fields that +# still move, which is what the note asked for: a `given` or `family` +# move on this name is UNEXPLAINED from here on rather than absorbed. +# Measured, not reasoned -- this name is the one shape in #445's +# class that the gates do NOT report, and its diff at 2.0.0 and +# 2.1.0, where the baseline read `given`, is unchanged at four +# fields. name_regex = "(?i)^Smith \\(n[ée]e Jones\\)$" -fields = ["given", "family", "nickname", "maiden"] +fields = ["nickname", "maiden"] + +[[change]] +issue = "fix(#445) a maiden marker makes the lone name word the family" +# rules.md#M4 -- "a maiden name standing beside exactly one name word +# makes that word the family name". A maiden name is a FORMER family +# name, which only means something beside a current one, so the word +# left standing beside it is the surname the bearer uses now. Seven names, one field set: 'Smith née Jones' +# read first 'Smith', middle 'née', last 'Jones' on 1.4.0 and reads +# family 'Smith', maiden 'Jones' here. +# +# Three of the four fields are #274's marker consumption -- v1 had no +# maiden support, so the marker sat in the name as a middle word and +# the last word was the family -- and the fix(#274) rule above +# carried these names until #445 added `given` to their diffs and +# they left its declaration. Only `given` is new; the rule declares +# all four because a rule explains a name's whole diff. +# +# Anchored, and spelled out one name per alternative, for the reason +# M4 itself gives. The shape regex a reader would write first -- one +# word, then a marker -- also matches M4's two carve-outs +# ('abd née Jones', 'J. née Jones Smith V'), and a rule declaring +# `given` over those two would stand ready to explain the very +# given -> family move the carve-outs exist to prevent. They stay +# with fix(#274), whose fields do not include it. +# +# 'Smith née Jones' and 'Smith née Jones PhD' are corpus rows this +# change itself added, being M4's own example lines; the other five +# were already here, and were five of the eight the fix(#274) note +# above listed as the checklist to re-measure. +name_regex = "(?i)^(?:Janey n[ée]e Jones|Jane n[ée]e Jones J\\. V|Jane n[ée]e Jones Smith|Jane n[ée]e and Jones Smith|John n[ée]e Jones Smith V|Smith n[ée]e Jones|Smith n[ée]e Jones PhD)$" +fields = ["given", "middle", "family", "maiden"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index d5336caf..33c33efe 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -625,7 +625,7 @@ name_regex = "(?i)^john\\s+van\\s+der\\s+berg\\s+ma$" fields = ["family", "suffix", "_ambiguities"] [[change]] -issue = "fix(#424) the maiden walk stops before the trailing numeral" +issue = "fix(#424/#445) the maiden walk stops before the trailing numeral, and the lone name word is the family" # 'John née Jones Smith V': rules.md#M2 -- "takes the words after it # -- up to any suffix word, or the trailing roman numeral assign # reads as the suffix (S2) -- as the maiden name". maiden 'Jones @@ -633,8 +633,18 @@ issue = "fix(#424) the maiden walk stops before the trailing numeral" # with the suffix-piece test too. No 1.4.0 twin: v1 had no maiden # support, and the fix(#274) rule carries the name there. A rules.md # example; no differential corpus name has the shape. +# +# Renamed for #445 (2026-08-27) rather than widened in place, because +# the diff has two causes now and one rule has to explain the whole +# of it. The walk leaves a single name word, and rules.md#M4 makes +# that word the family: given 'John' -> family 'John' joins the three +# fields above. Attributing that to #424 would be false, and leaving +# the declaration at three fields left this rule explaining nothing +# while the name went UNEXPLAINED -- which is how the gate reported +# it before this edit, at 2.0.0 and 2.1.0 alike. No 1.4.0 twin to +# rename: the fix(#445) rule there carries this name. name_regex = "(?i)^john\\s+n[ée]e\\s+jones\\s+smith\\s+v$" -fields = ["suffix", "maiden", "_ambiguities"] +fields = ["given", "family", "suffix", "maiden", "_ambiguities"] [[change]] issue = "fix(#424) a marker followed only by the numeral is just a word" @@ -824,7 +834,11 @@ issue = "fix(#412) a connective join no longer absorbs the maiden marker beside # # Checked against this baseline's own run: 2.0.0 reads both names the # way 2.1.0 does, so the same two arrive UNEXPLAINED here. The -# expected_since_2.1.0.toml copy carries the fuller account. +# expected_since_2.1.0.toml copy carries the fuller account. Since +# #445 (2026-08-27) this rule explains only 'Jane van der Berg née y +# Jones'; the other name is left with one name word once the marker +# takes the rest, so its diff gained `given` and it has a rule of its +# own at the end of this file, as it does at 2.1.0. name_regex = "(?i)\\bn[eé]e\\s+(y|and)\\b" fields = ["middle", "family", "maiden"] @@ -1035,7 +1049,9 @@ issue = "fix(#335) a marker-led clause leaves the one name word its bare reading # N3 reads a name that is only a nickname plus one name word as "that # word is the family name", and a marker-led clause is not a nickname # clause, so N3 no longer reaches this shape: the word keeps the -# reading the bare "Smith née Jones" gives it, which is `given`. +# reading the bare "Smith née Jones" gives it -- `given` when this +# rule was written, and `family` since #445 gave the bare spelling +# the same reading from the other side. # # Its own rule rather than a sixth alternative in the fix(#335) rule # above, for the reason that rule states: this diff moves FOUR fields @@ -1045,10 +1061,64 @@ issue = "fix(#335) a marker-led clause leaves the one name word its bare reading # reads that way on 2.0.0 and 2.1.0 too -- but it is inside this # diff, so the rule declares it. # -# revisit when #445 lands: this rule declares `family`, and `family` -# is exactly the field #445 would change. Fixing it there moves this -# diff, and this rule would absorb the change in silence at all three -# baselines -- the same failure the four-field split exists to avoid, -# displaced onto a different field. Re-measure this rule then. +# The revisit note this block carried was answered on 2026-08-27, and +# re-measured rather than reasoned about. #445 does change the bare +# reading -- the one name word beside a maiden clause is the family +# now (rules.md#M4) -- but this rule's diff is unchanged AT THIS +# BASELINE, because the baseline is what read `given` and the +# baseline cannot move. All four fields still differ, so the +# declaration stands as written. What did move is the 1.4.0 twin of +# this rule, where the baseline read family 'Smith' and the diff +# shrank to {nickname, maiden}; that ledger narrows it there. name_regex = "(?i)^Smith \\(n[ée]e Jones\\)$" fields = ["given", "family", "nickname", "maiden"] + +[[change]] +issue = "fix(#445) a maiden marker makes the lone name word the family" +# rules.md#M4 -- "a maiden name standing beside exactly one name word +# makes that word the family name". A maiden name is a FORMER family +# name, which only means something beside a current one, so the word +# left standing beside it is the surname the bearer uses now: 'Smith née Jones' read given 'Smith' with an +# empty family on this baseline and reads family 'Smith' here. +# +# Two fields and no more. The marker consumption these names also +# carry is this baseline's reading already, so `given` and `family` +# are the whole diff -- which is what makes them one rule, and what +# keeps out the two names whose diffs are compounds (the connective +# join below, the trailing numeral above). +# +# Anchored, and spelled out one name per alternative, for the reason +# M4 itself gives. The shape regex a reader would write first -- one +# word, then a marker -- also matches M4's two carve-outs +# ('abd née Jones', 'J. née Jones Smith V', both corpus names and +# neither moving), and a rule declaring `given` over those two would +# stand ready to explain the very given -> family move the carve-outs +# exist to prevent. +# +# 'Smith née Jones' and 'Smith née Jones PhD' are corpus rows this +# change itself added, being M4's own example lines. +# 'John née Jones Smith Ma' sits here rather than with an acronym +# rule of its own: this baseline already reads it maiden 'Jones Smith +# Ma', so the acronym is no part of the diff. At 1.4.0, where v1 read +# suffix 'Ma', it is the one name of this class that needs one. +name_regex = "(?i)^(?:Janey n[ée]e Jones|Jane n[ée]e Jones J\\. V|Jane n[ée]e Jones Smith|John n[ée]e Jones Smith Ma|Smith n[ée]e Jones|Smith n[ée]e Jones PhD)$" +fields = ["given", "family"] + +[[change]] +issue = "fix(#445) the lone name word beside a marker a connective join no longer absorbs" +# 'Jane née and Jones Smith', whose diff is a compound of #412 and +# #445 and so cannot sit in either's plain rule. #412 stopped the +# connective join from swallowing the marker, so the marker is taken +# and 'and Jones Smith' is the maiden name; what that leaves is one +# name word, and rules.md#M4 makes it the family. middle +# 'née and Jones', family 'Smith' -> family 'Jane', maiden +# 'and Jones Smith'. +# +# Its own rule rather than a widening of the fix(#412) rule above, +# which claims two names: the other one keeps a given name, and one +# rule holding both would have to declare `given` for both and +# pre-excuse its loss where losing it would be a regression. At 1.4.0 +# the fix(#445) rule carries this name with the same four fields, v1 +# having read the marker as a middle word. +name_regex = "(?i)^Jane n[ée]e and Jones Smith$" +fields = ["given", "middle", "family", "maiden"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index adf4880c..72408160 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -318,7 +318,7 @@ name_regex = "(?i)^john\\s+van\\s+der\\s+berg\\s+ma$" fields = ["family", "suffix", "_ambiguities"] [[change]] -issue = "fix(#424) the maiden walk stops before the trailing numeral" +issue = "fix(#424/#445) the maiden walk stops before the trailing numeral, and the lone name word is the family" # 'John née Jones Smith V': rules.md#M2 -- "takes the words after it # -- up to any suffix word, or the trailing roman numeral assign # reads as the suffix (S2) -- as the maiden name". maiden 'Jones @@ -326,8 +326,18 @@ issue = "fix(#424) the maiden walk stops before the trailing numeral" # with the suffix-piece test too. No 1.4.0 twin: v1 had no maiden # support, and the fix(#274) rule carries the name there. A rules.md # example; no differential corpus name has the shape. +# +# Renamed for #445 (2026-08-27) rather than widened in place, because +# the diff has two causes now and one rule has to explain the whole +# of it. The walk leaves a single name word, and rules.md#M4 makes +# that word the family: given 'John' -> family 'John' joins the three +# fields above. Attributing that to #424 would be false, and leaving +# the declaration at three fields left this rule explaining nothing +# while the name went UNEXPLAINED -- which is how the gate reported +# it before this edit, at 2.0.0 and 2.1.0 alike. No 1.4.0 twin to +# rename: the fix(#445) rule there carries this name. name_regex = "(?i)^john\\s+n[ée]e\\s+jones\\s+smith\\s+v$" -fields = ["suffix", "maiden", "_ambiguities"] +fields = ["given", "family", "suffix", "maiden", "_ambiguities"] [[change]] issue = "fix(#424) a marker followed only by the numeral is just a word" @@ -731,6 +741,13 @@ issue = "fix(#412) a connective join no longer absorbs the maiden marker beside # # Checked against this baseline's own run, not copied: both names # arrive UNEXPLAINED here and at 2.0.0 alike, and nothing else does. +# Since #445 (2026-08-27) this rule explains only the first of the +# two: the marker in 'Jane née and Jones Smith' takes the rest of the +# name and leaves one name word, so `given` empties into `family` and +# the diff outgrew these three fields. It has a rule of its own at +# the end of this file. The names were split rather than the fields +# widened, because 'Jane van der Berg née y Jones' keeps a given name +# and adding `given` here would have pre-excused its loss. # At 1.4.0 fix(#274) already classifies them -- v1 has no maiden # field, so every marker-bearing name is a diff there. # @@ -957,7 +974,9 @@ issue = "fix(#335) a marker-led clause leaves the one name word its bare reading # N3 reads a name that is only a nickname plus one name word as "that # word is the family name", and a marker-led clause is not a nickname # clause, so N3 no longer reaches this shape: the word keeps the -# reading the bare "Smith née Jones" gives it, which is `given`. +# reading the bare "Smith née Jones" gives it -- `given` when this +# rule was written, and `family` since #445 gave the bare spelling +# the same reading from the other side. # # Its own rule rather than a seventh alternative in the fix(#335) # rule above, for the reason that rule states: this diff moves FOUR @@ -967,10 +986,64 @@ issue = "fix(#335) a marker-led clause leaves the one name word its bare reading # reads that way on 2.0.0 and 2.1.0 too -- but it is inside this # diff, so the rule declares it. # -# revisit when #445 lands: this rule declares `family`, and `family` -# is exactly the field #445 would change. Fixing it there moves this -# diff, and this rule would absorb the change in silence at all three -# baselines -- the same failure the four-field split exists to avoid, -# displaced onto a different field. Re-measure this rule then. +# The revisit note this block carried was answered on 2026-08-27, and +# re-measured rather than reasoned about. #445 does change the bare +# reading -- the one name word beside a maiden clause is the family +# now (rules.md#M4) -- but this rule's diff is unchanged AT THIS +# BASELINE, because the baseline is what read `given` and the +# baseline cannot move. All four fields still differ, so the +# declaration stands as written. What did move is the 1.4.0 twin of +# this rule, where the baseline read family 'Smith' and the diff +# shrank to {nickname, maiden}; that ledger narrows it there. name_regex = "(?i)^Smith \\(n[ée]e Jones\\)$" fields = ["given", "family", "nickname", "maiden"] + +[[change]] +issue = "fix(#445) a maiden marker makes the lone name word the family" +# rules.md#M4 -- "a maiden name standing beside exactly one name word +# makes that word the family name". A maiden name is a FORMER family +# name, which only means something beside a current one, so the word +# left standing beside it is the surname the bearer uses now: 'Smith née Jones' read given 'Smith' with an +# empty family on this baseline and reads family 'Smith' here. +# +# Two fields and no more. The marker consumption these names also +# carry is this baseline's reading already, so `given` and `family` +# are the whole diff -- which is what makes them one rule, and what +# keeps out the two names whose diffs are compounds (the connective +# join below, the trailing numeral above). +# +# Anchored, and spelled out one name per alternative, for the reason +# M4 itself gives. The shape regex a reader would write first -- one +# word, then a marker -- also matches M4's two carve-outs +# ('abd née Jones', 'J. née Jones Smith V', both corpus names and +# neither moving), and a rule declaring `given` over those two would +# stand ready to explain the very given -> family move the carve-outs +# exist to prevent. +# +# 'Smith née Jones' and 'Smith née Jones PhD' are corpus rows this +# change itself added, being M4's own example lines. +# 'John née Jones Smith Ma' sits here rather than with an acronym +# rule of its own: this baseline already reads it maiden 'Jones Smith +# Ma', so the acronym is no part of the diff. At 1.4.0, where v1 read +# suffix 'Ma', it is the one name of this class that needs one. +name_regex = "(?i)^(?:Janey n[ée]e Jones|Jane n[ée]e Jones J\\. V|Jane n[ée]e Jones Smith|John n[ée]e Jones Smith Ma|Smith n[ée]e Jones|Smith n[ée]e Jones PhD)$" +fields = ["given", "family"] + +[[change]] +issue = "fix(#445) the lone name word beside a marker a connective join no longer absorbs" +# 'Jane née and Jones Smith', whose diff is a compound of #412 and +# #445 and so cannot sit in either's plain rule. #412 stopped the +# connective join from swallowing the marker, so the marker is taken +# and 'and Jones Smith' is the maiden name; what that leaves is one +# name word, and rules.md#M4 makes it the family. middle +# 'née and Jones', family 'Smith' -> family 'Jane', maiden +# 'and Jones Smith'. +# +# Its own rule rather than a widening of the fix(#412) rule above, +# which claims two names: the other one keeps a given name, and one +# rule holding both would have to declare `given` for both and +# pre-excuse its loss where losing it would be a regression. At 1.4.0 +# the fix(#445) rule carries this name with the same four fields, v1 +# having read the marker as a middle word. +name_regex = "(?i)^Jane n[ée]e and Jones Smith$" +fields = ["given", "middle", "family", "maiden"] From a473f9054efed13e4a1b484bb76faa1cb482b39f Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 27 Aug 2026 17:50:41 -0700 Subject: [PATCH 2/3] docs(decisions,release_log): record the lone-word rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit decisions.md#M4, written unwrapped, one paragraph per line: - Derek's rationale in his own terms -- a marker marks a previous surname, so including one makes no sense without another surname for it to clarify, and a clause that leaves one name word switches the assumption from given to family. - The three empty-family repairs of this bundle share a SYMPTOM, not a cause, and the entry says so rather than claiming one pattern: #410 and #411 were counts that included words which are no part of the name, and the grep that finds a fourth is a rule counting name words without asking which of them are name words. #445 changes no count at all -- its cause is O4's silence at n=1, and the site that grep would miss is _name_positions's `count == 1` branch, which is what O5 records. - Both carve-outs and the mechanism they rest on, with the corpus witness for each and the mutation each one catches. - That the interior half is a new reading while the bracketed spelling restores 1.4.0. - The N3 precedence corrected while drafting, and the M1 keying corrected in review: M4 is keyed on the maiden NAME, since a configured pair produces one with no marker anywhere. - The M4/H1 counting gap, recorded rather than closed. - What the ledger markers bought and what they got wrong, with the two different counts named: nine corpus names change reading, eight arrive as new diffs, and the markers named nine names of which seven moved. - The relationship to #449: this rule reduces that issue's population rather than competing with it. rules.md#M4 gains its `history: decisions.md#M4` pointer, held back from the previous commit so that commit's anchors all resolve. The release-log bullet is written from measurement: nine of the 1,090 corpus names change reading, eight arriving as new diffs at every baseline and the ninth having its 1.4.0 diff shrink instead. It says which spellings move -- the declared-pair spelling included, which carries no marker -- which two do not and that both carve-out witnesses are corpus names, and it tells a caller reading the lone word out of `given` that this is the release where it moves. Two bullets already in this release stated readings this change falsifies, and both are corrected: #335's interior example said the bare "Jane née Jones Smith" has no family at all, and #412's said "Jane née and Jones Smith" reads given `Jane`. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 14 ++++++++++++++ docs/design/rules.md | 2 +- docs/release_log.rst | 6 ++++-- 3 files changed, 19 insertions(+), 3 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 696ee4bc..ec57342a 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -535,6 +535,20 @@ Declined: - 2026-08-26 #335 — M3 makes the marker vocabulary act in a THIRD position. M2's claim acts on the word after a name word; M1's on the content of a configured pair; M3's now on the first word of any bracketed clause, under the default policy. #vocabulary-collisions C-i is keyed on the position a claim acts on, so extending where a rule acts re-asks C-i of the whole set it reads, and nothing in the process prompts that — the rule was designed, reviewed and approved without the set being re-examined. Review afterwards found one entry that fails C-i, the Czech abbreviation roz, and its disposition is this file's Excluded (MAIDEN_MARKERS) block, which is keyed to the vocabulary set rather than to any rule: it was removed. Two things worth separating, since the finding surfaced during this work and is easy to file under it — the defect was M2's and predates #335 (bare "Rosalind Roz Smith" lost its family name on master), and M3's own boundary already excluded the common one-word "(Roz)" spelling. What belongs to M3 is only the widening, and the lesson is the general one: re-ask C-i of a vocabulary set whenever a rule extends where its claim acts. - 2026-08-26 #335 — what configuring maiden_delimiters still buys, since M3 removes the commonest reason to reach for it: markerless clauses, and one-word clauses. "Cherice J. (Johnson) Williams" is a real US convention and a corpus name, and nothing in that clause says maiden, so it stays a nickname by default — only a caller who knows their data can say otherwise. The knob is a declaration about the DATA; M3 is a reading of what a clause declares about itself. +### M4 — a maiden marker makes the lone name word the family + +- 2026-08-27 #445 (M4, PR on fix/445-maiden-marked-lone-name) — Derek's rationale, in his terms: normally if there is only one name we assume it is a given name because we have to pick one, but maiden markers mark a previous surname before marriage, so it would not make sense to include one if there was no other surname for it to clarify — and given that, when a marker leaves only one name word we switch the assumption and read that word as the family name. `parse("Smith née Jones")` reported given 'Smith' with an EMPTY family through 2.1 and reads family 'Smith', maiden 'Jones' since. The rule is written in post_rules as H1's sibling and placed directly under it, so the interaction is decidable by reading rather than by running: where H1 fired there is no GIVEN left for M4 to move, and where H1 declined because the title addresses by given name ("Sir John née Jones") the `not titles` guard keeps M4 off the same word. A titled name is H1's at both outcomes, which is what keeps H1's given-name-title carve-out intact. +- 2026-08-27 #445 — this is the THIRD instance of one shape repaired in the 2.2 bundle, and the pattern is worth stating because a fourth site would be findable by looking for it rather than by waiting for a report. #410 (H1): a title plus one name word reported no family the moment a suffix, nickname or maiden name stood beside it. #411 (P5's reserve): the bound given-name join counted the marker and the maiden name as words it could spend, and spent the family name. #445 (the positional convention): the lone name word beside a maiden clause stayed `given` and the family stayed empty. Every one is an ANNOTATION beside the name being counted as part of the name, and every one showed as an empty `family` on input that plainly names a family. The shape to grep for is a rule that counts name words without first asking which of them are name words. +- 2026-08-27 #445 — O4 was SILENT at exactly one name word, which is why the shipped reading had no rule to point at and why O5 now exists. O4 reads a name by comparing where its words stand ("the first name word is the given name, the last is the family name"); with one word the first IS the last, so there is nothing to compare and the rule decides nothing. O5 records the reading as the convention it is — a guess fixed in advance so that the same input always reads the same way, not a determination about the word — and names H1, N3 and M4 as the rules that DO decide such a name. Written that way deliberately: a rule asserting "a lone name word is the given name" as a fact would have made #445 a contradiction of the documented behavior instead of an exception to a documented guess, and would make #449 one too. +- 2026-08-27 #445 — the two carve-outs are not inventions, and both rest on mechanisms.md#TWO-LAYER-ASSIGN: the positional layer never overrides a vocabulary claim, and M4 changes only what POSITION decided. A word the vocabulary claims as a given name keeps that reading ('abd née Jones' — `vocab:bound-given`), and so does a word written as an initial ('J. née Jones Smith V' — the `initial` shape tag). Both are read off tags classify already recorded rather than off a predicate of M4's own, and both witnesses are real corpus names rather than constructed cases, so each carve-out has something in the differential that would notice its loss. Mutation-checked on a scratch copy: dropping the `initial` carve-out fails that name's case row, its facade twin, M4's own boundary example and the pre-existing assertion in test_the_chain_and_the_walk_stop_where_the_peel_begins; dropping `vocab:bound-given` fails the other name's two runners, M4's boundary, P5's own doc example and the corpus-wide maiden-clause property on 'abdul'. +- 2026-08-27 #445 — the two halves of Derek's answer differ in their relationship to 1.4.0, and a reader should not have to re-derive it. The bracketed spelling RESTORES v1: `Smith (née Jones)` read family 'Smith', nickname 'née Jones' on 1.4.0, 2.0.0 and 2.1.0 alike, and now reads family 'Smith', maiden 'Jones' — the clause changes hands and the family name stays put, so the 1.4.0 gate's diff on that name SHRANK to {nickname, maiden}. The interior spelling is a NEW reading: `Jane née Jones Smith` read first 'Jane', middle 'née Jones', last 'Smith' on 1.4.0 and reads family 'Jane', maiden 'Jones Smith' now, the real surname being inside the maiden value by M2's greedy take. That the marker stands inside the name changes nothing for M4, which counts what the take LEAVES rather than where the marker stood — one name word left that way is one name word, which is the widest half of the decision and the half no earlier version agrees with. +- 2026-08-27 #445 — a precedence claim corrected while drafting, and it is exactly the class this bundle keeps producing. The first draft of M4 said a name carrying a nickname is N3's rather than this rule's. Measured false: `'Smitty' Jones Jr. née Smith` reads family 'Jones', because N3's count does not set a suffix aside, so N3 declines and M4 fires. A nickname holds nothing off — where N3 has already named the family M4 finds nothing left to move, and where N3 declined M4 names it. The general lesson is the one #410 and #411 already taught from the other side: a rule that counts name words and a rule that counts something else will disagree at the edges, and the disagreement is only visible if the interaction is measured rather than asserted. +- 2026-08-27 #445 — what the ledger's `# revisit when #445 lands` markers bought, and what they got wrong. Four blocks carried them, and each predicted that a "keep the family" fix would leave the diff a subset of the fields its rule already declared, so the rule would go on explaining the name and the change would be absorbed with the gate green. That was wrong in a way the rule's own shape settles: M4 MOVES the one name word rather than adding one, so `given` empties as `family` fills, the diff outgrows every declaration, and eight names arrived UNEXPLAINED at all three baselines. Two counts run through this and they are different sets, so name which is which: NINE corpus names change reading, of which eight arrive as new diffs — the ninth, 'Smith (née Jones)', has its 1.4.0 diff SHRINK instead, its family agreeing with v1 again. The four markers between them named NINE names to re-measure, of which seven moved: the two that did not are M4's carve-outs, and the two movers the markers could not name were not corpus names until this change's own examples added them. The markers earned their keep all the same — they named the right rule, the right field and very nearly the right set — so they are corrected in place rather than deleted, and the correction says what actually happened. One of the four moved in the OPPOSITE direction from its own prediction: the fix(#335) rule for 'Smith (née Jones)' at 1.4.0 feared absorbing a widened diff and instead had its diff shrink to {nickname, maiden}, so that rule is narrowed to the two fields that still move. +- 2026-08-27 #445 — the corpus-wide property test is the repair worth copying. tests/v2/test_parser.py's `test_a_maiden_clause_changes_nothing_else` asserts over the corpus that appending " née Jones" adds a maiden name and moves no other field, and M4 falsifies it for fourteen names. Skipping them would have bought a green suite and lost the check; instead the test computes M4's guard from the base parse's tokens and asserts the flip — family takes what `given` held, every other field standing still — so the class that used to be a hole is now the strongest witness the rule has, fourteen names against the six rows cases.py carries. It is also what executes M4's nickname precedence, `'Smitty' Jones Jr.` being one of the fourteen. +- 2026-08-27 #445 (review round) — M4 is keyed on the maiden NAME, not on a marker, and the statement was corrected to say so after two reviewers measured the same defect independently. The guard tests for a token in the MAIDEN role, and M1's caller-configured pair produces one with no marker anywhere: under `Policy(maiden_delimiters=frozenset({("(", ")")}))`, `Smith (Jones)` reads family 'Smith', maiden 'Jones', and so do M1's own boundary examples `Smith (Nee)` and `Smith (z domu)`. The BEHAVIOUR is right — the rationale transfers to a declared pair without a word of change, since what announces a former surname is the clause, not the vocabulary that marked it — so the fix was to the rule, which had said "a recognized maiden marker that takes its name and leaves exactly one name word" and now says "a maiden name standing beside exactly one name word". That covers M1, M2 and M3 uniformly, M1 is in M4's `interacts:` and M4 in M1's, and the configured path has an example line of its own. The general lesson is the one this bundle keeps teaching from a new angle: a statement written from the case that prompted the change describes that case, not the code, and only measuring the OTHER paths into the same guard finds the difference. +- 2026-08-27 #445 (review round) — a known gap recorded rather than closed, the way H1 carries its own. M4's guard counts GIVEN tokens; H1 counts nothing at all (it tests which roles are unoccupied). So a name word another rule has joined counts as several here and M4 declines where H1 fires: `Dr. Dean of Chemistry` reads family 'Dean of Chemistry' while `Dean of Chemistry née Jones` keeps given 'Dean of Chemistry', the connective join (P3) having left three GIVEN tokens. rules.md#P3 says the joined part is one name word wherever another rule counts them, so the two siblings genuinely disagree and this is a gap rather than a boundary. Not closed here: widening the count to units moves zero corpus names (measured), and a behaviour change nobody has approved does not belong in a branch whose blast radius was settled. The sibling claim in the code comment and in this entry is narrowed to match — sibling except in what it counts. +- 2026-08-27 #445 — the relationship to #449, briefly, because the two look like the same question and are not. #449 asks whether a lone name word with nothing to decide it should REPORT an ambiguity; #445 removes a class of names from that population by giving them something that decides. So #445 reduces #449's reach rather than competing with it, and #449's measurement — a naive "one name word, nothing decided it" condition fires on 19 of 1,085 corpus names, only three or four of them the case it is about — is re-taken after this lands. O5 cites #449 in prose and deliberately carries no `deviates:` marker: that marker asserts an intended output the runner then checks strictly, and #449 has not decided one. + ### O1 — East Slavic rotation - (v1 era, PR #154) — why patronymic handling is OPT-IN at all: unconditional detection breaks ordinary Latin names whose endings collide (Martin, Franklin, Benjamin), the finding that forced v1's Russian work behind a flag and set the opt-in shape v2 inherited. diff --git a/docs/design/rules.md b/docs/design/rules.md index e8fc8a31..0c2eb032 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -699,7 +699,7 @@ M4. Rationale: a maiden name is a FORMER family name, and a former "Smith née Jones PhD" → family="Smith" "abd née Jones" → given="abd" · boundary "J. née Jones Smith V" → given="J." · boundary - interacts: M1, M2, M3, H1, N3, O5 · implemented: nameparser/_pipeline/_post_rules.py + history: decisions.md#M4 · interacts: M1, M2, M3, H1, N3, O5 · implemented: nameparser/_pipeline/_post_rules.py ## Commas & structure (C) diff --git a/docs/release_log.rst b/docs/release_log.rst index b0882ce0..73a8c4e0 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -49,7 +49,7 @@ Release Log - Add the Polish maiden marker ``z domu`` to the default vocabulary, and let a ``maiden_markers`` entry be more than one word. ``"Maria Kowalska z domu Nowak"`` now reads family ``Kowalska``, maiden ``Nowak``, where every earlier version read the marker as part of the name (1.4.0: middle ``Kowalska z domu``, family ``Nowak``). The bracketed spelling moves with it -- ``"Maria Kowalska (z domu Nowak)"`` read nickname ``z domu Nowak`` and now reads family ``Kowalska``, maiden ``Nowak`` -- while a clause holding nothing but the marker stays a nickname, as a lone ``(née)`` does: ``"Maria Kowalska (z domu)"`` keeps nickname ``z domu``. **This changes what a caller who worked around the old limit should have configured.** Through 2.1 a multi-word entry in any vocabulary field could never match, and the ``UserWarning`` it raised said to split it into separate entries. For maiden markers that advice was wrong, not merely unhelpful: with ``z`` and ``domu`` stored as two entries, ``"Maria Kowalska z domu Nowak"`` reads maiden ``domu Nowak`` -- the marker's own second word inside the value -- and ``"Anna z Nowak"`` reads maiden ``Nowak`` with **no family name at all**, because a bare preposition is then a marker and a marker takes every word after it. If you added ``z`` and ``domu`` separately, remove both: the shipped phrase does the job, and dropping the two word entries is what restores ``"Anna z Nowak"`` to given ``Anna``, middle ``z``, family ``Nowak`` -- which is how it reads on the default vocabulary, unchanged from 1.4.0. The same applies to any other field-splitting workaround for this one field. A phrase is matched over whole tokens that stand together -- a bracketed clause or a comma between the words ends the run, so ``"Anna z (domu) Nowak"`` keeps its family name too -- and longest first, so a phrase and a word entry that starts it can both be configured: with ``geb`` shipped and ``geb von`` added, ``"Jane Smith geb von Braun"`` reads maiden ``Braun`` and ``"Jane Smith geb Braun"`` still reads maiden ``Braun``. ``given_name_titles`` was the only field exempt from the multi-word warning; ``maiden_markers`` is now the second, and the warning still fires for every other field. This addition takes the default marker set to 17 entries. Two differential corpus names move, at all three baselines, and both are examples this change adds to the rules doc; no corpus name held ``domu`` or a bare ``z`` before it (#434) - - Fix a bracketed maiden clause reading as a nickname because its brackets were not declared. ``"Jane Smith nee Jones"`` gave maiden ``Jones``, while ``"Jane Smith (née Jones)"`` gave nickname ``née Jones`` -- the parenthesis pair is a nickname delimiter by default and nothing looked inside it, so the marker word the writer put there was read as part of an alias. This is about every delimiter pair the parser ships, not only the parentheses these examples use: quotes are how a nickname is most often written, and ``"Jane Smith "née Jones""`` and ``"Jane Smith 'née Jones'"`` move from nickname to maiden exactly as the parenthesized spelling does. A bracketed clause whose content opens with a recognized marker word and carries a word after it now reads as the maiden name whichever bucket its pair sits in: ``"Jane Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, and so does the Japanese ``"山田 花子（旧姓 佐藤）"`` (family 山田, given 花子, maiden 佐藤), which needed ``Policy(maiden_delimiters=...)`` through 2.1. The clause is lifted out as a region rather than having its brackets erased, which is what keeps an INTERIOR clause from eating the name behind it: ``"Jane (née Jones) Smith"`` reads given ``Jane``, family ``Smith``, maiden ``Jones``, where erasing the brackets would leave the bare ``"Jane née Jones Smith"``, whose marker takes every word to the end of the name -- maiden ``Jones Smith`` and no family at all. The closing bracket is a boundary the writer typed, and it is now where the maiden name stops. Two clauses beside each other no longer merge: ``"Jane "Janey" Smith (née Jones)"`` read nickname ``Janey née Jones`` through 2.1 and now reads nickname ``Janey``, maiden ``Jones``, each clause keeping its own role, and this reaches ``HumanName``. What does NOT move is a clause that says nothing about maiden names. A lone marker in brackets is just a word in brackets -- ``"Jane Smith (née)"`` stays nickname ``née``, as 1.4.0 read it -- because a marker with no name after it marks nothing, and because a one-word clause may be the surname it looks like (``Nee`` is Irish Ní/Nee and a Chinese romanization). A markerless clause stays a nickname too: ``"Cherice J. (Johnson) Williams"`` keeps nickname ``Johnson``, which is what ``Policy(maiden_delimiters=...)`` remains for -- the parenthesized birth surname is a real convention, and only a caller who knows the data can say that is what these brackets mean. One accepted consequence: the word after the marker is not tested for being a name word, so a bracketed clause does not stop at a credential where the bare form does -- ``"Jane Smith (née V)"`` reads maiden ``V`` while ``"Jane Smith née V"`` reads suffix ``V``. The clause goes to ``maiden`` whole and the bracket is its boundary, so the trailing-suffix rule never sees the word as trailing the name. One more shape moves with it, and it is the one to know about if you parse names that are a surname and a clause: a name that is ONLY a marker clause and one name word now reads that word as the given name rather than the family. ``"Smith (née Jones)"`` reads given ``Smith``, maiden ``Jones`` where every earlier version read family ``Smith``, nickname ``née Jones``. That is the bracketed form agreeing with the bare one -- ``"Smith née Jones"`` has read given ``Smith`` since 1.4.0 -- because a marker-led clause is not a nickname clause and so does not reach the rule that would have made the word a family name. The empty ``family`` it leaves behind is the bare form's behavior and is not new here, though it is not 1.4.0's either -- v1 read ``"Smith née Jones"`` as first ``Smith``, middle ``née``, last ``Jones``, and #445 tracks whether the family should be kept. Seven of the 1,085 differential corpus names move against the 2.1.0 and 2.0.0 baselines and six against 1.4.0: the Japanese name is the odd one, and against 1.4.0 it is not a new diff at all, since 1.4 read the fullwidth brackets as name text and had no nickname there to change hands. Eight corpus names have the shape and seven move (measured); the eighth is ``"Jane Smith (née Jr.)"``, which the suffix rule takes first. So the count says how few bracketed marker clauses the corpora hold rather than how narrow the change is -- six of the seven are hand-written rule or issue examples, only the Japanese name arriving from elsewhere, and the coverage that bounds the behavior is ``tests/v2/cases.py`` (closes #335) + - Fix a bracketed maiden clause reading as a nickname because its brackets were not declared. ``"Jane Smith nee Jones"`` gave maiden ``Jones``, while ``"Jane Smith (née Jones)"`` gave nickname ``née Jones`` -- the parenthesis pair is a nickname delimiter by default and nothing looked inside it, so the marker word the writer put there was read as part of an alias. This is about every delimiter pair the parser ships, not only the parentheses these examples use: quotes are how a nickname is most often written, and ``"Jane Smith "née Jones""`` and ``"Jane Smith 'née Jones'"`` move from nickname to maiden exactly as the parenthesized spelling does. A bracketed clause whose content opens with a recognized marker word and carries a word after it now reads as the maiden name whichever bucket its pair sits in: ``"Jane Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, and so does the Japanese ``"山田 花子（旧姓 佐藤）"`` (family 山田, given 花子, maiden 佐藤), which needed ``Policy(maiden_delimiters=...)`` through 2.1. The clause is lifted out as a region rather than having its brackets erased, which is what keeps an INTERIOR clause from eating the name behind it: ``"Jane (née Jones) Smith"`` reads given ``Jane``, family ``Smith``, maiden ``Jones``, where erasing the brackets would leave the bare ``"Jane née Jones Smith"``, whose marker takes every word to the end of the name -- maiden ``Jones Smith``, with the real surname inside that value and the given name standing in ``family`` (it read no family at all until #445, below). The closing bracket is a boundary the writer typed, and it is now where the maiden name stops. Two clauses beside each other no longer merge: ``"Jane "Janey" Smith (née Jones)"`` read nickname ``Janey née Jones`` through 2.1 and now reads nickname ``Janey``, maiden ``Jones``, each clause keeping its own role, and this reaches ``HumanName``. What does NOT move is a clause that says nothing about maiden names. A lone marker in brackets is just a word in brackets -- ``"Jane Smith (née)"`` stays nickname ``née``, as 1.4.0 read it -- because a marker with no name after it marks nothing, and because a one-word clause may be the surname it looks like (``Nee`` is Irish Ní/Nee and a Chinese romanization). A markerless clause stays a nickname too: ``"Cherice J. (Johnson) Williams"`` keeps nickname ``Johnson``, which is what ``Policy(maiden_delimiters=...)`` remains for -- the parenthesized birth surname is a real convention, and only a caller who knows the data can say that is what these brackets mean. One accepted consequence: the word after the marker is not tested for being a name word, so a bracketed clause does not stop at a credential where the bare form does -- ``"Jane Smith (née V)"`` reads maiden ``V`` while ``"Jane Smith née V"`` reads suffix ``V``. The clause goes to ``maiden`` whole and the bracket is its boundary, so the trailing-suffix rule never sees the word as trailing the name. One more shape moves with it, and it is the one to know about if you parse names that are a surname and a clause: a name that is ONLY a marker clause and one name word. A marker-led clause is not a nickname clause, so the rule that reads a nickname plus one name word as a surname no longer sees this shape -- and what names the family instead is the maiden marker itself, which is #445 below, in this same release. ``"Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, where 1.4.0, 2.0.0 and 2.1.0 all read family ``Smith``, nickname ``née Jones``: the clause changes hands and the family name stays put. It is also the bracketed form agreeing with the bare ``"Smith née Jones"``, which #445 moves to family ``Smith`` from the given name every version through 2.1 read there. Without that rule this change would have left both spellings with an empty ``family``. Seven of the 1,085 differential corpus names move against the 2.1.0 and 2.0.0 baselines and six against 1.4.0: the Japanese name is the odd one, and against 1.4.0 it is not a new diff at all, since 1.4 read the fullwidth brackets as name text and had no nickname there to change hands. Eight corpus names have the shape and seven move (measured); the eighth is ``"Jane Smith (née Jr.)"``, which the suffix rule takes first. So the count says how few bracketed marker clauses the corpora hold rather than how narrow the change is -- six of the seven are hand-written rule or issue examples, only the Japanese name arriving from elsewhere, and the coverage that bounds the behavior is ``tests/v2/cases.py`` (closes #335) - Fix a particle chain and a maiden name taking a trailing generational numeral as a name word: ``"John van der Berg V"`` read family ``van der Berg V`` and ``"John née Jones Smith V"`` read maiden ``Jones Smith V``, where ``"John Smith V"`` reads suffix ``V``. Both stops asked "is this a suffix?" with the test that vetoes a bare ``V`` as an initial -- the same question the bound given-name join asked until #401 -- and now ask the suffix rule's own walk, over the name as they would leave it: family ``van der Berg``, maiden ``Jones Smith``, suffix ``V``, for ``I`` and ``X`` alike; ``"John van der J. V"`` and ``"J. née Jones Smith V"`` keep their readings, the word before the numeral being an initial as written or as left. The chain stops before a bare credential with words to spare too, as ``"John Smith Ma"`` does, where the suffix rule will still find them once the chain has run: ``"John van der Berg Ma"`` reads suffix ``Ma``, which is how 1.4.0 read it, while ``"Freiherr von Berg MA"`` keeps family ``von Berg MA``; a maiden name keeps such a credential, since the words the marker takes are the very words that made it one. A particle that is also suffix vocabulary now ends the chain where it opens the trailing run: ``"John Smith Mc V"`` reads suffix ``Mc, V``. The chain also no longer swallows the given name behind an unlisted abbreviation: ``"Xyz. van Johnson"`` and ``"Esq. van Gogh"`` read given ``van``, as ``"Dr. van Johnson"`` has since #367, the chain and the bound given-name join now asking the same test that reads the abbreviation as a title (closes #424) @@ -79,10 +79,12 @@ Release Log - Fix a bound given-name join leaving no family name when the name also carries a maiden clause, and stop the join absorbing the marker itself. ``"abdul Berg née Jones"`` read given ``abdul Berg`` with an EMPTY family, where ``"abdul Berg"`` alone correctly reads given ``abdul``, family ``Berg``. The join reserves a name word so it always leaves a family name behind -- two name words alone do not join -- but the reserve was counted while the maiden marker and the maiden name were still part of the name, and the pass that removes them runs afterwards. Four words counted, the join fired, and when the two departed nothing was left for the family. The words a marker will take no longer count toward the reserve, so that name reads given ``abdul``, family ``Berg``, maiden ``Jones``. Excluding them is not sufficient on its own: where a suffix sits INSIDE the name the marker walk stops early, enough words survive to clear the reserve, and the join fired and took the marker -- ``"abd née Jones Jr Smith Berg"`` read given ``abd née``. The join now declines outright when the piece it would absorb is a marker, which is what the rule meant all along: it joins the bound word to a name word, and a marker is not one. A name with a word to spare is untouched -- ``"abd Allah Smith née Jones"`` still joins, giving given ``abd Allah``, family ``Smith``, maiden ``Jones``. This is pre-existing for the spellings that were already vocabulary: ``abdul``, ``abdel`` and ``abdal`` read this way at 2.0.0 and 2.1.0 alike. ``abd`` is different -- it reads CORRECTLY at 2.1.0 and acquired the defect only when this same release made it a bound given-name word, so for that spelling this fixes a regression introduced earlier in the cycle; and the particle spellings reached the reserve for the first time through the maiden-marker chain stop above. Two consequences worth knowing. The bound-given join no longer swallows a maiden marker at all, so ``"van der Berg, abdul née Jones"`` reads given ``abdul``, family ``van der Berg``, maiden ``Jones`` where it read given ``abdul née``, middle ``Jones``; the connective join is the remaining case, tracked at `#412 `_. And where the bound word is ALSO suffix vocabulary, a declining join after a family comma leaves the post-nominal reading and the name has no given name: ``"Berg, abd née Jones"`` reads family ``Berg``, suffix ``abd``, maiden ``Jones``, matching how ``"Berg, abd"`` alone has always parsed. ``abd`` is the only shipped word in both sets (closes #411) - - Fix a maiden clause changing how the rest of the name is read, and a connective join keeping the marker in the surname. A maiden marker and the name it introduces are not part of the name they follow, but the grouping rules that count a name's words -- the three-word test that keeps a single-letter connective a name word, and the bound given-name reserve -- counted them, because the pass that removes them ran after those rules. ``"juan y garcia"`` reads given ``juan``, middle ``y``, family ``garcia``, but ``"juan y garcia nee jones"`` counted five words, joined the ``y``, and read given ``juan y garcia`` with NO family name at all; ``"John e Smith nee jones"``, ``"Lt.Gov. juan e garcia nee jones"`` and six more differential corpus names lose the family the same way under an appended clause. The marker pass now runs before every join, so the joins and the counts see only the name that remains, and a name of two or more name words reads as it reads without its maiden clause, plus the maiden name: ``"juan y garcia nee jones"`` is given ``juan``, middle ``y``, family ``garcia``, maiden ``jones``. (A title plus ONE name word is the #410 shape, fixed separately below: ``"Dr. Jane née Smith"`` reads family ``Jane`` now, as ``"Dr. Jane"`` does.) The same order closes the last of the join-swallows. The connective join used to merge the marker into a multi-word piece before the marker rule could see it, so ``"Jane van der Berg née y Jones"`` kept family ``van der Berg née y Jones`` with no maiden name; it now reads family ``van der Berg``, maiden ``y Jones`` -- the marker takes the words after it, connective included -- and ``"Jane née and Jones Smith"`` reads given ``Jane``, maiden ``and Jones Smith`` where it read middle ``née and Jones``, family ``Smith``. It also retires the particle chain's marker stop from the #399 fix above, whose condition restated the marker rule's and disagreed with it one suffix later (``"Jane van der Berg née Jr Jones"``, #417): a marker the rule declines -- nothing after it but a suffix -- is an ordinary word and rides inside the chain, which is the reading ``"Jane van der Berg née"`` already had, and the parsed fields of that name do not change. Two limits. A bound given-name word still never joins onto a marker standing as a word of its own, so ``"Berg, abdul née PhD"`` keeps given ``abdul``; a declined marker the particle chain has taken travels with the chain, so ``"Abd van der Berg née Jr Jones"`` reads given ``Abd van der Berg née`` where it read given ``Abd van der Berg``, middle ``née Jr``. And the marker rule now sees the words as written rather than as joined, so a suffix-vocabulary word inside the maiden name stops it even with a connective beside it: ``"Jane née Jr y Jones"`` read maiden ``Jr y Jones`` and now reads family ``Jr y Jones`` with no maiden name, and ``"Jane Smith née Jones Jr y Smith"`` read maiden ``Jones Jr y Smith`` and now reads maiden ``Jones``, family ``Jr y Smith``. One consequence: ``"abd née Jones Jr Smith Berg"`` reads given ``abd Jr`` -- as ``"abd Jr Smith Berg"`` already does -- where the #411 fix above read given ``abd``. Four of the 1025 differential corpus names move. The two connective-join names move at the 2.0.0 and 2.1.0 baselines alike; at 1.4.0 both were already classified as maiden-marker changes, 1.4 having no maiden field. ``"Juan y Garcia née Jones"`` and ``"Jane née Jr y Jones"``, the examples this fix adds to the rules doc and so to the rules corpus, move at all three and at 2.0.0/2.1.0 respectively. The eight corpus names that lose a family only under an APPENDED clause are not in the corpora in that form, so the gate cannot see them; the case table pins that half (closes #412, closes #417, closes #418) + - Fix a maiden clause changing how the rest of the name is read, and a connective join keeping the marker in the surname. A maiden marker and the name it introduces are not part of the name they follow, but the grouping rules that count a name's words -- the three-word test that keeps a single-letter connective a name word, and the bound given-name reserve -- counted them, because the pass that removes them ran after those rules. ``"juan y garcia"`` reads given ``juan``, middle ``y``, family ``garcia``, but ``"juan y garcia nee jones"`` counted five words, joined the ``y``, and read given ``juan y garcia`` with NO family name at all; ``"John e Smith nee jones"``, ``"Lt.Gov. juan e garcia nee jones"`` and six more differential corpus names lose the family the same way under an appended clause. The marker pass now runs before every join, so the joins and the counts see only the name that remains, and a name of two or more name words reads as it reads without its maiden clause, plus the maiden name: ``"juan y garcia nee jones"`` is given ``juan``, middle ``y``, family ``garcia``, maiden ``jones``. (A title plus ONE name word is the #410 shape, fixed separately below: ``"Dr. Jane née Smith"`` reads family ``Jane`` now, as ``"Dr. Jane"`` does.) The same order closes the last of the join-swallows. The connective join used to merge the marker into a multi-word piece before the marker rule could see it, so ``"Jane van der Berg née y Jones"`` kept family ``van der Berg née y Jones`` with no maiden name; it now reads family ``van der Berg``, maiden ``y Jones`` -- the marker takes the words after it, connective included -- and ``"Jane née and Jones Smith"`` reads maiden ``and Jones Smith`` where it read middle ``née and Jones``, family ``Smith`` -- the one name word the marker leaves goes to ``family`` since #445 below, and read as given ``Jane`` until it did. It also retires the particle chain's marker stop from the #399 fix above, whose condition restated the marker rule's and disagreed with it one suffix later (``"Jane van der Berg née Jr Jones"``, #417): a marker the rule declines -- nothing after it but a suffix -- is an ordinary word and rides inside the chain, which is the reading ``"Jane van der Berg née"`` already had, and the parsed fields of that name do not change. Two limits. A bound given-name word still never joins onto a marker standing as a word of its own, so ``"Berg, abdul née PhD"`` keeps given ``abdul``; a declined marker the particle chain has taken travels with the chain, so ``"Abd van der Berg née Jr Jones"`` reads given ``Abd van der Berg née`` where it read given ``Abd van der Berg``, middle ``née Jr``. And the marker rule now sees the words as written rather than as joined, so a suffix-vocabulary word inside the maiden name stops it even with a connective beside it: ``"Jane née Jr y Jones"`` read maiden ``Jr y Jones`` and now reads family ``Jr y Jones`` with no maiden name, and ``"Jane Smith née Jones Jr y Smith"`` read maiden ``Jones Jr y Smith`` and now reads maiden ``Jones``, family ``Jr y Smith``. One consequence: ``"abd née Jones Jr Smith Berg"`` reads given ``abd Jr`` -- as ``"abd Jr Smith Berg"`` already does -- where the #411 fix above read given ``abd``. Four of the 1025 differential corpus names move. The two connective-join names move at the 2.0.0 and 2.1.0 baselines alike; at 1.4.0 both were already classified as maiden-marker changes, 1.4 having no maiden field. ``"Juan y Garcia née Jones"`` and ``"Jane née Jr y Jones"``, the examples this fix adds to the rules doc and so to the rules corpus, move at all three and at 2.0.0/2.1.0 respectively. The eight corpus names that lose a family only under an APPENDED clause are not in the corpora in that form, so the gate cannot see them; the case table pins that half (closes #412, closes #417, closes #418) - Fix a title-plus-surname name losing its family name whenever anything stood beside it. ``"Dr. Smith"`` reads family ``Smith``, but ``"Dr. Smith née Jones"`` read given ``Smith`` with no family at all, and so did ``"Dr. Smith PhD"`` and ``"Dr. "Smitty" Smith"``. A title followed by a single name word names the family, and that rule declined whenever the name also carried a suffix, a nickname or a maiden name -- it counted them as further name words, which none of them is: each stands beside the name rather than in it, and none makes the name any longer. The rule now counts name words alone. ``"Dr. Smith née Jones"`` reads family ``Smith``, maiden ``Jones``; ``"Dr. Smith PhD"`` family ``Smith``, suffix ``PhD``; ``"Dr. "Smitty" Smith"`` family ``Smith``, nickname ``Smitty``; and the particle spelling this was found through, ``"Freiherr von Richthofen geb. Albrecht"``, reads family ``von Richthofen``, maiden ``Albrecht`` -- 2.2's own particle-chain stop had routed that canonical shape into the rule for the first time (#399). A given-name title is unchanged and still names no family: ``"Sir John née Jones"`` keeps given ``John`` with an empty family, exactly as ``"Sir John"`` does. One name moves where the nickname LEADS, rather than standing beside a name that already reads: ``"'Smitty' Dr. Jones"``. The lone-word nickname rule declines there because the title counts against its piece count, which leaves a title and one name word for this fix to read as the family -- given ``Jones`` through 2.1, family ``Jones`` now. Five names of the differential corpora change reading, the same five at every baseline. Four of them arrive as new diffs and are classified with this fix: ``"Dr. Smith née Jones"``, ``"Senator "Rick" Edmonds"``, ``"Xyz. (Bud) Smith"`` and ``"'Smitty' Dr. Jones"``. The fifth, ``"Freiherr von Richthofen V"``, was already classified under 2.2's particle-chain fix and stays there -- it is the suffix flavor in corpus form, and reads family ``von Richthofen`` with suffix ``V`` where it read given ``von Richthofen`` before. v1 read the two nicknames as 2.1 did; it had no maiden support at all, reading ``"Dr. Smith née Jones"`` as first ``Smith``, middle ``née``, last ``Jones``. The v1 test suite shipped the correct reading for the nickname shape as a known-failing test, which now passes (closes #410) + - Fix a name that is a surname and a maiden clause reporting no family at all. ``"Smith née Jones"`` read given ``Smith`` with an empty ``family``; it now reads family ``Smith``, maiden ``Jones``. A maiden marker announces a FORMER surname, and a former surname only means something beside a current one -- nobody writes a maiden clause where there is no surname for it to set apart -- so where the marker has taken its name and a single name word is left standing, that word is the surname the bearer uses now. With one name word and nothing else to go on the library has to pick a field and picks ``given``, which is a convention rather than a determination and is now written down as such; a maiden marker joins the title and the nickname as one of the things that decides such a name instead. Every spelling of the shape moves, wherever the clause stands and however it is written -- a marker taking the words after it, a marker inside a bracket, or a bracket pair you declared to mean maiden, which carries no marker at all: under ``Policy(maiden_delimiters=frozenset({("(", ")")}))``, ``"Smith (Jones)"`` reads family ``Smith``, maiden ``Jones``. ``"Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, which RESTORES 1.4.0 -- v1 read family ``Smith`` with the clause as a nickname, and only the clause changes hands now. ``"Jane née Jones Smith"``, where the marker stands INSIDE the name and takes the rest of it, reads family ``Jane``, maiden ``Jones Smith``, which no earlier version read (1.4.0 gave first ``Jane``, middle ``née Jones``, last ``Smith``): the rule counts what the clause LEAVES rather than where the marker stood. An annotation beside the name is no part of it, so ``"Smith née Jones PhD"`` reads family ``Smith``, suffix ``PhD``, and ``"'Smitty' Jones Jr. née Smith"`` reads family ``Jones``. Two shapes deliberately do NOT move, and both witnesses are names the differential corpora already carried rather than cases constructed for the release: a word the vocabulary claims as a given name keeps that reading, so ``"abd née Jones"`` still reads given ``abd``, and so does a word written as an initial, so ``"J. née Jones Smith V"`` still reads given ``J.`` -- this rule changes what POSITION decided and does not reach what a word already is. A title still decides the name behind it: ``"Dr. Smith née Jones"`` reads family ``Smith`` as it does since the title fix above, and the given-name title in ``"Sir John née Jones"`` still keeps given ``John`` with no family. A name with two name words beside the clause is untouched -- ``"Jane Smith née Jones"`` reads given ``Jane``, family ``Smith``, maiden ``Jones`` as before. If you have code that reads the lone name word beside a maiden clause out of ``given``, this is the release where it moves to ``family``. Nine of the 1,089 differential corpus names change reading: eight arrive as new diffs at every baseline and are classified with this fix, and the ninth is ``"Smith (née Jones)"``, whose family agrees with 1.4.0 again, so its diff against that baseline SHRANK to the clause alone (closes #445) + **Deprecations** - Rename the four vocabularies whose 1.x names described the fields they feed in v1's words, so the data layer matches the ``Lexicon``: From e274ca05c0dc38ddd2da9d339590f13b81b54a85 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 27 Aug 2026 18:39:32 -0700 Subject: [PATCH 3/3] test(differential,docs): the absorber the revisit note missed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Smith (née Jones)` reads family 'Smith', nickname 'née Jones' on 1.4.0, 2.0.0 and 2.1.0 alike -- measured on all three wheels. Only the BARE spelling ever read `given`. The first pass narrowed the 1.4.0 rule on a measurement of that wheel and left the 2.x copies at ["given", "family", "nickname", "maiden"] on the reasoning that those baselines "read `given`, and the baseline cannot move". They did not, so two rules stood over a {nickname, maiden} diff declaring four fields, and classify's subset test would have explained a `given`/`family` regression on that name in silence -- the absorption those very `# revisit when #445 lands` notes existed to prevent, surviving the round that answered them. Both are narrowed to ["nickname", "maiden"], the matching _CORPUS_CLAIMS role tuples with them, and both passages rewritten to say what is true. Verified after narrowing: declared fields now equal the real diff at all three baselines, the name is still explained, and a {given, family} diff on it is UNEXPLAINED at 2.0.0 and 2.1.0. At 1.4.0 it is not, because the fields-only fix(suffix-routing) catch-all matches every name with {given, family, suffix}; that is long-standing and out of scope, and the ledger now says so rather than implying the narrowing closed the hole everywhere. The seed was cases.py's note, whose "Was given 'Smith' on 1.4.0's bare spelling ..., on 2.0.0, on 2.1.0 and here" reads correctly about the bare spelling and falsely about the row it sits in. Disambiguated, and it now records that a ledger took the loose reading. Also corrected, all measured: - decisions.md's pattern bullet, which the previous commit's message described but whose edit never landed -- a script asserted two replacements, failed the second and wrote neither. The text now says what the message said: the three repairs share a symptom, not a cause; #445 changes no count; and `_name_positions`'s `count == 1` branch is the site the recommended grep would miss. - decisions.md: THREE of the four markers moved opposite to their own prediction, not one, with how the miss happened. - decisions.md: the bound-given mutation list was one short (six, not five -- O5's own `abd née Jones` line fails too). - release_log: the corpus is 1,090, not 1,089; the ninth name's diff shrank at all three baselines, not just 1.4.0; and the #434 bullet's split-entry argument said `"Anna z Nowak"` has no family at all, where it now reads family `Anna` (the argument survives, the symptom did not). - test_parser: the recount was itself one low at every stage, in a sentence criticising an earlier author for not recounting. Pasting the one-liner gives 646/636/632. #445 adds five rules-corpus rows, three of which clear the marker predicate, so every stage moved by three. - expected_since_1.4.0: `J. née Jones Smith V` is explained by fix(#424) accepted, not fix(#274) -- its diff carries `suffix`, which #274's three fields cannot hold. Asked of classify directly. The safety argument holds through that rule, which declares no `given` either. - expected_since_1.4.0: "Four claimed names still do not move" measures five, re-measured name by name against the wheel over all 31 the regex claims. - test_ledger_guards: the roster comment's arithmetic (six plus two where the set holds seven), rewritten from the measured sets. - Three over-long comment lines in the new blocks, re-wrapped. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 6 +- docs/release_log.rst | 4 +- tests/v2/cases.py | 21 ++++--- tests/v2/test_ledger_guards.py | 17 ++++-- tests/v2/test_parser.py | 16 +++-- tools/differential/expected_since_1.4.0.toml | 63 ++++++++++++++------ tools/differential/expected_since_2.0.0.toml | 44 ++++++++------ tools/differential/expected_since_2.1.0.toml | 44 ++++++++------ 8 files changed, 138 insertions(+), 77 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index ec57342a..ccc115f1 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -538,12 +538,12 @@ Declined: ### M4 — a maiden marker makes the lone name word the family - 2026-08-27 #445 (M4, PR on fix/445-maiden-marked-lone-name) — Derek's rationale, in his terms: normally if there is only one name we assume it is a given name because we have to pick one, but maiden markers mark a previous surname before marriage, so it would not make sense to include one if there was no other surname for it to clarify — and given that, when a marker leaves only one name word we switch the assumption and read that word as the family name. `parse("Smith née Jones")` reported given 'Smith' with an EMPTY family through 2.1 and reads family 'Smith', maiden 'Jones' since. The rule is written in post_rules as H1's sibling and placed directly under it, so the interaction is decidable by reading rather than by running: where H1 fired there is no GIVEN left for M4 to move, and where H1 declined because the title addresses by given name ("Sir John née Jones") the `not titles` guard keeps M4 off the same word. A titled name is H1's at both outcomes, which is what keeps H1's given-name-title carve-out intact. -- 2026-08-27 #445 — this is the THIRD instance of one shape repaired in the 2.2 bundle, and the pattern is worth stating because a fourth site would be findable by looking for it rather than by waiting for a report. #410 (H1): a title plus one name word reported no family the moment a suffix, nickname or maiden name stood beside it. #411 (P5's reserve): the bound given-name join counted the marker and the maiden name as words it could spend, and spent the family name. #445 (the positional convention): the lone name word beside a maiden clause stayed `given` and the family stayed empty. Every one is an ANNOTATION beside the name being counted as part of the name, and every one showed as an empty `family` on input that plainly names a family. The shape to grep for is a rule that counts name words without first asking which of them are name words. +- 2026-08-27 #445 — the third time in the 2.2 bundle that a name carrying a maiden clause reported no family, and the three share a SYMPTOM rather than a cause. State that carefully, because the first draft of this entry claimed one cause for all three and was wrong. #410 (H1): a title plus one name word reported no family the moment a suffix, nickname or maiden name stood beside it. #411 (P5's reserve): the bound given-name join counted the marker and the maiden name as words it could spend, and spent the family name. Those two ARE one cause -- a count that included words which are no part of the name -- and the grep that would find a fourth is a rule counting name words without first asking which of them are name words. #445 is not that: it changes no count anywhere (post_rules gains a new block, assign gains only a comment), and `parse("Smith")` with no clause beside it at all already read given 'Smith' with an empty family, so nothing was being swallowed. Its cause is the one the next entry gives: O4 decides nothing at one name word, and the reading came from a convention nobody had written down. The site that grep would miss is `_name_positions`'s `count == 1` branch, silent for every order and every name -- which is exactly what O5 now records. - 2026-08-27 #445 — O4 was SILENT at exactly one name word, which is why the shipped reading had no rule to point at and why O5 now exists. O4 reads a name by comparing where its words stand ("the first name word is the given name, the last is the family name"); with one word the first IS the last, so there is nothing to compare and the rule decides nothing. O5 records the reading as the convention it is — a guess fixed in advance so that the same input always reads the same way, not a determination about the word — and names H1, N3 and M4 as the rules that DO decide such a name. Written that way deliberately: a rule asserting "a lone name word is the given name" as a fact would have made #445 a contradiction of the documented behavior instead of an exception to a documented guess, and would make #449 one too. -- 2026-08-27 #445 — the two carve-outs are not inventions, and both rest on mechanisms.md#TWO-LAYER-ASSIGN: the positional layer never overrides a vocabulary claim, and M4 changes only what POSITION decided. A word the vocabulary claims as a given name keeps that reading ('abd née Jones' — `vocab:bound-given`), and so does a word written as an initial ('J. née Jones Smith V' — the `initial` shape tag). Both are read off tags classify already recorded rather than off a predicate of M4's own, and both witnesses are real corpus names rather than constructed cases, so each carve-out has something in the differential that would notice its loss. Mutation-checked on a scratch copy: dropping the `initial` carve-out fails that name's case row, its facade twin, M4's own boundary example and the pre-existing assertion in test_the_chain_and_the_walk_stop_where_the_peel_begins; dropping `vocab:bound-given` fails the other name's two runners, M4's boundary, P5's own doc example and the corpus-wide maiden-clause property on 'abdul'. +- 2026-08-27 #445 — the two carve-outs are not inventions, and both rest on mechanisms.md#TWO-LAYER-ASSIGN: the positional layer never overrides a vocabulary claim, and M4 changes only what POSITION decided. A word the vocabulary claims as a given name keeps that reading ('abd née Jones' — `vocab:bound-given`), and so does a word written as an initial ('J. née Jones Smith V' — the `initial` shape tag). Both are read off tags classify already recorded rather than off a predicate of M4's own, and both witnesses are real corpus names rather than constructed cases, so each carve-out has something in the differential that would notice its loss. Mutation-checked on a scratch copy: dropping the `initial` carve-out fails that name's case row, its facade twin, M4's own boundary example and the pre-existing assertion in test_the_chain_and_the_walk_stop_where_the_peel_begins; dropping `vocab:bound-given` fails the other name's two runners, M4's boundary, P5's own doc example, O5's own `abd née Jones` line and the corpus-wide maiden-clause property on 'abdul' -- six in all, the O5 line having been added by the review round, and the list is exhaustive as re-run against the final tree. - 2026-08-27 #445 — the two halves of Derek's answer differ in their relationship to 1.4.0, and a reader should not have to re-derive it. The bracketed spelling RESTORES v1: `Smith (née Jones)` read family 'Smith', nickname 'née Jones' on 1.4.0, 2.0.0 and 2.1.0 alike, and now reads family 'Smith', maiden 'Jones' — the clause changes hands and the family name stays put, so the 1.4.0 gate's diff on that name SHRANK to {nickname, maiden}. The interior spelling is a NEW reading: `Jane née Jones Smith` read first 'Jane', middle 'née Jones', last 'Smith' on 1.4.0 and reads family 'Jane', maiden 'Jones Smith' now, the real surname being inside the maiden value by M2's greedy take. That the marker stands inside the name changes nothing for M4, which counts what the take LEAVES rather than where the marker stood — one name word left that way is one name word, which is the widest half of the decision and the half no earlier version agrees with. - 2026-08-27 #445 — a precedence claim corrected while drafting, and it is exactly the class this bundle keeps producing. The first draft of M4 said a name carrying a nickname is N3's rather than this rule's. Measured false: `'Smitty' Jones Jr. née Smith` reads family 'Jones', because N3's count does not set a suffix aside, so N3 declines and M4 fires. A nickname holds nothing off — where N3 has already named the family M4 finds nothing left to move, and where N3 declined M4 names it. The general lesson is the one #410 and #411 already taught from the other side: a rule that counts name words and a rule that counts something else will disagree at the edges, and the disagreement is only visible if the interaction is measured rather than asserted. -- 2026-08-27 #445 — what the ledger's `# revisit when #445 lands` markers bought, and what they got wrong. Four blocks carried them, and each predicted that a "keep the family" fix would leave the diff a subset of the fields its rule already declared, so the rule would go on explaining the name and the change would be absorbed with the gate green. That was wrong in a way the rule's own shape settles: M4 MOVES the one name word rather than adding one, so `given` empties as `family` fills, the diff outgrows every declaration, and eight names arrived UNEXPLAINED at all three baselines. Two counts run through this and they are different sets, so name which is which: NINE corpus names change reading, of which eight arrive as new diffs — the ninth, 'Smith (née Jones)', has its 1.4.0 diff SHRINK instead, its family agreeing with v1 again. The four markers between them named NINE names to re-measure, of which seven moved: the two that did not are M4's carve-outs, and the two movers the markers could not name were not corpus names until this change's own examples added them. The markers earned their keep all the same — they named the right rule, the right field and very nearly the right set — so they are corrected in place rather than deleted, and the correction says what actually happened. One of the four moved in the OPPOSITE direction from its own prediction: the fix(#335) rule for 'Smith (née Jones)' at 1.4.0 feared absorbing a widened diff and instead had its diff shrink to {nickname, maiden}, so that rule is narrowed to the two fields that still move. +- 2026-08-27 #445 — what the ledger's `# revisit when #445 lands` markers bought, and what they got wrong. Four blocks carried them, and each predicted that a "keep the family" fix would leave the diff a subset of the fields its rule already declared, so the rule would go on explaining the name and the change would be absorbed with the gate green. That was wrong in a way the rule's own shape settles: M4 MOVES the one name word rather than adding one, so `given` empties as `family` fills, the diff outgrows every declaration, and eight names arrived UNEXPLAINED at all three baselines. Two counts run through this and they are different sets, so name which is which: NINE corpus names change reading, of which eight arrive as new diffs — the ninth, 'Smith (née Jones)', has its 1.4.0 diff SHRINK instead, its family agreeing with v1 again. The four markers between them named NINE names to re-measure, of which seven moved: the two that did not are M4's carve-outs, and the two movers the markers could not name were not corpus names until this change's own examples added them. The markers earned their keep all the same — they named the right rule, the right field and very nearly the right set — so they are corrected in place rather than deleted, and the correction says what actually happened. THREE of the four moved in the opposite direction from their own prediction, and getting that number right took two passes. The fix(#335) rule for 'Smith (née Jones)' appears in all three ledgers, and every copy feared absorbing a WIDENED diff while the diff in fact SHRANK to {nickname, maiden}. The first pass narrowed only the 1.4.0 copy, on a measurement of that wheel, and left the other two at four fields on the reasoning that 2.0.0 and 2.1.0 'read `given`, and the baseline cannot move'. They did not: only the BARE spelling ever read `given`, and this rule holds the bracketed one, which reads family 'Smith', nickname 'née Jones' on every released version. So two rules stood over a two-field diff declaring four, and classify's subset test would have explained a `given`/`family` regression on that name in silence -- the absorption those very notes existed to prevent, surviving the round that answered them. A third reviewer found it by instrumenting the comparator to dump real diff-field sets, and that is the lesson worth keeping: ledger prose cannot be checked by reading it, and the round that corrects a marker is exactly where a wrong premise gets written down with confidence. - 2026-08-27 #445 — the corpus-wide property test is the repair worth copying. tests/v2/test_parser.py's `test_a_maiden_clause_changes_nothing_else` asserts over the corpus that appending " née Jones" adds a maiden name and moves no other field, and M4 falsifies it for fourteen names. Skipping them would have bought a green suite and lost the check; instead the test computes M4's guard from the base parse's tokens and asserts the flip — family takes what `given` held, every other field standing still — so the class that used to be a hole is now the strongest witness the rule has, fourteen names against the six rows cases.py carries. It is also what executes M4's nickname precedence, `'Smitty' Jones Jr.` being one of the fourteen. - 2026-08-27 #445 (review round) — M4 is keyed on the maiden NAME, not on a marker, and the statement was corrected to say so after two reviewers measured the same defect independently. The guard tests for a token in the MAIDEN role, and M1's caller-configured pair produces one with no marker anywhere: under `Policy(maiden_delimiters=frozenset({("(", ")")}))`, `Smith (Jones)` reads family 'Smith', maiden 'Jones', and so do M1's own boundary examples `Smith (Nee)` and `Smith (z domu)`. The BEHAVIOUR is right — the rationale transfers to a declared pair without a word of change, since what announces a former surname is the clause, not the vocabulary that marked it — so the fix was to the rule, which had said "a recognized maiden marker that takes its name and leaves exactly one name word" and now says "a maiden name standing beside exactly one name word". That covers M1, M2 and M3 uniformly, M1 is in M4's `interacts:` and M4 in M1's, and the configured path has an example line of its own. The general lesson is the one this bundle keeps teaching from a new angle: a statement written from the case that prompted the change describes that case, not the code, and only measuring the OTHER paths into the same guard finds the difference. - 2026-08-27 #445 (review round) — a known gap recorded rather than closed, the way H1 carries its own. M4's guard counts GIVEN tokens; H1 counts nothing at all (it tests which roles are unoccupied). So a name word another rule has joined counts as several here and M4 declines where H1 fires: `Dr. Dean of Chemistry` reads family 'Dean of Chemistry' while `Dean of Chemistry née Jones` keeps given 'Dean of Chemistry', the connective join (P3) having left three GIVEN tokens. rules.md#P3 says the joined part is one name word wherever another rule counts them, so the two siblings genuinely disagree and this is a gap rather than a boundary. Not closed here: widening the count to units moves zero corpus names (measured), and a behaviour change nobody has approved does not belong in a branch whose blast radius was settled. The sibling claim in the code comment and in this entry is narrowed to match — sibling except in what it counts. diff --git a/docs/release_log.rst b/docs/release_log.rst index 73a8c4e0..da67c853 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -47,7 +47,7 @@ Release Log - Remove the Czech/Slovak abbreviation ``roz.`` from the default maiden markers, where it had been eating English given names. Marker matching is whole-token, case-folded and period-insensitive, so ``Roz`` -- the ordinary diminutive of Rosalind -- was the same string as the marker, and a maiden marker takes every word after it: ``"Rosalind Roz Smith"`` read maiden ``Smith`` with **no family name at all**, and ``"Rosalind Roz Jones Smith"`` read maiden ``Jones Smith``. Both now read the way 1.4.0 read them, given ``Rosalind`` with ``Roz`` a middle name and ``Smith`` the family. The defect is as old as the marker vocabulary and belongs to the bare marker rule, not to the bracketed-clause change in the bullet below; it was found while reviewing that change, which would have widened it -- with ``roz`` still shipped, ``"Jane Smith (Roz Jones)"`` reads maiden ``Jones`` where 2.1.0 read nickname ``Roz Jones``. What stops working is the abbreviation itself, for Czech and Slovak callers: ``"Anna Nováková roz. Svobodová"`` now reads middle ``Nováková roz.``, family ``Svobodová`` rather than family ``Nováková``, maiden ``Svobodová`` -- which is also how 1.4.0 read it, so this is a return to the v1 reading rather than a new one. The full participle is untouched and needs no configuration: ``"Anna Nováková rozená Svobodová"`` still reads family ``Nováková``, maiden ``Svobodová``, and it is only the abbreviation that was ambiguous. A caller who wants the abbreviation back adds it to their own lexicon -- ``Parser(lexicon=dataclasses.replace(lex, maiden_markers=lex.maiden_markers | {"roz"}))`` -- which restores the old reading exactly. This removal takes the default set to 16 entries. No differential corpus name contains the string in any case or position, so nothing moves against any baseline (closes the collision found in #335's review) - - Add the Polish maiden marker ``z domu`` to the default vocabulary, and let a ``maiden_markers`` entry be more than one word. ``"Maria Kowalska z domu Nowak"`` now reads family ``Kowalska``, maiden ``Nowak``, where every earlier version read the marker as part of the name (1.4.0: middle ``Kowalska z domu``, family ``Nowak``). The bracketed spelling moves with it -- ``"Maria Kowalska (z domu Nowak)"`` read nickname ``z domu Nowak`` and now reads family ``Kowalska``, maiden ``Nowak`` -- while a clause holding nothing but the marker stays a nickname, as a lone ``(née)`` does: ``"Maria Kowalska (z domu)"`` keeps nickname ``z domu``. **This changes what a caller who worked around the old limit should have configured.** Through 2.1 a multi-word entry in any vocabulary field could never match, and the ``UserWarning`` it raised said to split it into separate entries. For maiden markers that advice was wrong, not merely unhelpful: with ``z`` and ``domu`` stored as two entries, ``"Maria Kowalska z domu Nowak"`` reads maiden ``domu Nowak`` -- the marker's own second word inside the value -- and ``"Anna z Nowak"`` reads maiden ``Nowak`` with **no family name at all**, because a bare preposition is then a marker and a marker takes every word after it. If you added ``z`` and ``domu`` separately, remove both: the shipped phrase does the job, and dropping the two word entries is what restores ``"Anna z Nowak"`` to given ``Anna``, middle ``z``, family ``Nowak`` -- which is how it reads on the default vocabulary, unchanged from 1.4.0. The same applies to any other field-splitting workaround for this one field. A phrase is matched over whole tokens that stand together -- a bracketed clause or a comma between the words ends the run, so ``"Anna z (domu) Nowak"`` keeps its family name too -- and longest first, so a phrase and a word entry that starts it can both be configured: with ``geb`` shipped and ``geb von`` added, ``"Jane Smith geb von Braun"`` reads maiden ``Braun`` and ``"Jane Smith geb Braun"`` still reads maiden ``Braun``. ``given_name_titles`` was the only field exempt from the multi-word warning; ``maiden_markers`` is now the second, and the warning still fires for every other field. This addition takes the default marker set to 17 entries. Two differential corpus names move, at all three baselines, and both are examples this change adds to the rules doc; no corpus name held ``domu`` or a bare ``z`` before it (#434) + - Add the Polish maiden marker ``z domu`` to the default vocabulary, and let a ``maiden_markers`` entry be more than one word. ``"Maria Kowalska z domu Nowak"`` now reads family ``Kowalska``, maiden ``Nowak``, where every earlier version read the marker as part of the name (1.4.0: middle ``Kowalska z domu``, family ``Nowak``). The bracketed spelling moves with it -- ``"Maria Kowalska (z domu Nowak)"`` read nickname ``z domu Nowak`` and now reads family ``Kowalska``, maiden ``Nowak`` -- while a clause holding nothing but the marker stays a nickname, as a lone ``(née)`` does: ``"Maria Kowalska (z domu)"`` keeps nickname ``z domu``. **This changes what a caller who worked around the old limit should have configured.** Through 2.1 a multi-word entry in any vocabulary field could never match, and the ``UserWarning`` it raised said to split it into separate entries. For maiden markers that advice was wrong, not merely unhelpful: with ``z`` and ``domu`` stored as two entries, ``"Maria Kowalska z domu Nowak"`` reads maiden ``domu Nowak`` -- the marker's own second word inside the value -- and ``"Anna z Nowak"`` reads maiden ``Nowak`` and loses its surname, because a bare preposition is then a marker and a marker takes every word after it -- family ``Anna`` today, the given name standing in for the surname the clause ate, and no family at all before #445 below. If you added ``z`` and ``domu`` separately, remove both: the shipped phrase does the job, and dropping the two word entries is what restores ``"Anna z Nowak"`` to given ``Anna``, middle ``z``, family ``Nowak`` -- which is how it reads on the default vocabulary, unchanged from 1.4.0. The same applies to any other field-splitting workaround for this one field. A phrase is matched over whole tokens that stand together -- a bracketed clause or a comma between the words ends the run, so ``"Anna z (domu) Nowak"`` keeps its family name too -- and longest first, so a phrase and a word entry that starts it can both be configured: with ``geb`` shipped and ``geb von`` added, ``"Jane Smith geb von Braun"`` reads maiden ``Braun`` and ``"Jane Smith geb Braun"`` still reads maiden ``Braun``. ``given_name_titles`` was the only field exempt from the multi-word warning; ``maiden_markers`` is now the second, and the warning still fires for every other field. This addition takes the default marker set to 17 entries. Two differential corpus names move, at all three baselines, and both are examples this change adds to the rules doc; no corpus name held ``domu`` or a bare ``z`` before it (#434) - Fix a bracketed maiden clause reading as a nickname because its brackets were not declared. ``"Jane Smith nee Jones"`` gave maiden ``Jones``, while ``"Jane Smith (née Jones)"`` gave nickname ``née Jones`` -- the parenthesis pair is a nickname delimiter by default and nothing looked inside it, so the marker word the writer put there was read as part of an alias. This is about every delimiter pair the parser ships, not only the parentheses these examples use: quotes are how a nickname is most often written, and ``"Jane Smith "née Jones""`` and ``"Jane Smith 'née Jones'"`` move from nickname to maiden exactly as the parenthesized spelling does. A bracketed clause whose content opens with a recognized marker word and carries a word after it now reads as the maiden name whichever bucket its pair sits in: ``"Jane Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, and so does the Japanese ``"山田 花子（旧姓 佐藤）"`` (family 山田, given 花子, maiden 佐藤), which needed ``Policy(maiden_delimiters=...)`` through 2.1. The clause is lifted out as a region rather than having its brackets erased, which is what keeps an INTERIOR clause from eating the name behind it: ``"Jane (née Jones) Smith"`` reads given ``Jane``, family ``Smith``, maiden ``Jones``, where erasing the brackets would leave the bare ``"Jane née Jones Smith"``, whose marker takes every word to the end of the name -- maiden ``Jones Smith``, with the real surname inside that value and the given name standing in ``family`` (it read no family at all until #445, below). The closing bracket is a boundary the writer typed, and it is now where the maiden name stops. Two clauses beside each other no longer merge: ``"Jane "Janey" Smith (née Jones)"`` read nickname ``Janey née Jones`` through 2.1 and now reads nickname ``Janey``, maiden ``Jones``, each clause keeping its own role, and this reaches ``HumanName``. What does NOT move is a clause that says nothing about maiden names. A lone marker in brackets is just a word in brackets -- ``"Jane Smith (née)"`` stays nickname ``née``, as 1.4.0 read it -- because a marker with no name after it marks nothing, and because a one-word clause may be the surname it looks like (``Nee`` is Irish Ní/Nee and a Chinese romanization). A markerless clause stays a nickname too: ``"Cherice J. (Johnson) Williams"`` keeps nickname ``Johnson``, which is what ``Policy(maiden_delimiters=...)`` remains for -- the parenthesized birth surname is a real convention, and only a caller who knows the data can say that is what these brackets mean. One accepted consequence: the word after the marker is not tested for being a name word, so a bracketed clause does not stop at a credential where the bare form does -- ``"Jane Smith (née V)"`` reads maiden ``V`` while ``"Jane Smith née V"`` reads suffix ``V``. The clause goes to ``maiden`` whole and the bracket is its boundary, so the trailing-suffix rule never sees the word as trailing the name. One more shape moves with it, and it is the one to know about if you parse names that are a surname and a clause: a name that is ONLY a marker clause and one name word. A marker-led clause is not a nickname clause, so the rule that reads a nickname plus one name word as a surname no longer sees this shape -- and what names the family instead is the maiden marker itself, which is #445 below, in this same release. ``"Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, where 1.4.0, 2.0.0 and 2.1.0 all read family ``Smith``, nickname ``née Jones``: the clause changes hands and the family name stays put. It is also the bracketed form agreeing with the bare ``"Smith née Jones"``, which #445 moves to family ``Smith`` from the given name every version through 2.1 read there. Without that rule this change would have left both spellings with an empty ``family``. Seven of the 1,085 differential corpus names move against the 2.1.0 and 2.0.0 baselines and six against 1.4.0: the Japanese name is the odd one, and against 1.4.0 it is not a new diff at all, since 1.4 read the fullwidth brackets as name text and had no nickname there to change hands. Eight corpus names have the shape and seven move (measured); the eighth is ``"Jane Smith (née Jr.)"``, which the suffix rule takes first. So the count says how few bracketed marker clauses the corpora hold rather than how narrow the change is -- six of the seven are hand-written rule or issue examples, only the Japanese name arriving from elsewhere, and the coverage that bounds the behavior is ``tests/v2/cases.py`` (closes #335) @@ -83,7 +83,7 @@ Release Log - Fix a title-plus-surname name losing its family name whenever anything stood beside it. ``"Dr. Smith"`` reads family ``Smith``, but ``"Dr. Smith née Jones"`` read given ``Smith`` with no family at all, and so did ``"Dr. Smith PhD"`` and ``"Dr. "Smitty" Smith"``. A title followed by a single name word names the family, and that rule declined whenever the name also carried a suffix, a nickname or a maiden name -- it counted them as further name words, which none of them is: each stands beside the name rather than in it, and none makes the name any longer. The rule now counts name words alone. ``"Dr. Smith née Jones"`` reads family ``Smith``, maiden ``Jones``; ``"Dr. Smith PhD"`` family ``Smith``, suffix ``PhD``; ``"Dr. "Smitty" Smith"`` family ``Smith``, nickname ``Smitty``; and the particle spelling this was found through, ``"Freiherr von Richthofen geb. Albrecht"``, reads family ``von Richthofen``, maiden ``Albrecht`` -- 2.2's own particle-chain stop had routed that canonical shape into the rule for the first time (#399). A given-name title is unchanged and still names no family: ``"Sir John née Jones"`` keeps given ``John`` with an empty family, exactly as ``"Sir John"`` does. One name moves where the nickname LEADS, rather than standing beside a name that already reads: ``"'Smitty' Dr. Jones"``. The lone-word nickname rule declines there because the title counts against its piece count, which leaves a title and one name word for this fix to read as the family -- given ``Jones`` through 2.1, family ``Jones`` now. Five names of the differential corpora change reading, the same five at every baseline. Four of them arrive as new diffs and are classified with this fix: ``"Dr. Smith née Jones"``, ``"Senator "Rick" Edmonds"``, ``"Xyz. (Bud) Smith"`` and ``"'Smitty' Dr. Jones"``. The fifth, ``"Freiherr von Richthofen V"``, was already classified under 2.2's particle-chain fix and stays there -- it is the suffix flavor in corpus form, and reads family ``von Richthofen`` with suffix ``V`` where it read given ``von Richthofen`` before. v1 read the two nicknames as 2.1 did; it had no maiden support at all, reading ``"Dr. Smith née Jones"`` as first ``Smith``, middle ``née``, last ``Jones``. The v1 test suite shipped the correct reading for the nickname shape as a known-failing test, which now passes (closes #410) - - Fix a name that is a surname and a maiden clause reporting no family at all. ``"Smith née Jones"`` read given ``Smith`` with an empty ``family``; it now reads family ``Smith``, maiden ``Jones``. A maiden marker announces a FORMER surname, and a former surname only means something beside a current one -- nobody writes a maiden clause where there is no surname for it to set apart -- so where the marker has taken its name and a single name word is left standing, that word is the surname the bearer uses now. With one name word and nothing else to go on the library has to pick a field and picks ``given``, which is a convention rather than a determination and is now written down as such; a maiden marker joins the title and the nickname as one of the things that decides such a name instead. Every spelling of the shape moves, wherever the clause stands and however it is written -- a marker taking the words after it, a marker inside a bracket, or a bracket pair you declared to mean maiden, which carries no marker at all: under ``Policy(maiden_delimiters=frozenset({("(", ")")}))``, ``"Smith (Jones)"`` reads family ``Smith``, maiden ``Jones``. ``"Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, which RESTORES 1.4.0 -- v1 read family ``Smith`` with the clause as a nickname, and only the clause changes hands now. ``"Jane née Jones Smith"``, where the marker stands INSIDE the name and takes the rest of it, reads family ``Jane``, maiden ``Jones Smith``, which no earlier version read (1.4.0 gave first ``Jane``, middle ``née Jones``, last ``Smith``): the rule counts what the clause LEAVES rather than where the marker stood. An annotation beside the name is no part of it, so ``"Smith née Jones PhD"`` reads family ``Smith``, suffix ``PhD``, and ``"'Smitty' Jones Jr. née Smith"`` reads family ``Jones``. Two shapes deliberately do NOT move, and both witnesses are names the differential corpora already carried rather than cases constructed for the release: a word the vocabulary claims as a given name keeps that reading, so ``"abd née Jones"`` still reads given ``abd``, and so does a word written as an initial, so ``"J. née Jones Smith V"`` still reads given ``J.`` -- this rule changes what POSITION decided and does not reach what a word already is. A title still decides the name behind it: ``"Dr. Smith née Jones"`` reads family ``Smith`` as it does since the title fix above, and the given-name title in ``"Sir John née Jones"`` still keeps given ``John`` with no family. A name with two name words beside the clause is untouched -- ``"Jane Smith née Jones"`` reads given ``Jane``, family ``Smith``, maiden ``Jones`` as before. If you have code that reads the lone name word beside a maiden clause out of ``given``, this is the release where it moves to ``family``. Nine of the 1,089 differential corpus names change reading: eight arrive as new diffs at every baseline and are classified with this fix, and the ninth is ``"Smith (née Jones)"``, whose family agrees with 1.4.0 again, so its diff against that baseline SHRANK to the clause alone (closes #445) + - Fix a name that is a surname and a maiden clause reporting no family at all. ``"Smith née Jones"`` read given ``Smith`` with an empty ``family``; it now reads family ``Smith``, maiden ``Jones``. A maiden marker announces a FORMER surname, and a former surname only means something beside a current one -- nobody writes a maiden clause where there is no surname for it to set apart -- so where the marker has taken its name and a single name word is left standing, that word is the surname the bearer uses now. With one name word and nothing else to go on the library has to pick a field and picks ``given``, which is a convention rather than a determination and is now written down as such; a maiden marker joins the title and the nickname as one of the things that decides such a name instead. Every spelling of the shape moves, wherever the clause stands and however it is written -- a marker taking the words after it, a marker inside a bracket, or a bracket pair you declared to mean maiden, which carries no marker at all: under ``Policy(maiden_delimiters=frozenset({("(", ")")}))``, ``"Smith (Jones)"`` reads family ``Smith``, maiden ``Jones``. ``"Smith (née Jones)"`` reads family ``Smith``, maiden ``Jones``, which RESTORES 1.4.0 -- v1 read family ``Smith`` with the clause as a nickname, and only the clause changes hands now. ``"Jane née Jones Smith"``, where the marker stands INSIDE the name and takes the rest of it, reads family ``Jane``, maiden ``Jones Smith``, which no earlier version read (1.4.0 gave first ``Jane``, middle ``née Jones``, last ``Smith``): the rule counts what the clause LEAVES rather than where the marker stood. An annotation beside the name is no part of it, so ``"Smith née Jones PhD"`` reads family ``Smith``, suffix ``PhD``, and ``"'Smitty' Jones Jr. née Smith"`` reads family ``Jones``. Two shapes deliberately do NOT move, and both witnesses are names the differential corpora already carried rather than cases constructed for the release: a word the vocabulary claims as a given name keeps that reading, so ``"abd née Jones"`` still reads given ``abd``, and so does a word written as an initial, so ``"J. née Jones Smith V"`` still reads given ``J.`` -- this rule changes what POSITION decided and does not reach what a word already is. A title still decides the name behind it: ``"Dr. Smith née Jones"`` reads family ``Smith`` as it does since the title fix above, and the given-name title in ``"Sir John née Jones"`` still keeps given ``John`` with no family. A name with two name words beside the clause is untouched -- ``"Jane Smith née Jones"`` reads given ``Jane``, family ``Smith``, maiden ``Jones`` as before. If you have code that reads the lone name word beside a maiden clause out of ``given``, this is the release where it moves to ``family``. Nine of the 1,090 differential corpus names change reading: eight arrive as new diffs at every baseline and are classified with this fix, and the ninth is ``"Smith (née Jones)"``, which every released version already read family ``Smith`` -- so its diff SHRANK, at all three baselines, to the clause changing hands (closes #445) **Deprecations** diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 86af6544..3db72716 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -1543,13 +1543,20 @@ def __post_init__(self) -> None: "both spellings from the other side: the marker " "announces a FORMER surname, so the one name word left " "beside it is the current one. The bracketed spelling " - "RESTORES 1.4.0's family, which read family 'Smith', " - "nickname 'née Jones' (measured 2026-08-26) -- the " - "clause reading maiden rather than nickname is #335's " - "half and the only thing left that differs. Was " - "given 'Smith' on 1.4.0's bare spelling (first " - "'Smith' / middle 'née' / last 'Jones'), on 2.0.0, on " - "2.1.0 and here until this rule"), + "RESTORES the family EVERY released version read: " + "1.4.0, 2.0.0 and 2.1.0 all read this name family " + "'Smith', nickname 'née Jones' (measured on the wheels " + "2026-08-27), so the clause reading maiden rather than " + "nickname is #335's half and the only thing left that " + "differs -- at all three baselines alike. Be exact " + "about which spelling read `given`, because a ledger " + "took the loose wording this note used to carry and " + "narrowed one baseline where three needed it: the BARE " + "spelling is what read given 'Smith' on 2.0.0, on " + "2.1.0 and here until this rule (1.4.0 read it first " + "'Smith' / middle 'née' / last 'Jones'). The bracketed " + "spelling this row holds never read `given` on any " + "released version"), Case("maiden_marker_makes_the_lone_name_word_the_family", "Smith née Jones", {"family": "Smith", "maiden": "Jones"}, diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index d2cb5d85..facc1a4b 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -942,10 +942,15 @@ class _LatinCopy(NamedTuple): frozenset({"^", ",\\s*"}), # fix(#445)'s movers, one corpus name per alternative -- a list of # names, not a copy of any wordlist, so there is no vocabulary for - # it to drift from. Two sets because the two field shapes split - # differently: the 1.4 ledger's seven names all move four fields, - # while at 2.0.0/2.1.0 six of them move two and the other two are - # compounds with rules of their own. + # it to drift from. Two sets because the ledgers group the nine + # movers differently. At 1.4.0 seven of them move the same four + # fields and share the first set; the eighth ('John née Jones + # Smith Ma') moves those plus `suffix`, and the ninth ('Smith + # (née Jones)') keeps the rule it already had. At 2.0.0/2.1.0 the + # second set holds six that move two fields: five of the 1.4 + # seven, plus that acronym name, whose acronym is no part of the + # diff at those baselines. The two the second set drops are the + # compounds with #412 and #424, each with a rule of its own. # # Spelled out rather than written as the shape -- one word, then a # marker -- because that shape also matches rules.md#M4's two @@ -1460,7 +1465,7 @@ def _claim(rule: dict) -> _Claim: }, "expected_since_2.0.0.toml": { "fix(#335) a marker-led clause leaves the one name word its bare reading": - _Claim(1, ('family', 'given', 'maiden', 'nickname'), "c09cc7dba88b"), + _Claim(1, ('maiden', 'nickname'), "c09cc7dba88b"), "fix(#434) a multi-word maiden marker takes the maiden name": _Claim(1, ('family', 'maiden', 'middle'), "c428798fc6ef"), "fix(#434) a multi-word marker leads a bracketed clause to the maiden name": @@ -1578,7 +1583,7 @@ def _claim(rule: dict) -> _Claim: }, "expected_since_2.1.0.toml": { "fix(#335) a marker-led clause leaves the one name word its bare reading": - _Claim(1, ('family', 'given', 'maiden', 'nickname'), "c09cc7dba88b"), + _Claim(1, ('maiden', 'nickname'), "c09cc7dba88b"), "fix(#434) a multi-word maiden marker takes the maiden name": _Claim(1, ('family', 'maiden', 'middle'), "c428798fc6ef"), "fix(#434) a multi-word marker leads a bracketed clause to the maiden name": diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index c01b7753..14a9e0cf 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -1378,12 +1378,16 @@ def test_the_clause_free_corpus_is_not_empty() -> None: already -- the delimiter strip, then the move from word membership to the marker predicate, which a phrase entry made necessary -- and the one-liner in that filter's comment recounts - every stage: 645 names before either, 635 after the strip, 631 - once the predicate decides. Recounted 2026-08-27 by pasting it: - the four rules-corpus rows #445 added move the last stage by two - (the other two carry a marker), and the first two figures were - already one low before that -- a corpus row landed without the - recount the comment asks for. So the floor is what says a future + every stage: 646 names before either, 636 after the strip, 632 + once the predicate decides. Recounted 2026-08-27 by pasting it, + and recounted AGAIN in the review round, which is the point of + shipping the one-liner: the first recount was taken before the + last of #445's rules-corpus rows existed and came out one low at + every stage. #445 adds five rows in all, of which three clear the + marker predicate ('Garcia', 'Smith', 'Smith (Jones)'), so every + stage moves by three -- and the first two figures were already + one low before any of it, a corpus row having landed without the + recount this comment asks for. So the floor is what says a future widening emptied it. Deliberately far below today's count: this asks whether the filter still selects a corpus, not what the corpus holds.""" diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index dfe86fab..b94aa73c 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -137,13 +137,19 @@ issue = "fix(#274) maiden markers consumed" # today's corpus, which has grown since. Today it explains 12 # (measured 2026-08-27, after #445). # -# Four claimed names still do not move, recorded rather than excluded: -# 'Jane Smith, nee Jones' (post-comma, M2's remaining Accepted limit), -# 'Jane van der Berg nee PhD' (only a suffix follows, so the consumer -# declines), and the connective and bound-given joins (#412). Each -# would need the regex to ask a vocabulary question -- suffix or -# connective membership -- a second alternation this rule cannot -# spend. +# Five claimed names still do not move, recorded rather than +# excluded: 'Jane Smith, née Jones' (post-comma, M2's remaining +# Accepted limit), 'Jane van der Berg née PhD' and 'Jane Smith née V' +# (only a suffix follows, so the consumer declines), and +# 'Jane née Jr y Jones' and 'Jane van der Berg née Jr Jones' (the +# connective and bound-given joins, #412). Each would need the regex +# to ask a vocabulary question -- suffix or connective membership -- +# a second alternation this rule cannot spend. The count said four +# until the review round re-measured it: the corpus grew and the +# sentence did not, which is this file's standing hazard and the +# reason every count in it carries the date it was taken. Measured +# 2026-08-27 against the 1.4.0 wheel, name by name over all 31 the +# regex claims. # # #445 landed 2026-08-27, and the prediction this note used to carry # was wrong in a way worth keeping on the record. It reasoned that a @@ -161,13 +167,20 @@ issue = "fix(#274) maiden markers consumed" # Six of the eight moved: five by the fix(#445) rule at the end of # this file, and 'John née Jones Smith Ma' by the fix(#424/#445) # acronym rule above it, its diff carrying the suffix v1 read as -# well. The other two still belong here, and they are -# rules.md#M4's own carve-outs: 'abd née Jones', whose word the -# bound-given vocabulary has claimed, and 'J. née Jones Smith V', -# whose word is an initial. Their diffs are unchanged to the field, -# which is what makes them a boundary rather than an oversight -- if -# either ever gains a `given` move it lands UNEXPLAINED, since this -# rule does not declare that field. The two other fix(#445) names, +# well. The other two are rules.md#M4's own carve-outs, and their +# diffs are unchanged to the field, which is what makes them a +# boundary rather than an oversight. 'abd née Jones' -- whose word +# the bound-given vocabulary has claimed -- is still this rule's. +# 'J. née Jones Smith V' -- whose word is an initial -- is NOT, and +# never was: its 1.4.0 diff carries `suffix`, which these three +# fields cannot hold, so 'fix(#424) accepted: the maiden walk keeps +# the numeral an initial before the marker vetoes' is what explains +# it (asked of classify directly, 2026-08-27, rather than read off +# the regexes). The safety argument holds through that rule instead: +# it declares no `given` either, so a `given` move on either +# carve-out lands UNEXPLAINED rather than absorbed. +# +# The two other fix(#445) names, # 'Smith née Jones' and 'Smith née Jones PhD', are corpus rows M4's # own example lines added. name_regex = "(?i)(?