Skip to content

Design: file-strict, a strict choice reads only the chosen files (#753) - #985

Merged
derek73 merged 5 commits into
masterfrom
docs/file-strict-design
Oct 31, 2026
Merged

derek73 merged 5 commits into
masterfrom
docs/file-strict-design

Conversation

@derek73

@derek73 derek73 commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Docs-only design note for #753, phase 3. It redefines strict as file-strict: a strict choice reads only the chosen languages' files plus native-script vocabulary, and mul.toml and und.toml are not among them. For a credential, the colliding reading leaves along with the name reading, so Minh Do cannot read suffix Do. For other kinds, 16 words keep an ungated reading under strict, such as Do Van Minh → family under strict pt; the remedy there is data in the chosen file. Either way, strict no longer needs 3i, the bearers'-languages attribute that name-strict was waiting on. Nothing is built here.

Where it lands

  • docs/design/vocabulary-model-rfc.md, phase 3:
    • a new subsection, "Strict: the chosen files and nothing else";
    • the status row, the prefer/strict bullets and the effect-table row amended;
    • the two name-strict paragraphs replaced;
    • 3i set aside in the work order, with a new 3j row, and the interleave updated.
  • docs/design/decisions/vocabulary-model.md:
    • a new entry, "2026-10-30 Phase 3: language packs that prefer or restrict their language #753, file-strict", which holds the measurements and their recipes;
    • its SUPERSEDES line names decision 6's strict variant, item 2's definition of strict, item 4, and 3i in item 11; superseded-in-part markers are added on those heads;
    • the Standing and Open lines updated;
    • the 3i evidence is kept in the entry, should 3i be wanted again.

Decided on 2026-10-30, recorded as premises:

  • the definition of file-strict;
  • mul.toml and und.toml are excluded, and mul.toml stays as it is, with an Open on migrating its words later;
  • strict vi drops English credentials, accepted in principle.

The 2026-10-28 parser_for(..., strict=True) argument over the whole choice stands. Everything else in the API is proposed, not decided.

Measured on 2026-10-31, master b577e9a8, using only the cached local corpora. The simulation runs the shipped choose and compile_readings over the chosen and native-script index rows, applied as a choice's delta is. Records are scored box by box with local_agreement.read. ORCID records whose given box opens with a title word are left out, because ORCID has no title box.

Country default choice strict strict + en
VN 65.48% none: no file holds vi 64.28% (+349 −589) 64.34% (+306 −534)
KR 92.42% 92.42% 92.54% (+41 −17) 92.51% (+33 −16)
CN 84.54% 84.54% 84.65% (+23 −3) 84.61% (+16 −3)
JP 95.59% 95.59% 95.64% (+30 −20) 95.54% (+11 −20)
CZ 95.12% 96.54% (+635 −352) 96.42% (+626 −367) 96.43% (+625 −364)
SK 95.01% 96.58% (+347 −144) 96.53% (+347 −150) 96.59% (+348 −143)
PL 96.88% 96.88% 96.74% (+4 −32) 96.78% (+4 −25)
DE 95.01% 95.00% (+0 −1) 94.42% (+8 −125) 94.44% (+7 −120)
ES 48.04% 93.47% (+9,295 −215) 93.21% (+9,286 −258) 93.22% (+9,285 −255)
MX 31.33% 93.01% (+12,534 −204) 92.93% (+12,524 −210) 92.93% (+12,523 −209)
BR 47.30% 47.33% (+5 −0) 46.90% (+18 −99) 47.22% (+9 −26)
PT 69.99% 69.99% 69.87% (+1 −25) 69.90% (+1 −19)
NP 90.85% 90.86% (+1 −0) 90.74% (+18 −37) 90.85% (+3 −3)
IN 79.82% 79.82% 79.85% (+23 −17) 79.80% (+6 −10)

Reading the table:

  • The large ES, MX, CZ and SK gains come from the packs' two_family_names, in the choice and strict alike.
  • Against the choice, what strict breaks is mostly immigrant names: DE de 45 and van 39 of 124 breaks, BR junior 62 of 99.
  • The VN ORCID loss is a box convention. Holders often box the middle names with the family, and the default's French le and Dutch van happen to land on that box.

Other results:

  • The van chain. In ORCID VN, of 802 records with van before the last word, agreement goes 479 → 583. In PSC VN, under strict + en, it goes 62 → 147.
  • Cost in English-context data. PSC strict vi moves 865 of 1,499 Vietnamese filers off their boxes, mostly over Mr/Ms/Mrs/Miss; strict + en moves 36. In ClinicalTrials.gov, among 5,686 distinct Korean-surnamed officials' names, strict ko reads a credential as the given name 3,569 times; strict + en never does.
  • What each file lacks to stand alone. Every strict choice loses dr and prof (bare Dr; Dr. stays a title by its period), and every English-context population loses the salutations. cs and sk lose ing, pt loses junior, es loses Catalan i.
  • Files with no readings. hu, kk, kok, vi and zu hold only exclusions. 26 more files have no Latin-script vocabulary reading.
  • The derived markers do not always leave with the reading they gate. 16 words, 27 word-and-tag pairs over 13 tags, each strict choice compared with the plain choice of the same tag, keep a vocabulary reading in a chosen file whose gating name reading sits only in a dropped file. For example, sultan under ar (its given reading is in tr.toml), da/do under pt, and the generation st that cs, hr, sk and sl turn on (Novák, St loses its suffix-or-name report). No credential acronym is ungated today; that holds only while name readings are added to more files rather than moved.

API recommendations (proposed, for review):

  • keep strict=True, and raise on a strict call that chooses no language;
  • keep the shipped constants for packs that carry something, and add locales.choose(*tags) for a bare choice;
  • leave fragments, [rules] patches, given_names and surnames untouched;
  • a caller's own listing of a dropped word is dropped, as 3b's delta does.

The candidates follow-ups, each in its own PR and none taken here, are:

  • Vietnamese degree prefixes as vi.toml readings;
  • nl's prepositions and sushri as data;
  • the 16 ungated words read one by one.

ông and bà stay excluded.

Checks:

  • The design-doc tests pass (2,186).
  • The design-docs-reviewer recomputed every table cell independently through the real parser_for. Its two rounds found 10 and then 2 defects, all fixed. They covered unmarked superseded passages, the decided/proposed status of strict=True, per-word counts, the claim that a word is read by "position alone" (shapes still act), the mul.toml note figures, the Korean-surname mechanism, and the dating of the kept evidence.
  • An independent review from the orchestrating session rebuilt the simulation from the recipe and reproduced the figures. It then found 11 more defects, fixed in 850dc1c8. A second review confirmed those and found that the ungated-word comparator missed readings a choice turns on (st, nic); that is fixed in 45a213d8, which also dates the credential claim. The main ones: the no-harm claim, now limited to credentials; stacked credentials do not survive (Minh Do, M.D., Ph.D. → given M.D.); ctgov figures are now distinct names; the plain nl choice's In Hwan cost is carried over; and the 3e sushri head is marked superseded.
  • Not run: the full suite, since the change is docs only.

🤖 Generated with Claude Code

derek73 and others added 3 commits October 31, 2026 13:49
Strict redefined as file-strict: only the chosen languages' files plus
native-script vocabulary, mul.toml and und.toml excluded, so it needs no
3i. Records the definition, measurements on the cached local corpora
(ORCID per country, the van chain, credential and salutation cost, the
words each file lacks, files with no readings, the ungated words) and a
proposed API, in the RFC's phase 3 and decisions/vocabulary-model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mark the superseded name-strict passages (decision 6, item 2), keep
2026-10-28's strict=True argument as decided, place 3j in the interleave,
count per-word moves by containment, state that shape rules still act,
correct the mul.toml note claim and the Korean surname mechanism, keep
3i's evidence in the entry, and note vi cannot yet be chosen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#753)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added docs Documentation fixes and updates design labels Oct 31, 2026
@derek73 derek73 self-assigned this Oct 31, 2026
derek73 and others added 2 commits October 31, 2026 14:19
…unts and labels (#753)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ds; date the credential claim (#753)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 3.0 milestone Oct 31, 2026
@derek73
derek73 merged commit 8438ccb into master Oct 31, 2026
13 checks passed
@derek73
derek73 deleted the docs/file-strict-design branch October 31, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design docs Documentation fixes and updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant