Skip to content

File-strict: parser_for(..., strict=True) reads only the chosen languages' files (#753, 3j) - #986

Merged
derek73 merged 7 commits into
masterfrom
feat/file-strict-3j
Nov 1, 2026
Merged

derek73 merged 7 commits into
masterfrom
feat/file-strict-3j

Conversation

@derek73

@derek73 derek73 commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Builds #753 sub-phase 3j, file-strict, as merged in #985 (decisions/vocabulary-model's "2026-10-30 #753, file-strict" entry and the RFC's "Strict: the chosen files and nothing else").

What it does

parser_for(..., strict=True) compiles only the chosen languages' vocabulary files, plus every native-script reading (a word with a letter and no Latin letter). Each tag selects its own file and its ancestors' files, as a plain choice does. mul.toml and und.toml are not among them. Strict applies as a delta against the default compile, as a plain choice does, through _reading_index.choose and compile_readings over strict_pool.

The four API points (decided 2026-10-31)

  • (a) A strict call that chooses no language raises ValueError. The message names the packs and offers a pack that chooses a language, e.g. Locale(code='zh', lexicon=Lexicon.empty(), languages=frozenset({'zh'})). A test runs the offered code.
  • (b) There is no public locales.choose.
  • (c) _reading_index.strict_ungated derives the words strict ungates. It compares each strict choice with the plain choice of the same tags, not with the default's markers. STRICT_UNGATED in tests/v2/test_reading_index.py holds the result: 16 words, 27 word-and-tag pairs over 13 tags, exactly the decisions entry's list.
  • (d) A strict choice accepts a tag whose file holds no reading (hu, kk, kok, vi, zu). A plain choice is still refused, and its message now says strict accepts the tag. To know these files, the reading index now records every file's tag (FILES in nameparser/config/_index.py, _reading_index.files()).

Other changes

  • UNCHOOSABLE's comment and selected's refusal now say that a plain choice never removes a reading of mul/und and that a strict one drops both.

  • The build now refuses prefer and strict as TOML keys, with a message saying that a strength is the caller's choice. The RFC had already decided this message. The old message said both were "reserved for phase 3". The vocab README table is updated to match.

  • CLI:

    • --strict is new; without --locale it is a usage error (exit 2).
    • --locale falls back to a bare choice when the code is a language tag but not a pack (--locale ko). A code that is neither exits 2 and lists both the pack codes and the tags.
    • Under --strict, a pack that chooses no language uses its own code as the language (--locale zh --strict). --locale tr_az --strict exits 2 and suggests a tag to give (--locale tr --strict).
    • A pack's language with a region or script subtag (zh-CN, zh-Hant, es-MX, ja-JP, pt-PT) exits 2 naming the pack (use --locale zh), since a bare choice of it would silently drop the pack's segmentation, policy, list and segmenter. A tag spelled for a pack's code is that pack: pt-BR, PT-BR and pt_br all resolve to locales.PT_BR, and tr-AZ to locales.TR_AZ.

    These CLI conveniences were chosen in this PR for the debug helper and are recorded as such.

Docs

  • docs/design:
  • docs/locales.rst has a new section, "Reading one language only", next to "Choosing languages", where choices are documented. Its worked example is romanized Korean. docs/usage.rst documents the --strict flag and links to that section.
  • Release-log bullet. The default reading of Kim Do Hyun was checked against the 1.4.0 and 2.8.0 wheels.

Verification

  • ORCID table: re-run on the built strict=True with the decisions entry's recipe. Every cell matches: 14 countries × 4 columns.

  • Worked examples: checked on the built code.

    Name Default Strict ko
    Kim Do Hyun family Do Hyun middle Do, family Hyun
    Kim Bin Hyun family Bin Hyun middle Bin
    Ra Min Jun title Ra given Ra
    Kim Min Ra suffix Ra family Ra (suffix again under ko + en)

    Also: Gen Hoshino reads given Gen under strict ja, and De Xu reads given De, family Xu under strict zh. Every example in the 2026-10-30 entry still holds.

  • No default parse moves: tests/v2/test_snapshot.py passes unchanged.

  • Checks run:

    • uv run pytest: 34,198 passed.
    • uv run pytest -n0 -m serial, uv run mypy, uv run ruff check and tools/vocab/build.py --check: all pass.
    • The full suite also passes on Python 3.15, without namedivider (this venv) and with it.
    • Sphinx doctest and HTML builds are clean.
  • Review: the design-docs reviewer found nothing of high severity; its low findings are fixed. A later independent review's items (the CLI subtag refusal above, the tr_az --strict remedy, the supersession marks on the 2026-10-30 entry, a mixed-script is_native_script row) and Codecov's patch coverage (the CLI error tests now run in process) are fixed too, and a second design-docs pass over them found nothing further after its two fixes.

Not in this PR: Vietnamese readings in vi.toml, nl prepositions, the 16 ungated words.

🤖 Generated with Claude Code

derek73 and others added 3 commits October 31, 2026 17:46
…sen files (#753, 3j)

A strict choice compiles the chosen languages' vocabulary files, each tag
selecting its own and its ancestors', plus native-script vocabulary, and
nothing else; mul.toml and und.toml leave. It applies as a delta, as a
plain choice does. A strict call choosing no language raises with a fix
that runs; a strict choice accepts a file holding no reading (vi), which
a plain one still refuses. _reading_index.strict_ungated derives the
words strict ungates against the plain choice of the same tags, held to
the reviewed set (16 words, 27 word-and-tag pairs). The index now lists
every file's tag (FILES). The CLI gains --strict and falls back from a
pack code to a language tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The RFC's strict section, its status and 3j's row, decisions/vocabulary-model's
2026-10-31 phase 3j entry (the four API points, the CLI, the ORCID table
re-run on the built code matching every cell), its Standing and Open lines,
mechanisms.md#LOCALE-PACKS-PURE-DATA and AGENTS.md. docs/locales.rst gains
"Reading one language only" with romanized Korean as the worked example,
docs/usage.rst the --strict flag, and the release log a bullet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#753, 3j)

AGENTS.md named the wrong test for strict_ungated; the CLI help and
usage.rst said any language with a file works plainly, where vi needs
--strict; parser_for's docstring pointed only at the entry whose vi bullet
3j superseded; the RFC's ORCID cell said no file holds vi. The build now
refuses prefer and strict as data, naming the caller's call, as the RFC's
phase 3 API had decided, in place of 'reserved for phase 3'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added enhancement docs Documentation fixes and updates design vocabulary Per-word vocabulary model and language packs labels Nov 1, 2026
@derek73 derek73 self-assigned this Nov 1, 2026
derek73 and others added 4 commits October 31, 2026 18:05
…ict refusal (#753, 3j)

Codecov counted the CLI's new error branches uncovered: their tests ran
the command in a subprocess, which coverage does not see. They now call
main() in process; a TR_AZ row covers the refusal offering en for a
pack whose code is no language.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tag, supersession marks (#753, 3j)

--locale zh-CN, es-MX and ja-JP had fallen back to a bare choice,
dropping the pack's segmentation, policy, list and segmenter in silence;
each now exits 2 naming the pack, while pt-BR resolves to pt_br.
--locale tr_az --strict now names a tag to give instead. The 2026-10-30
file-strict entry's head marks every passage 3j decided or reversed, and
the 3j entry points at docs/locales.rst, where the bare spelling is
taught. A mixed-script row holds is_native_script to 'no Latin letter'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…waits for strict (#753, 3j)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t a region (#753, 3j)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit 928bccb into master Nov 1, 2026
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design docs Documentation fixes and updates enhancement vocabulary Per-word vocabulary model and language packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant