Repository navigation
File-strict: parser_for(..., strict=True) reads only the chosen languages' files (#753, 3j) - #986
Merged
Merged
Conversation
…sen files (#753, 3j) A strict choice compiles the chosen languages' vocabulary files, each tag selecting its own and its ancestors', plus native-script vocabulary, and nothing else; mul.toml and und.toml leave. It applies as a delta, as a plain choice does. A strict call choosing no language raises with a fix that runs; a strict choice accepts a file holding no reading (vi), which a plain one still refuses. _reading_index.strict_ungated derives the words strict ungates against the plain choice of the same tags, held to the reviewed set (16 words, 27 word-and-tag pairs). The index now lists every file's tag (FILES). The CLI gains --strict and falls back from a pack code to a language tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The RFC's strict section, its status and 3j's row, decisions/vocabulary-model's 2026-10-31 phase 3j entry (the four API points, the CLI, the ORCID table re-run on the built code matching every cell), its Standing and Open lines, mechanisms.md#LOCALE-PACKS-PURE-DATA and AGENTS.md. docs/locales.rst gains "Reading one language only" with romanized Korean as the worked example, docs/usage.rst the --strict flag, and the release log a bullet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#753, 3j) AGENTS.md named the wrong test for strict_ungated; the CLI help and usage.rst said any language with a file works plainly, where vi needs --strict; parser_for's docstring pointed only at the entry whose vi bullet 3j superseded; the RFC's ORCID cell said no file holds vi. The build now refuses prefer and strict as data, naming the caller's call, as the RFC's phase 3 API had decided, in place of 'reserved for phase 3'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ict refusal (#753, 3j) Codecov counted the CLI's new error branches uncovered: their tests ran the command in a subprocess, which coverage does not see. They now call main() in process; a TR_AZ row covers the refusal offering en for a pack whose code is no language. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tag, supersession marks (#753, 3j) --locale zh-CN, es-MX and ja-JP had fallen back to a bare choice, dropping the pack's segmentation, policy, list and segmenter in silence; each now exits 2 naming the pack, while pt-BR resolves to pt_br. --locale tr_az --strict now names a tag to give instead. The 2026-10-30 file-strict entry's head marks every passage 3j decided or reversed, and the 3j entry points at docs/locales.rst, where the bare spelling is taught. A mixed-script row holds is_native_script to 'no Latin letter'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…waits for strict (#753, 3j) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t a region (#753, 3j) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds #753 sub-phase 3j, file-strict, as merged in #985 (decisions/vocabulary-model's "2026-10-30 #753, file-strict" entry and the RFC's "Strict: the chosen files and nothing else").
What it does
parser_for(..., strict=True)compiles only the chosen languages' vocabulary files, plus every native-script reading (a word with a letter and no Latin letter). Each tag selects its own file and its ancestors' files, as a plain choice does.mul.tomlandund.tomlare not among them. Strict applies as a delta against the default compile, as a plain choice does, through_reading_index.chooseandcompile_readingsoverstrict_pool.The four API points (decided 2026-10-31)
ValueError. The message names the packs and offers a pack that chooses a language, e.g.Locale(code='zh', lexicon=Lexicon.empty(), languages=frozenset({'zh'})). A test runs the offered code.locales.choose._reading_index.strict_ungatedderives the words strict ungates. It compares each strict choice with the plain choice of the same tags, not with the default's markers.STRICT_UNGATEDintests/v2/test_reading_index.pyholds the result: 16 words, 27 word-and-tag pairs over 13 tags, exactly the decisions entry's list.FILESinnameparser/config/_index.py,_reading_index.files()).Other changes
UNCHOOSABLE's comment andselected's refusal now say that a plain choice never removes a reading ofmul/undand that a strict one drops both.The build now refuses
preferandstrictas TOML keys, with a message saying that a strength is the caller's choice. The RFC had already decided this message. The old message said both were "reserved for phase 3". The vocab README table is updated to match.CLI:
--strictis new; without--localeit is a usage error (exit 2).--localefalls back to a bare choice when the code is a language tag but not a pack (--locale ko). A code that is neither exits 2 and lists both the pack codes and the tags.--strict, a pack that chooses no language uses its own code as the language (--locale zh --strict).--locale tr_az --strictexits 2 and suggests a tag to give (--locale tr --strict).zh-CN,zh-Hant,es-MX,ja-JP,pt-PT) exits 2 naming the pack (use --locale zh), since a bare choice of it would silently drop the pack's segmentation, policy, list and segmenter. A tag spelled for a pack's code is that pack:pt-BR,PT-BRandpt_brall resolve tolocales.PT_BR, andtr-AZtolocales.TR_AZ.These CLI conveniences were chosen in this PR for the debug helper and are recorded as such.
Docs
docs/design:mechanisms.md#LOCALE-PACKS-PURE-DATAandAGENTS.md.docs/locales.rsthas a new section, "Reading one language only", next to "Choosing languages", where choices are documented. Its worked example is romanized Korean.docs/usage.rstdocuments the--strictflag and links to that section.Kim Do Hyunwas checked against the 1.4.0 and 2.8.0 wheels.Verification
ORCID table: re-run on the built
strict=Truewith the decisions entry's recipe. Every cell matches: 14 countries × 4 columns.Worked examples: checked on the built code.
koKim Do HyunDo HyunDo, familyHyunKim Bin HyunBin HyunBinRa Min JunRaRaKim Min RaRaRa(suffix again underko+en)Also:
Gen Hoshinoreads givenGenunder strictja, andDe Xureads givenDe, familyXuunder strictzh. Every example in the 2026-10-30 entry still holds.No default parse moves:
tests/v2/test_snapshot.pypasses unchanged.Checks run:
uv run pytest: 34,198 passed.uv run pytest -n0 -m serial,uv run mypy,uv run ruff checkandtools/vocab/build.py --check: all pass.Review: the design-docs reviewer found nothing of high severity; its low findings are fixed. A later independent review's items (the CLI subtag refusal above, the
tr_az --strictremedy, the supersession marks on the 2026-10-30 entry, a mixed-scriptis_native_scriptrow) and Codecov's patch coverage (the CLI error tests now run in process) are fixed too, and a second design-docs pass over them found nothing further after its two fixes.Not in this PR: Vietnamese readings in vi.toml,
nlprepositions, the 16 ungated words.🤖 Generated with Claude Code