Skip to content

Attach the installed Japanese segmenter under python -m nameparser --locale ja - #984

Merged
derek73 merged 5 commits into
masterfrom
feat/cli-ja-segmenter
Oct 31, 2026
Merged

derek73 merged 5 commits into
masterfrom
feat/cli-ja-segmenter

Conversation

@derek73

@derek73 derek73 commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Summary

python -m nameparser --locale ja took the pack but had no way to attach a segmenter, so it divided nothing and warned (rules/D1, the segmenterless-activation warning). It now attaches locales.ja_segmenter() when namedivider is installed:

$ python -m nameparser --locale ja --json "山田太郎"
{"title": "", "given": "太郎", "middle": "", "family": "山田", "suffix": "", "nickname": "", "maiden": ""}
  • Only a missing package falls back. The CLI tests for namedivider with importlib.util.find_spec; any failure to load an installed one (numpy's ImportError, an OSError) propagates with its own message rather than being reported as "not installed".
  • Without the extra the name reads as before, and the CLI prints nameparser: --locale ja divides unspaced names only with namedivider-python installed: pip install 'nameparser[ja]' in place of the parser's warning, which it filters by message: that warning's other remedies are code a command line cannot pass, and a print still shows under PYTHONWARNINGS=ignore.
  • CLI only. parser_for(locales.JA) still needs segmenter=locales.ja_segmenter() and still warns without one. decisions/D1's new entry records the library-wide form as declined: the same code would parse 高橋一郎 differently with and without the extra, a Parser holding ja_segmenter() cannot be pickled (local closure), so parser_for(locales.JA) would stop pickling wherever the extra is installed, and the pack-and-segmenter pairing is explicit by design.
  • Only the ja pack consults the factory.

Tests (tests/v2/test_cli.py)

In process, with the install stood in for so every branch runs in every environment: the segmenter attaches under ja and JA and survives each --order, with no warning; the missing extra prints the note once and warns nothing, with and without --order; a broken install fails with its own ImportError or OSError; zh never calls the factory. Plus one subprocess run per environment, each skipping where it cannot apply (the extra one runs only in the ja-extra job). Mutations checked against them: a broad except ImportError (3 fail), the segmenter dropped under --order (6), the warning not filtered (3), the factory called for every pack (1). #983's warns-once test is gone: the CLI no longer reaches that warning.

Docs: usage.rst's "has no way to attach one" paragraph rewritten, locales.rst's "equivalent to parser_for(locales.get(...))" qualified for ja, release-log bullet.

No parser behavior changes, so no rules or snapshot moves.

Test plan

  • uv run pytest without the extra (34,158 passed)
  • uv run --with 'namedivider-python>=0.4' pytest (34,181 passed)
  • uv run mypy, uv run ruff check, sphinx-build -b html clean for the edited pages

🤖 Generated with Claude Code

…locale ja

The CLI took the ja pack but had no way to attach a segmenter, so
--locale ja divided nothing and warned. It now calls
locales.ja_segmenter() for the ja pack and passes it to parser_for,
falling back to the segmenterless parser, and its warning, when the
extra is not installed. parser_for(locales.JA) stays explicit:
decisions/D1 records the library-wide form as declined, since the same
code would parse differently with and without the extra.

The #983 warning test assumed --locale ja always warns; it now makes
the extra missing, so it holds in the ja-extra job too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 self-assigned this Oct 31, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 3.0 milestone Oct 31, 2026
derek73 and others added 3 commits October 31, 2026 12:58
Review of #984: the fallback caught every ImportError, so an installed
but broken namedivider (numpy failing to import) was reported as not
installed, its cause dropped; and the parser's warning it fell back to
offered remedies that are code a command line cannot pass, and showed
nothing under PYTHONWARNINGS=ignore. The CLI now tests for the package
with find_spec, lets any load failure propagate, and in its absence
prints a note naming the install and filters the parser's warning by
message.

Tests: the segmenter survives every --order and an uppercase JA; the
missing extra notes once and warns nothing; a broken install fails with
its own ImportError or OSError; the zh check records calls rather than
raising, since a broad catch would have swallowed the AssertionError.
#983's warns-once test goes: the CLI no longer reaches that warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ocales.rst

Review of #984: a Parser holding ja_segmenter() cannot be pickled at
all (its segmenter is a local closure), so the declined library-wide
attach would stop parser_for(locales.JA) pickling wherever the extra is
installed, not carry a callable. The entry gains its verdict headline
and #984, the Standing line points at it, and both describe the note
that now replaces the warning. locales.rst no longer reads as if the
CLI were parser_for(locales.get(code)) for every code; usage.rst and
the release log say note, not warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every in-process test stands in for _ja_extra_installed, and the
subprocess runs that reach it are not measured, so its line went
uncovered (codecov/patch 94.73%). The new test calls it and holds it to
the find_spec answer the subprocess tests' skips are keyed on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit 3330729 into master Oct 31, 2026
13 checks passed
@derek73
derek73 deleted the feat/cli-ja-segmenter branch October 31, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant