feat(ml): normalize Camie suggestion names to human-readable #56

Merged

bvandeusen merged 2 commits from dev into main 2026-06-03 13:18:45 -04:00

Conversation 0 Commits 2 Files Changed 4

+148 -12

bvandeusen commented 2026-06-03 13:18:40 -04:00

Owner

Copy Link

Copy Source

Summary

Camie's booru-style vocab strings (uchiha_sasuke_(naruto), #unicus_(idolmaster), 1000-nen_ikiteru_(vocaloid), :/) surfaced raw in SuggestionsPanel — and worse, the same raw string was written to tag.name on Accept, polluting the DB with underscored_lowercase names that don't match the operator's Title Case convention.

• New backend/app/services/ml/tag_name.py with normalize(raw) → str | None: strip leading junk (#, ., +, ;, ~, _), drop trailing _(disambiguator) blocks iteratively, strip wrapping quotes, underscores → spaces, space after colon, title-case each space-separated word, preserve hyphens / apostrophes / digits, drop entries with no alphanumerics (emoticons like :/, +_+, ^_^).
• Wired into SuggestionService.for_image: alias_map still keyed on the raw Camie string (alias rows are hand-curated against raw keys), but display_name and the existing-tag lookup use the normalized form. Existing-tag lookup widened to func.lower(Tag.name).in_([raw, display]) so legacy underscore-named Tag rows from before normalization still surface as "existing" instead of "+ new".
• No surname/givenname swap — no reliable signal in the vocab. Operator confirmed skip.

Test plan

• tests/test_ml_tag_name.py — 24 parametrized cases (all 9 rules + edge cases: emoticons, repeated disambig blocks, quoted forms, leading junk, hyphens preserved, digit-only year tags, apostrophe inside word)
• tests/test_ml_suggestions.py two existing assertions updated to title-cased forms ("Sword", "Brand New Tag")
• CI run 340 (e450145) green across all 6 jobs (lint, frontend-build, backend-lint-and-test, intimp, intapi, intcore)
• Operator UI spot-check after deploy: open a post with suggestions, confirm names read "Light Purple Hair" not light_purple_hair

Plan: Scribe task #590.

## Summary Camie's booru-style vocab strings (`uchiha_sasuke_(naruto)`, `#unicus_(idolmaster)`, `1000-nen_ikiteru_(vocaloid)`, `:/`) surfaced raw in SuggestionsPanel — and worse, the **same** raw string was written to `tag.name` on Accept, polluting the DB with `underscored_lowercase` names that don't match the operator's Title Case convention. - New `backend/app/services/ml/tag_name.py` with `normalize(raw) → str | None`: strip leading junk (`#`, `.`, `+`, `;`, `~`, `_`), drop trailing `_(disambiguator)` blocks iteratively, strip wrapping quotes, underscores → spaces, space after colon, title-case each space-separated word, preserve hyphens / apostrophes / digits, drop entries with no alphanumerics (emoticons like `:/`, `+_+`, `^_^`). - Wired into `SuggestionService.for_image`: alias_map still keyed on the raw Camie string (alias rows are hand-curated against raw keys), but `display_name` and the existing-tag lookup use the normalized form. Existing-tag lookup widened to `func.lower(Tag.name).in_([raw, display])` so legacy underscore-named Tag rows from before normalization still surface as "existing" instead of "+ new". - No surname/givenname swap — no reliable signal in the vocab. Operator confirmed skip. ## Test plan - [x] `tests/test_ml_tag_name.py` — 24 parametrized cases (all 9 rules + edge cases: emoticons, repeated disambig blocks, quoted forms, leading junk, hyphens preserved, digit-only year tags, apostrophe inside word) - [x] `tests/test_ml_suggestions.py` two existing assertions updated to title-cased forms ("Sword", "Brand New Tag") - [x] CI run 340 (e450145) green across all 6 jobs (lint, frontend-build, backend-lint-and-test, intimp, intapi, intcore) - [ ] Operator UI spot-check after deploy: open a post with suggestions, confirm names read "Light Purple Hair" not `light_purple_hair` Plan: Scribe task #590.

bvandeusen added 2 commits 2026-06-03 13:18:40 -04:00

feat(ml): normalize Camie suggestion names to human-readable ... a6e8d4b52e

Camie's booru-style vocab strings (`uchiha_sasuke_(naruto)`,
`#unicus_(idolmaster)`, `1000-nen_ikiteru_(vocaloid)`, `:/`) were
surfacing raw in SuggestionsPanel — and worse, the SAME raw string was
written to tag.name on Accept, polluting the DB with `underscored_lowercase`
names that don't match the operator's "Title Case" tag convention.

Add backend/app/services/ml/tag_name.py with a single normalize()
applying nine rules (strip leading junk #/./+/;/~/_/ws, drop trailing
_(disambiguator) blocks iteratively, strip wrapping quotes, underscores
to spaces, space after colon, title-case each word's first char,
preserve hyphens/apostrophes/digits, drop entries with no letters).

Wire into SuggestionService.for_image:
- raw Camie key kept for alias_map lookup (alias rows are hand-curated
  against raw keys; don't disturb)
- display_name = normalize(raw); None means drop the candidate
- existing-tag lookup widened to case-insensitive match against BOTH
  raw and normalized forms so legacy underscore-named Tag rows accepted
  before this change still surface as "existing" not "+ new"

fix(ml): preserve digit-only tag names in normalize (year tags) ... e450145304

Rule 8 'no letters -> drop' was over-eager: bare digit tags like '2005'
returned None even though they're legitimate (booru year-tag shape).
Widen the keep-condition to any alphanumeric. Emoticons (':/', '^_^',
'+_+') still drop since they contain neither letters nor digits.

bvandeusen merged commit 279dff3fb6 into main 2026-06-03 13:18:45 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#56