6 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	f7b3e15014	feat: the drop carrying the teaser's own image links it (4392) ... Crop-to-source matching was held until the cheap signals could be shown insufficient. They can: of artist 8's 27 teasers with a drop inside a day, 11 go unlinked, and five are screenshot teasers with no working name at all. So it was tried, on exactly those pairs. Every teaser image correlated against every window of every nearby drop image at five scales, ground truth being the pairs the working name independently confirms, control being unrelated same-artist posts a month away. **It does not separate** — true pairs score as low as 0.401 while the control reaches 0.605, and no threshold divides them. The reason is the one the naive version was rejected for, which turns out to apply just as hard to the careful one: a single artist's work is stylistically homogeneous, so a whole-image comparison between two of their pieces is high whether or not it is the same piece. That is now written down in the module docstring with its numbers, so the next person to reach for it inherits the measurement instead of repeating it. What survived asks a narrower question the measurement shows IS answerable: not "is this a crop of that" but "is this the same image". Same pairs, same control, using the pHash FC already stores on every image — pairs the name confirms score 0, 0 and 20 bits of 256; the nearest unrelated pair in a 29-sample control scores 108. The threshold sits at 32, which is the number gallery_service already calls a near-duplicate, inside a 76-bit gap. It earns its place by being the only signal needing no cooperation from the creator: it works on a teaser called `Screenshot 2026-08-13`, and on a creator whose two platforms share no naming convention. It is quiet most of the time, because a teaser is usually a crop rather than a copy — but where it fires it is close to certain, and it recovers `Cute Selfie, Cute Dress` from the unreachable list. utils/phash warns the hash alone must not decide a MERGE, since variants of one piece collide at this distance. That does not invert here — it is the point. A merge destroys a file, so a variant colliding with its original is a loss; this asks whether two POSTS are about the same piece, and a variant of the drop's image is exactly that. Nothing is deleted either way. Gated on posts like the other two: an image on many of the creator's posts is a banner, not a piece. `_rarity` is public as `rarity` now that all three signals share it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-24 08:27:20 -04:00
bvandeusenandClaude Opus 5	3313c3b10a	feat: a report that shows what the near-dup gates decide about real artwork (4223) ... The three pixel constants added with the #4223 fix were chosen without ever measuring real files — CI only has synthetic split/solid fixtures, and FC verifies nowhere else. This prints the measurements they should have been chosen from: per pair, the hash distance, the mean drift, the changed-pixel fraction, the verdict, and which gate produced it. It drives the real find_similar with the real confirm rather than restating the decision, so it cannot drift from what the importer does. Read-only: opens files, touches no database. Also splits fingerprint_diff out of fingerprints_match — same computation, now returning the numbers instead of only the boolean, so the report can show how far a pair sat from a limit rather than which side of it it fell on. Runs inside the published :dev image (PIL + imagehash already there, no local env needed) with the art folder mounted read-only — rule 147's channel, so nothing has to reach main to be tried. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 06:30:54 -04:00
bvandeusenandClaude Opus 5	84f873eb1b	fix: ruff UP037 — unquote the fingerprint_path return annotation ... PIL's Image is imported at module scope, so the annotation never needed to be a string. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 00:17:15 -04:00
bvandeusenandClaude Opus 5	a05cb66635	fix: variant artwork was dropped as a near-duplicate even at threshold 0 (4223) ... The operator reported a 15-image variant pack landing as 3 records, then reported variants STILL being dropped with phash_threshold at 0 — the floor of the dial. No setting could have fixed it: at hash_size=8 a pHash is 64 bits of coarse light/dark layout, so two variants sharing a composition produce the SAME bits. Distance 0 meant "identical hash", not "identical image", and the dial was simultaneously too coarse to keep variants and too tight to catch a re-encoded rescale. The hash no longer decides a merge on its own. find_similar now runs three gates, cheapest first: the threshold proposes candidates, aspect ratio (ASPECT_TOL, matching the tier-1 video path) rejects crops and re-canvases, and a pixel-level confirm on the two files accepts. Every gate fails closed — unknown dimensions, an unreadable candidate, a hash of the wrong width all mean "not a duplicate", because too strict keeps a redundant copy the operator can see while too loose deletes artwork only a source re-walk returns. - utils/phash.py: HASH_SIZE 8 -> 16 (256-bit, what ImageRepo always used); aspect_matches, fingerprint/fingerprint_path/fingerprints_match (PIL-only, mean drift + changed-pixel fraction), find_similar gains `confirm`. - importer: _pixel_confirmer supplies gate 3 on both dedup sites, lazily and cached, so a non-matching import costs no extra I/O. - 0098: widens image_record.phash to 64 chars and NULLs every value — a stored 64-bit hash cannot be compared to a 256-bit one, and backfill_phash is NULL-only, keyset-paginated and now on the daily beat, so the library re-hashes itself. Dedup degrades to sha256 until it finishes. - phash_threshold counts bits and the denominator went 64 -> 256, so the setting is reset to the new default of 24 (there is no honest carry-over) and the slider is rescaled to 0-64. - gallery_service dup_threshold 8 -> 32: the same fraction of the hash, so the Explore rail keeps the variance the operator tuned in on 2026-07-01. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 00:16:14 -04:00
bvandeusen	c0c9e56fb9	fix(importer): skip transparency check on animated images (operator-flagged 2026-05-26: animated WebP triggered 5+ min PIL multi-frame decode → Celery hard-timeout SIGKILL); compute_phash seeks frame 0 defensively — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-26 00:45:08 -04:00
bvandeusenandClaude Opus 4.7	d2e8dd3a3b	feat(phash): compute_phash + find_similar util (hash_size=8, fits existing column) ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-17 22:02:42 -04:00