Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e2eeb63115 | ||
|
|
e704c70f32 | ||
|
|
0fad744bfb | ||
|
|
32874ca678 | ||
|
|
edd2daa16a | ||
|
|
23e062dd4a | ||
|
|
a360d69ee8 | ||
|
|
dfd28a0aa6 | ||
|
|
efcb548ebf | ||
|
|
4fa7975963 | ||
|
|
7b1570f2a5 |
@@ -142,6 +142,16 @@ concurrency:
|
||||
# Deriving it per job invites the two halves to disagree: sign-extension would
|
||||
# derive dev's extension version while build-web bundled main's, and the
|
||||
# release download would 404 on a version that exists perfectly well.
|
||||
#
|
||||
# On every other trigger it is the COMMIT that fired (`github.sha`), never the
|
||||
# branch name. A branch is re-resolved by each job's checkout when that job
|
||||
# starts, so a push landing mid-run moved the later jobs onto the new tip: run
|
||||
# 7499 signed 423275a's extension, then build-web checked out 83e1382 (pushed
|
||||
# while 7499 ran), derived a version nobody had signed, and 404'd on the
|
||||
# download (#4427). The guard failed closed that time; a job without one would
|
||||
# have published a commit the run's own lanes never tested — the lanes check
|
||||
# out `github.sha` by default, so pinning here makes the publish build exactly
|
||||
# what they passed.
|
||||
# IS THIS A BASE REFRESH? Asked in five places and previously spelled five
|
||||
# ways — `github.event_name == 'schedule'` in an `if:`, `$GITHUB_EVENT_NAME` in
|
||||
# one shell, an `EVENT:` env passed into another, and a bare expression on
|
||||
@@ -178,7 +188,16 @@ concurrency:
|
||||
# where a step-level `if:` needs the answer before any shell runs.
|
||||
env:
|
||||
IS_REFRESH: ${{ (github.event_name == 'schedule' || format('{0}', github.event.inputs.refresh) == 'true') && 'true' || 'false' }}
|
||||
BUILD_REF: ${{ (github.event_name == 'schedule' || format('{0}', github.event.inputs.refresh) == 'true') && 'main' || github.ref }}
|
||||
BUILD_REF: ${{ (github.event_name == 'schedule' || format('{0}', github.event.inputs.refresh) == 'true') && 'main' || github.sha }}
|
||||
# What the six LANES check out. Empty — the checkout default, the triggering
|
||||
# commit (or a PR's merge ref) — on every trigger but the refresh, where it is
|
||||
# `main`: the refresh publishes main, so the gate has to test main (#4430). It
|
||||
# is not BUILD_REF itself because a pull_request run's `github.sha` is a merge
|
||||
# commit the default checkout reaches through its ref, not by sha. Each job
|
||||
# still resolves `main` when it starts, so a merge to main during the ~5 min of
|
||||
# a Sunday-06:00 refresh could put the lanes and the build one commit apart;
|
||||
# the build jobs' own guards assert the branch, not the commit.
|
||||
LANE_REF: ${{ (github.event_name == 'schedule' || format('{0}', github.event.inputs.refresh) == 'true') && 'main' || '' }}
|
||||
|
||||
# Requires repo secret RELEASE_TOKEN — a Forgejo PAT with scopes:
|
||||
# - write:package, read:package (for docker push to git.fabledsword.com)
|
||||
@@ -219,6 +238,8 @@ jobs:
|
||||
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ env.LANE_REF }}
|
||||
- name: Ruff lint
|
||||
# agent/ included so the GPU-agent is linted before its image is built
|
||||
# (build.yml only `docker build`s it — this is where it gets checked).
|
||||
@@ -265,6 +286,7 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ env.LANE_REF }}
|
||||
# The derivation needs real history: a depth-1 clone sees one commit
|
||||
# and produces a wrong, too-low value RATHER THAN FAILING. Checking
|
||||
# that here is half the point of the lane.
|
||||
@@ -320,6 +342,7 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ env.LANE_REF }}
|
||||
# Full history for tests/test_artifact_identity.py, which derives
|
||||
# each artifact's revision to check the identity scheme. On a
|
||||
# depth-1 clone that derivation either fails or returns the tip sha
|
||||
@@ -366,6 +389,8 @@ jobs:
|
||||
working-directory: frontend
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ env.LANE_REF }}
|
||||
# No package-lock.json is tracked yet (we don't run npm locally per
|
||||
# feedback-no-local-runs). Using `npm install` instead of `npm ci`.
|
||||
# If we want strict lockfile-based reproducibility later, commit a
|
||||
@@ -395,6 +420,8 @@ jobs:
|
||||
image: node:24-bookworm-slim
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ env.LANE_REF }}
|
||||
# Not --no-save: vitest and web-ext are both real devDependencies now,
|
||||
# and the suite needs vitest resolvable from node_modules.
|
||||
- name: Install dev dependencies
|
||||
@@ -497,6 +524,8 @@ jobs:
|
||||
--health-retries 10
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ env.LANE_REF }}
|
||||
- name: Integration suite (resolve service IPs, migrate, test)
|
||||
run: |
|
||||
set -eux
|
||||
@@ -605,8 +634,8 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# Not the triggering ref — see the `env:` block at the top. On a
|
||||
# scheduled refresh this is `main`; on everything else it is the ref
|
||||
# that fired, so this is a no-op on every ordinary path.
|
||||
# scheduled refresh this is `main`; on everything else it is the
|
||||
# commit that fired, the same one the lanes above tested.
|
||||
ref: ${{ env.BUILD_REF }}
|
||||
# Full history is load-bearing, not a convenience: the version this
|
||||
# job signs is derived from the commit TIME of the newest packaged
|
||||
@@ -945,8 +974,8 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# Not the triggering ref — see the `env:` block at the top. On a
|
||||
# scheduled refresh this is `main`; on everything else it is the ref
|
||||
# that fired, so this is a no-op on every ordinary path.
|
||||
# scheduled refresh this is `main`; on everything else it is the
|
||||
# commit that fired, the same one the lanes above tested.
|
||||
ref: ${{ env.BUILD_REF }}
|
||||
# Full history: this job RE-DERIVES the extension version rather than
|
||||
# being handed it, and a depth-1 clone derives a wrong, too-low value
|
||||
@@ -2174,8 +2203,8 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# Not the triggering ref — see the `env:` block at the top. On a
|
||||
# scheduled refresh this is `main`; on everything else it is the ref
|
||||
# that fired, so this is a no-op on every ordinary path.
|
||||
# scheduled refresh this is `main`; on everything else it is the
|
||||
# commit that fired, the same one the lanes above tested.
|
||||
ref: ${{ env.BUILD_REF }}
|
||||
# Full history: this job derives its artifact's version from the
|
||||
# commit its shipped files last changed in (milestone 313). A
|
||||
|
||||
@@ -22,7 +22,8 @@ through afterwards.
|
||||
- **ML tagging.** Runs image models in-container to suggest tags, group
|
||||
characters, find near-duplicates and power similarity search. Suggestions are
|
||||
reviewable — it proposes, you confirm, and it learns which proposals you keep
|
||||
rejecting.
|
||||
rejecting. It ships switched off: turn it on under Settings → System when
|
||||
you want it, and it fetches its model weights then.
|
||||
- **Deduplication and provenance.** Everything that arrives is hashed and
|
||||
deduplicated by content, metadata sidecars are read wherever the source
|
||||
writes them, and every file keeps a record of where it came from.
|
||||
@@ -119,9 +120,12 @@ needs every credential entered again by hand.
|
||||
|
||||
A few other things are worth knowing about the first few minutes:
|
||||
|
||||
- **The ML worker downloads its model weights on first boot**, several GB from
|
||||
HuggingFace into `./models`. Until that finishes, tagging is queued rather
|
||||
than broken. It is idempotent — a restart resumes rather than refetches.
|
||||
- **ML tagging starts switched off, and nothing is downloaded at boot.** Give
|
||||
the ML lane a slot under **Settings → System** and it fetches its model
|
||||
weights then — several GB from HuggingFace into `./models`, shown as a job
|
||||
under **Settings → Activity** that you can watch and retry. Until it
|
||||
finishes, tagging is queued rather than broken, and the fetch only takes
|
||||
what is missing, so turning the lane off and on again does not refetch.
|
||||
- **The gallery starts empty**, and that is the expected state. Add a creator
|
||||
under **Subscriptions** and it fills as posts come down.
|
||||
- **If you already have a library on disk**, there is no screen that imports
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Retire the sketch/doodle WIP title tier — its tags, its review flags, its toggle.
|
||||
|
||||
Milestone 430, #4428. The soft tier (#1474) tagged `wip` on any post titled
|
||||
sketch / doodle / scribble. Measured on the operator's library it was 6,096 of
|
||||
8,876 wip tags, and its conflict audit filled the Gallery's review strip with
|
||||
2,086 cards, because most finished art scores >= 0.5 on some content head. A
|
||||
"sketch" is usually finished work, so the operator retired it (2026-09-25).
|
||||
|
||||
Data:
|
||||
|
||||
* A soft tag the operator stood behind is kept and relabelled `manual`: one they
|
||||
confirmed (tag_positive_confirmation), or one whose review flag they resolved
|
||||
while leaving the tag on (the strip's "Keep tag").
|
||||
* Every other `wip_title_soft` row is deleted.
|
||||
* Unresolved review flags whose tag is no longer on the image are deleted: the
|
||||
question they ask no longer applies. That is the audit's cards, and any older
|
||||
orphan the same way.
|
||||
|
||||
Then `import_settings.wip_soft_title_tagging_enabled` is dropped. The downgrade
|
||||
restores the column only; deleted tags are not recreated.
|
||||
|
||||
Revision ID: 0112
|
||||
Revises: 0111
|
||||
Create Date: 2026-09-25
|
||||
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0112"
|
||||
down_revision = "0111"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def retire_soft_wip_tags(conn) -> None:
|
||||
"""The data half, on a plain connection, so a test can run it directly."""
|
||||
conn.execute(sa.text("""
|
||||
UPDATE image_tag it SET source = 'manual'
|
||||
WHERE it.source = 'wip_title_soft'
|
||||
AND (
|
||||
EXISTS (
|
||||
SELECT 1 FROM tag_positive_confirmation c
|
||||
WHERE c.image_record_id = it.image_record_id AND c.tag_id = it.tag_id
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM presentation_review pr
|
||||
WHERE pr.image_record_id = it.image_record_id AND pr.tag_id = it.tag_id
|
||||
AND pr.resolved_at IS NOT NULL
|
||||
)
|
||||
)
|
||||
"""))
|
||||
conn.execute(sa.text("DELETE FROM image_tag WHERE source = 'wip_title_soft'"))
|
||||
conn.execute(sa.text("""
|
||||
DELETE FROM presentation_review pr
|
||||
WHERE pr.resolved_at IS NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM image_tag it
|
||||
WHERE it.image_record_id = pr.image_record_id AND it.tag_id = pr.tag_id
|
||||
)
|
||||
"""))
|
||||
|
||||
|
||||
def upgrade():
|
||||
retire_soft_wip_tags(op.get_bind())
|
||||
op.drop_column("import_settings", "wip_soft_title_tagging_enabled")
|
||||
|
||||
|
||||
def downgrade():
|
||||
op.add_column(
|
||||
"import_settings",
|
||||
sa.Column(
|
||||
"wip_soft_title_tagging_enabled",
|
||||
sa.Boolean(),
|
||||
nullable=False,
|
||||
server_default=sa.text("false"),
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Re-date images whose post's date arrived after they were linked.
|
||||
|
||||
#4431. The native ingesters import a post's media before its record, and the
|
||||
date travels in the record (`_post.json`). Every natively downloaded image was
|
||||
therefore linked to an undated post and kept its download time in both gallery
|
||||
date columns, while the post itself was dated correctly. The importer now
|
||||
re-dates a post's images when its record lands; this repairs the images that
|
||||
landed before that.
|
||||
|
||||
Both columns get back the rules the importer keeps:
|
||||
|
||||
* `effective_date` is the primary post's date (left alone when that post has
|
||||
none, as the importer does);
|
||||
* `earliest_post_date` is the earliest dated post the image is linked to.
|
||||
|
||||
Only rows that differ are written. The downgrade does nothing: the old values
|
||||
were download times that no one chose.
|
||||
|
||||
Revision ID: 0113
|
||||
Revises: 0112
|
||||
Create Date: 2026-09-25
|
||||
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0113"
|
||||
down_revision = "0112"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def redate_images(conn) -> None:
|
||||
"""The data step, on a plain connection, so a test can run it directly."""
|
||||
conn.execute(sa.text("""
|
||||
UPDATE image_record ir SET effective_date = p.post_date
|
||||
FROM post p
|
||||
WHERE p.id = ir.primary_post_id
|
||||
AND p.post_date IS NOT NULL
|
||||
AND ir.effective_date IS DISTINCT FROM p.post_date
|
||||
"""))
|
||||
conn.execute(sa.text("""
|
||||
UPDATE image_record ir SET earliest_post_date = m.earliest
|
||||
FROM (
|
||||
SELECT ip.image_record_id, MIN(p.post_date) AS earliest
|
||||
FROM image_provenance ip JOIN post p ON p.id = ip.post_id
|
||||
WHERE p.post_date IS NOT NULL
|
||||
GROUP BY ip.image_record_id
|
||||
) m
|
||||
WHERE m.image_record_id = ir.id
|
||||
AND ir.earliest_post_date IS DISTINCT FROM m.earliest
|
||||
"""))
|
||||
|
||||
|
||||
def upgrade():
|
||||
redate_images(op.get_bind())
|
||||
|
||||
|
||||
def downgrade():
|
||||
pass
|
||||
@@ -57,7 +57,6 @@ _EDITABLE_FIELDS = (
|
||||
"translation_target_lang",
|
||||
"translation_min_confidence",
|
||||
"wip_title_tagging_enabled",
|
||||
"wip_soft_title_tagging_enabled",
|
||||
)
|
||||
|
||||
# Per-host external-download toggles — all plain booleans, validated uniformly.
|
||||
@@ -196,12 +195,6 @@ async def update_import_settings():
|
||||
return jsonify(
|
||||
{"error": "wip_title_tagging_enabled must be a boolean"}
|
||||
), 400
|
||||
if "wip_soft_title_tagging_enabled" in body and not isinstance(
|
||||
body["wip_soft_title_tagging_enabled"], bool
|
||||
):
|
||||
return jsonify(
|
||||
{"error": "wip_soft_title_tagging_enabled must be a boolean"}
|
||||
), 400
|
||||
|
||||
async with get_session() as session:
|
||||
row = await ImportSettings.load(session)
|
||||
|
||||
@@ -152,6 +152,8 @@ async def list_runs():
|
||||
queue=<name> filter to one queue
|
||||
status=<status> filter to one status (running/ok/error/timeout/retry)
|
||||
task=<substr> case-insensitive substring match on task_name
|
||||
celery_task_id=<id> exactly one run — how a page follows a job it
|
||||
started without having to know its lane
|
||||
limit=<int> default 50, max 200
|
||||
before_id=<int> cursor for keyset pagination
|
||||
|
||||
@@ -167,6 +169,7 @@ async def list_runs():
|
||||
queue = request.args.get("queue")
|
||||
status = request.args.get("status")
|
||||
task = request.args.get("task")
|
||||
celery_task_id = request.args.get("celery_task_id")
|
||||
before_id_raw = request.args.get("before_id")
|
||||
before_id = int(before_id_raw) if before_id_raw else None
|
||||
|
||||
@@ -176,6 +179,8 @@ async def list_runs():
|
||||
stmt = stmt.where(TaskRun.queue == queue)
|
||||
if status:
|
||||
stmt = stmt.where(TaskRun.status == status)
|
||||
if celery_task_id:
|
||||
stmt = stmt.where(TaskRun.celery_task_id == celery_task_id)
|
||||
if task:
|
||||
# Task names contain literal underscores (download_source,
|
||||
# vacuum_analyze) — escape LIKE wildcards so a search for
|
||||
|
||||
@@ -223,11 +223,6 @@ def make_celery() -> Celery:
|
||||
"schedule": 86400.0, # auto-tag wip/editor process art (#1464);
|
||||
# no-op unless process_auto_apply_enabled (opt-in)
|
||||
},
|
||||
"soft-wip-conflict-audit-daily": {
|
||||
"task": "backend.app.tasks.ml.scheduled_soft_wip_conflict_audit",
|
||||
"schedule": 86400.0, # flag ring-loud soft-WIP (sketch/doodle) tags
|
||||
# for review (#1474); no-op with no content heads
|
||||
},
|
||||
"prune-presentation-reviews-daily": {
|
||||
"task": "backend.app.tasks.ml.prune_presentation_reviews",
|
||||
"schedule": 86400.0, # retention: drop resolved review flags >30d
|
||||
|
||||
@@ -18,11 +18,18 @@ dark for that interval. Monitoring NEVER breaks the thing it's
|
||||
monitoring.
|
||||
"""
|
||||
|
||||
import functools
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from celery.exceptions import SoftTimeLimitExceeded
|
||||
from celery.signals import task_failure, task_postrun, task_prerun, task_retry
|
||||
from celery.signals import (
|
||||
task_failure,
|
||||
task_postrun,
|
||||
task_prerun,
|
||||
task_retry,
|
||||
worker_ready,
|
||||
)
|
||||
|
||||
from .models import TaskRun
|
||||
from .tasks._sync_engine import sync_session_factory
|
||||
@@ -53,42 +60,29 @@ _INT32_MIN = -2_147_483_648
|
||||
|
||||
|
||||
def _queue_for(task) -> str:
|
||||
"""Reverse the task→queue routing from celery_app.task_routes.
|
||||
Keep in sync if task_routes is reordered.
|
||||
"""The queue Celery routes this task to — asked of the router itself.
|
||||
|
||||
Audit 2026-06-02: backup/admin/library_audit prefixes were
|
||||
missing here even though task_routes sent all three to
|
||||
'maintenance'. The TaskRun.queue column then lied for those
|
||||
rows (claimed 'default') so per-queue dashboard filters and
|
||||
per-queue threshold overrides silently missed them.
|
||||
This was a hand-kept copy of `celery_app.task_routes`, and it drifted
|
||||
twice (the 2026-06-02 audit, then #4432). Long-lane jobs were recorded as
|
||||
`maintenance`, and translation and gpu_queue runs as `default`, where the
|
||||
5-minute stall sweep failed healthy 35-minute translation runs. The router
|
||||
answers from the same table the broker uses, so the two cannot disagree.
|
||||
"""
|
||||
name = getattr(task, "name", "") or ""
|
||||
if name.startswith("backend.app.tasks.import_file."):
|
||||
return "import"
|
||||
if name.startswith("backend.app.tasks.ml."):
|
||||
return "ml"
|
||||
if name.startswith("backend.app.tasks.thumbnail."):
|
||||
return "thumbnail"
|
||||
if name.startswith((
|
||||
"backend.app.tasks.download.",
|
||||
# External file-host fetches share the download lane (celery_app
|
||||
# routes external.* → download). Mirror it here or TaskRun.queue
|
||||
# lies 'default' for them, so per-queue dashboard filters and the
|
||||
# per-queue threshold override miss them — the same gap the
|
||||
# 2026-06-02 audit fixed for backup/admin/library_audit.
|
||||
"backend.app.tasks.external.",
|
||||
)):
|
||||
return "download"
|
||||
if name.startswith("backend.app.tasks.scan."):
|
||||
return "scan"
|
||||
if name.startswith((
|
||||
"backend.app.tasks.maintenance.",
|
||||
"backend.app.tasks.backup.",
|
||||
"backend.app.tasks.admin.",
|
||||
"backend.app.tasks.library_audit.",
|
||||
)):
|
||||
return "maintenance"
|
||||
return "default"
|
||||
app = getattr(task, "app", None)
|
||||
if app is None:
|
||||
from .celery_app import celery as app
|
||||
return _routed_queue(app, name)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=1024)
|
||||
def _routed_queue(app, name: str) -> str:
|
||||
try:
|
||||
queue = app.amqp.router.route({}, name).get("queue")
|
||||
except Exception: # noqa: BLE001 — monitoring never breaks the task
|
||||
log.warning("task_run: could not resolve the queue for %s", name)
|
||||
return "default"
|
||||
return getattr(queue, "name", None) or (queue if isinstance(queue, str) else "default")
|
||||
|
||||
|
||||
def _target_id_from_args(args) -> int | None:
|
||||
@@ -225,3 +219,43 @@ def _on_retry(sender=None, request=None, reason=None, einfo=None, **_):
|
||||
error_message=str(reason) if reason else None,
|
||||
retry_count=getattr(request, "retries", 0),
|
||||
)
|
||||
|
||||
|
||||
def _consumed_queues(consumer) -> set[str]:
|
||||
"""The queue names this worker process consumes (its `-Q`), or empty when
|
||||
they can't be read — which makes the boot hook below a no-op, not a guess."""
|
||||
try:
|
||||
return {q.name for q in consumer.task_consumer.queues}
|
||||
except Exception: # noqa: BLE001 — shape varies across celery versions
|
||||
return set()
|
||||
|
||||
|
||||
@worker_ready.connect
|
||||
def _on_worker_ready(sender=None, **_):
|
||||
"""The download lane clears what its previous process left behind (#4433).
|
||||
|
||||
A restart SIGKILLs any walk that outlives the stop grace, so its event is
|
||||
never finalized and its platform lock is never released. Both used to wait
|
||||
out timers — a 30-min stall sweep that then blamed the source, and a 27-min
|
||||
lock TTL that stalled every other source on the platform. Only the process
|
||||
consuming `download` does this; the other lanes booting beside it must not.
|
||||
Best-effort: a failure here is logged and the worker still starts.
|
||||
"""
|
||||
if "download" not in _consumed_queues(sender):
|
||||
return
|
||||
booted_at = datetime.now(UTC)
|
||||
try:
|
||||
from .services.download_recovery import interrupt_orphaned_download_events
|
||||
from .services.platform_lock import release_all_platform_locks
|
||||
|
||||
with sync_session_factory()() as session:
|
||||
closed = interrupt_orphaned_download_events(session, booted_at=booted_at)
|
||||
session.commit()
|
||||
released = release_all_platform_locks()
|
||||
if closed or released:
|
||||
log.info(
|
||||
"download lane boot: closed %d orphaned download event(s) as "
|
||||
"interrupted, released %d platform lock(s)", closed, released,
|
||||
)
|
||||
except Exception: # noqa: BLE001 — never block the worker from starting
|
||||
log.exception("download lane boot recovery failed")
|
||||
|
||||
@@ -238,13 +238,6 @@ class ImportSettings(Base):
|
||||
wip_title_tagging_enabled: Mapped[bool] = mapped_column(
|
||||
Boolean, nullable=False, default=True, server_default="true",
|
||||
)
|
||||
# Soft WIP title tier (#1474): also tag sketch/doodle/scribble titles, but with
|
||||
# a PROVISIONAL source (`wip_title_soft`) that never trains the head, since these
|
||||
# are lower-precision (a finished "sketch" isn't WIP). OFF by default — a lower-
|
||||
# precision tier is opt-in (the ring-loud audit surfaces false positives).
|
||||
wip_soft_title_tagging_enabled: Mapped[bool] = mapped_column(
|
||||
Boolean, nullable=False, default=False, server_default="false",
|
||||
)
|
||||
|
||||
@classmethod
|
||||
async def load(cls, session) -> ImportSettings:
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
"""Closing the download runs a worker restart orphaned (#4433).
|
||||
|
||||
Its own module, importing nothing but the model, because its caller is the
|
||||
worker boot hook in `celery_signals` — which every download task imports via
|
||||
`celery_app`. Living in `tasks.maintenance` put the whole maintenance import
|
||||
graph, the membership roster included, on the fetch path, which
|
||||
`test_gated_reason` forbids.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import literal, update
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
|
||||
from ..models import DownloadEvent
|
||||
|
||||
DOWNLOAD_INTERRUPTED_MESSAGE = (
|
||||
"interrupted by a worker restart — the next check picks it up where it left off"
|
||||
)
|
||||
|
||||
|
||||
def interrupt_orphaned_download_events(session, *, booted_at: datetime) -> int:
|
||||
"""Close the download events a restart orphaned, without blaming the source.
|
||||
|
||||
Called when the download lane comes up (#4433). Anything still
|
||||
pending/running from before this boot belongs to the previous process:
|
||||
a walk that outlived the 90s stop grace was SIGKILLed, and a queued or
|
||||
serialize-deferred task is held unacked until Redis redelivers it about an
|
||||
hour later. Left alone, the 30-min stall sweep would error each one and
|
||||
bump `consecutive_failures`, backing the source off as if the platform had
|
||||
failed.
|
||||
|
||||
Instead they end as `skipped` (terminal, not a failure) and the source is
|
||||
not touched: `last_checked_at` keeps its old value, so the next tick finds
|
||||
it due and the walk resumes from its checkpoint. A redelivered message
|
||||
that arrives later finds no pending event and opens a fresh one.
|
||||
|
||||
An event promoted to running after the boot has `started_at` reset to its
|
||||
real start (download_service), so it is never caught here. Does NOT commit.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
result = session.execute(
|
||||
update(DownloadEvent)
|
||||
.where(DownloadEvent.status.in_(["pending", "running"]))
|
||||
.where(DownloadEvent.started_at < booted_at)
|
||||
.values(
|
||||
status="skipped",
|
||||
finished_at=now,
|
||||
error=DOWNLOAD_INTERRUPTED_MESSAGE,
|
||||
metadata_=DownloadEvent.metadata_.op("||")(
|
||||
literal({"error_type": "interrupted"}, JSONB)
|
||||
),
|
||||
)
|
||||
.returning(DownloadEvent.id)
|
||||
)
|
||||
return len(result.all())
|
||||
@@ -49,10 +49,8 @@ from .audits import single_color
|
||||
from .link_extract import extract_external_links
|
||||
from .thumbnailer import Thumbnailer
|
||||
from .wip_title import (
|
||||
WIP_TITLE_SOFT_SOURCE,
|
||||
WIP_TITLE_SOURCE,
|
||||
apply_wip_image_tags,
|
||||
matches_soft_wip_title,
|
||||
matches_wip_title,
|
||||
resolve_wip_tag_id,
|
||||
)
|
||||
@@ -1040,9 +1038,7 @@ class Importer:
|
||||
removal sticks. The existing catalogue is covered separately by the
|
||||
operator-triggered backfill sweep. Gated by the settings toggle, and
|
||||
best-effort: any failure is logged, never allowed to fail the import."""
|
||||
hard_on = self.settings.wip_title_tagging_enabled
|
||||
soft_on = self.settings.wip_soft_title_tagging_enabled
|
||||
if not (hard_on or soft_on):
|
||||
if not self.settings.wip_title_tagging_enabled:
|
||||
return
|
||||
if record.primary_post_id is None:
|
||||
return
|
||||
@@ -1050,21 +1046,14 @@ class Importer:
|
||||
title = self.session.execute(
|
||||
select(Post.post_title).where(Post.id == record.primary_post_id)
|
||||
).scalar_one_or_none()
|
||||
# HARD tier ("WIP"/"work in progress") wins — higher precision, and it
|
||||
# trains the head; SOFT (sketch/doodle, #1474) is the provisional fallback
|
||||
# that never trains (source wip_title_soft).
|
||||
if hard_on and matches_wip_title(title):
|
||||
source = WIP_TITLE_SOURCE
|
||||
elif soft_on and matches_soft_wip_title(title):
|
||||
source = WIP_TITLE_SOFT_SOURCE
|
||||
else:
|
||||
if not matches_wip_title(title):
|
||||
return
|
||||
if self._wip_tag_id is _UNSET:
|
||||
self._wip_tag_id = resolve_wip_tag_id(self.session)
|
||||
if self._wip_tag_id is None:
|
||||
return
|
||||
apply_wip_image_tags(
|
||||
self.session, [record.id], self._wip_tag_id, source=source
|
||||
self.session, [record.id], self._wip_tag_id, source=WIP_TITLE_SOURCE
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 — a tag must never fail an import
|
||||
log.warning(
|
||||
@@ -1141,9 +1130,51 @@ class Importer:
|
||||
if post.artist_id is None:
|
||||
post.artist_id = artist.id
|
||||
self._apply_post_fields(post, sd)
|
||||
self._redate_post_images(post)
|
||||
self.session.commit()
|
||||
return True
|
||||
|
||||
def _redate_post_images(self, post: Post) -> None:
|
||||
"""Carry a post's date onto the images already linked to it (#4431).
|
||||
|
||||
The native ingesters import a post's media BEFORE its record: the
|
||||
per-media sidecar holds only the image identity (post-first, #856), and
|
||||
the date arrives with `_post.json`. So `_attach_provenance` links each
|
||||
image to a post that has no date yet, and the image keeps its download
|
||||
time. This runs when the record lands, and applies the same two rules
|
||||
`_attach_provenance` applies: `effective_date` is the PRIMARY post's
|
||||
date, and `earliest_post_date` is the earliest date across every post
|
||||
the image is linked to. Only rows that differ are written."""
|
||||
if post.post_date is None:
|
||||
return
|
||||
self.session.flush()
|
||||
self.session.execute(
|
||||
update(ImageRecord)
|
||||
.where(ImageRecord.primary_post_id == post.id)
|
||||
.where(ImageRecord.effective_date.is_distinct_from(post.post_date))
|
||||
.values(effective_date=post.post_date)
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
linked = select(ImageProvenance.image_record_id).where(
|
||||
ImageProvenance.post_id == post.id
|
||||
)
|
||||
earliest = (
|
||||
select(func.min(Post.post_date))
|
||||
.select_from(ImageProvenance)
|
||||
.join(Post, Post.id == ImageProvenance.post_id)
|
||||
.where(ImageProvenance.image_record_id == ImageRecord.id)
|
||||
.where(Post.post_date.is_not(None))
|
||||
.correlate(ImageRecord)
|
||||
.scalar_subquery()
|
||||
)
|
||||
self.session.execute(
|
||||
update(ImageRecord)
|
||||
.where(ImageRecord.id.in_(linked))
|
||||
.where(ImageRecord.earliest_post_date.is_distinct_from(earliest))
|
||||
.values(earliest_post_date=earliest)
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
|
||||
def attach_in_place(
|
||||
self,
|
||||
path: Path,
|
||||
|
||||
@@ -97,8 +97,8 @@ def _sigmoid(z, np):
|
||||
|
||||
def _conflict_scores(Xn, Wc, bc, np):
|
||||
"""The presentation conflict signal (#141): per row, the MAX content-head
|
||||
probability and WHICH head produced it. Shared by the system-tag sweep's guard-2
|
||||
and the soft-wip audit — both ask "does this ALSO look like real content?"."""
|
||||
probability and WHICH head produced it — the system-tag sweep's guard-2 asks
|
||||
"does this ALSO look like real content?"."""
|
||||
cprobs = _sigmoid(Xn @ Wc.T + bc, np)
|
||||
return cprobs.max(axis=1), cprobs.argmax(axis=1)
|
||||
|
||||
@@ -106,10 +106,9 @@ def _conflict_scores(Xn, Wc, bc, np):
|
||||
def _insert_presentation_review(
|
||||
session, *, image_record_id, tag_id, conflict_tag_id, conflict_score, mode,
|
||||
):
|
||||
"""Single-source the ring-loud PresentationReview row shape so the two writers
|
||||
(system-tag sweep guard-2 + soft-wip audit) can't drift on columns or `mode` —
|
||||
they share the (image_record_id, tag_id) composite PK, so a divergent `mode`
|
||||
would be a silent first-writer-wins bug."""
|
||||
"""Single-source the ring-loud PresentationReview row shape, so every writer of
|
||||
the (image_record_id, tag_id) composite PK agrees on columns and `mode` — a
|
||||
divergent `mode` would be a silent first-writer-wins bug."""
|
||||
session.execute(
|
||||
pg_insert(PresentationReview)
|
||||
.values(
|
||||
@@ -963,68 +962,6 @@ def system_tag_auto_apply_sweep(
|
||||
}
|
||||
|
||||
|
||||
def soft_wip_conflict_audit(session: Session, dry_run: bool = False) -> dict:
|
||||
"""Ring-loud audit for the SOFT WIP-title cohort (#1474). Images auto-tagged
|
||||
`wip` from a low-precision sketch/doodle title (source='wip_title_soft') that ALSO
|
||||
score >= the process conflict threshold on a content head are probably FINISHED
|
||||
art mis-tagged as process — flag them (PresentationReview, mode='process') so the
|
||||
review strip surfaces them ("also looks like <X>", Keep tag / Remove tag). Does
|
||||
NOT remove the tag; the operator decides. No-op when there are no content heads.
|
||||
numpy-only. Returns {n_scanned, n_flagged}."""
|
||||
import numpy as np
|
||||
|
||||
from ..wip_title import WIP_TITLE_SOFT_SOURCE, resolve_wip_tag_id
|
||||
|
||||
settings = _settings(session)
|
||||
ver = settings.embedder_model_version
|
||||
conflict_thr = float(settings.process_conflict_threshold)
|
||||
conf = _conflict_heads(session, ver)
|
||||
wip_id = resolve_wip_tag_id(session)
|
||||
if not conf or wip_id is None:
|
||||
return {"n_scanned": 0, "n_flagged": 0}
|
||||
Wc = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in conf])
|
||||
bc = np.asarray([r.bias for r in conf], dtype=np.float32)
|
||||
conf_tag_ids = [r.tag_id for r in conf]
|
||||
|
||||
soft_ids = [iid for (iid,) in session.execute(
|
||||
select(image_tag.c.image_record_id)
|
||||
.where(image_tag.c.tag_id == wip_id)
|
||||
.where(image_tag.c.source == WIP_TITLE_SOFT_SOURCE)
|
||||
)]
|
||||
# Skip images already flagged for this tag (idempotent re-runs).
|
||||
flagged = {iid for (iid,) in session.execute(
|
||||
select(PresentationReview.image_record_id)
|
||||
.where(PresentationReview.tag_id == wip_id)
|
||||
)}
|
||||
soft_ids = [i for i in soft_ids if i not in flagged]
|
||||
|
||||
n_flagged = 0
|
||||
scanned = 0
|
||||
for start in range(0, len(soft_ids), _AUTO_APPLY_CHUNK):
|
||||
chunk = soft_ids[start:start + _AUTO_APPLY_CHUNK]
|
||||
emb = _load_embeddings(session, chunk)
|
||||
cids = [i for i in chunk if i in emb]
|
||||
if not cids:
|
||||
continue
|
||||
scanned += len(cids)
|
||||
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
|
||||
max_c, arg_c = _conflict_scores(Xn, Wc, bc, np)
|
||||
for k in range(len(cids)):
|
||||
if float(max_c[k]) >= conflict_thr:
|
||||
n_flagged += 1
|
||||
if not dry_run:
|
||||
_insert_presentation_review(
|
||||
session,
|
||||
image_record_id=cids[k], tag_id=wip_id,
|
||||
conflict_tag_id=conf_tag_ids[int(arg_c[k])],
|
||||
conflict_score=float(max_c[k]),
|
||||
mode="process",
|
||||
)
|
||||
if not dry_run:
|
||||
session.commit()
|
||||
return {"n_scanned": scanned, "n_flagged": n_flagged}
|
||||
|
||||
|
||||
def retract_auto_applied_heads(session: Session) -> int:
|
||||
"""Soft auto-apply (milestone 139): re-score every standing source='head_auto'
|
||||
tag against its CURRENT head and REMOVE the ones now BELOW the head's
|
||||
|
||||
@@ -32,11 +32,8 @@ from ...models.tag import image_tag
|
||||
# `process_auto` (#1464): wip/editor screenshot applied by the process sweep are
|
||||
# ALSO provisional — the head must learn only from title (`wip_title`) + manual
|
||||
# labels, never its own auto-applied output, or it would runaway (operator 2026-07-12).
|
||||
# `wip_title_soft` (#1474): the soft title tier (sketch/doodle) is LOW-precision, so
|
||||
# it's provisional too — a finished piece titled "sketch" must not train the wip head.
|
||||
_AUTO_SOURCES = (
|
||||
"head_auto", "ccip_auto", "ml_auto", "presentation_auto", "process_auto",
|
||||
"wip_title_soft",
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -60,3 +60,22 @@ def platform_lock(platform: str, *, ttl_seconds: int):
|
||||
except redis.RedisError as exc: # pragma: no cover - broker outage
|
||||
log.warning("platform_lock unavailable for %s: %s", platform, exc)
|
||||
return None
|
||||
|
||||
|
||||
def release_all_platform_locks() -> int:
|
||||
"""Drop every serialized platform's lock. Returns how many were held.
|
||||
|
||||
Only for the download lane's boot (#4433). A worker that restarts mid-walk
|
||||
is SIGKILLed past its stop grace, so its `finally` never releases the lock,
|
||||
and the TTL keeps every other source on that platform bouncing for up to
|
||||
27 minutes after the new worker is ready. At boot no walk of ours can be
|
||||
running, so a held lock names a dead one. Assumes one download consumer —
|
||||
the only shape FC deploys; a second replica booting would free a live
|
||||
walk's lock (not corrupt it: the walk runs on, a second walk may overlap it).
|
||||
"""
|
||||
try:
|
||||
client = _redis()
|
||||
return int(client.delete(*(f"{_LOCK_PREFIX}{p}" for p in SERIALIZED_PLATFORMS)))
|
||||
except redis.RedisError as exc: # pragma: no cover - broker outage
|
||||
log.warning("could not release platform locks at boot: %s", exc)
|
||||
return 0
|
||||
|
||||
@@ -27,13 +27,10 @@ from .image_tag_apply import insert_image_tags
|
||||
|
||||
# image_tag.source stamped on title-heuristic WIP tags — distinct from the other
|
||||
# apply sources so provenance stays legible and a future undo can target only these.
|
||||
# HARD tier ("WIP"/"work in progress") is high-precision → trains the wip head.
|
||||
# Only the artist's own "WIP"/"work in progress" label counts — high-precision, so it
|
||||
# trains the wip head. A sketch/doodle/scribble tier (#1474) was retired in milestone
|
||||
# 430: a "sketch" is usually finished art, and its 6k tags flooded the review strip.
|
||||
WIP_TITLE_SOURCE = "wip_title"
|
||||
# SOFT tier (sketch/doodle/scribble, #1474) is LOWER-precision — a finished "sketch"
|
||||
# is often not WIP. This source is PROVISIONAL (in training_data._AUTO_SOURCES) so it
|
||||
# NEVER trains the wip head; a soft-tagged image that also looks like real content is
|
||||
# surfaced by the ring-loud audit for review.
|
||||
WIP_TITLE_SOFT_SOURCE = "wip_title_soft"
|
||||
|
||||
# A standalone "WIP" / "W.I.P" token, or the phrase "work in progress"
|
||||
# (space/underscore/hyphen separated). The letter-boundary lookarounds are what
|
||||
@@ -45,20 +42,10 @@ _WIP_RE = re.compile(
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
# Soft tier: sketch / doodle / scribble (+ plurals), letter-boundary anchored so
|
||||
# "sketchbook" / "kadoodle" don't trip it. Deliberately conservative — recall is
|
||||
# secondary because the soft source doesn't train the head and the ring-loud audit
|
||||
# catches false positives.
|
||||
_SOFT_WIP_RE = re.compile(
|
||||
r"(?<![A-Za-z])(?:sketch|sketches|doodle|doodles|scribble|scribbles)(?![A-Za-z])",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
# Coarse SQL prefilters for the backfill sweep — narrow the post scan to rows that
|
||||
# Coarse SQL prefilter for the backfill sweep — narrows the post scan to rows that
|
||||
# COULD match before the precise regex confirms. Case-insensitive ILIKE patterns.
|
||||
# Each MUST stay a SUPERSET of its regex or the sweep would silently miss posts.
|
||||
# It MUST stay a SUPERSET of the regex or the sweep would silently miss posts.
|
||||
WIP_TITLE_SQL_PREFILTER = ("%wip%", "%work%progress%")
|
||||
SOFT_WIP_TITLE_SQL_PREFILTER = ("%sketch%", "%doodle%", "%scribble%")
|
||||
|
||||
# Chunk bulk inserts so a large sweep can't blow past psycopg's 65535-parameter
|
||||
# ceiling (3 params/row → ~21k rows max; 5k stays comfortably under).
|
||||
@@ -66,19 +53,12 @@ _INSERT_CHUNK = 5000
|
||||
|
||||
|
||||
def matches_wip_title(title: str | None) -> bool:
|
||||
"""True when a post title explicitly marks it work-in-progress (HARD tier)."""
|
||||
"""True when a post title explicitly marks it work-in-progress."""
|
||||
if not title:
|
||||
return False
|
||||
return _WIP_RE.search(title) is not None
|
||||
|
||||
|
||||
def matches_soft_wip_title(title: str | None) -> bool:
|
||||
"""True when a title carries a SOFT WIP cue (sketch/doodle/scribble, #1474)."""
|
||||
if not title:
|
||||
return False
|
||||
return _SOFT_WIP_RE.search(title) is not None
|
||||
|
||||
|
||||
def resolve_wip_tag_id(session: Session) -> int | None:
|
||||
"""The seeded ``wip`` system tag's id (migration 0075), or None if absent."""
|
||||
return session.execute(
|
||||
|
||||
@@ -137,7 +137,13 @@ IMPORT_BATCH_KEEP_DAYS = 30
|
||||
# (the import queue itself stays at the 5-min default for single
|
||||
# files); time_limit=2100.
|
||||
QUEUE_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
|
||||
"ml": 25,
|
||||
# ml: the scheduled auto-apply sweeps and refresh_character_prototypes run
|
||||
# to a 35-min hard limit (2100s); 25 swept them mid-run (#4432). The two
|
||||
# 65-min jobs have their own entries below.
|
||||
"ml": 40,
|
||||
# import: import_media_file's hard limit is 6 min (360s), one past the
|
||||
# 5-min default this queue fell to (#4432).
|
||||
"import": 10,
|
||||
# download_source legitimately walks 5-25 min (Patreon/gallery-dl
|
||||
# deep creators); its hard time_limit is DOWNLOAD_HARD_TIME_LIMIT
|
||||
# (1500s = 25m). The 5-min default flagged healthy in-flight walks as
|
||||
@@ -154,6 +160,12 @@ QUEUE_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
|
||||
# overrides below cover the outliers (backups, library audit).
|
||||
"maintenance": 75,
|
||||
"scan": 75,
|
||||
# The long lane (#4432). Until TaskRun.queue asked the router, nothing was
|
||||
# recorded here: these runs read as `maintenance` (75) or, for
|
||||
# translation, `default` (5 — which failed healthy 35-min runs). The
|
||||
# longest task without its own entry below is the admin family at a
|
||||
# 40-min hard limit; 45 = 40 + 5.
|
||||
"maintenance_long": 45,
|
||||
}
|
||||
TASK_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
|
||||
"backend.app.tasks.import_file.import_archive_file": 40,
|
||||
@@ -179,6 +191,10 @@ TASK_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
|
||||
# external-fetch entry above — without an override a healthy in-flight walk
|
||||
# is swept 'RecoverySweep' at the bare 5-min default. 30 = 25 + 5.
|
||||
"backend.app.tasks.admin.reclaim_orphaned_attachments_task": 30,
|
||||
# Head training and the manual head apply run to 65 min (3900s) — past the
|
||||
# ml queue's threshold (#4432). 70 = 65 + 5.
|
||||
"backend.app.tasks.ml.train_heads": 70,
|
||||
"backend.app.tasks.ml.apply_head_tags": 70,
|
||||
}
|
||||
|
||||
|
||||
@@ -1041,8 +1057,7 @@ def cleanup_old_download_events() -> int:
|
||||
|
||||
def _backfill_wip_tier(session, tag_id, prefilter, matcher, source) -> int:
|
||||
"""One keyset-paginated pass over posts whose title matches a WIP tier, applying
|
||||
`tag_id` (stamped `source`) to their images. Shared by the hard + soft tiers
|
||||
(#1458 / #1474). Coarse `prefilter` (ILIKE superset) narrows the scan; the precise
|
||||
`tag_id` (stamped `source`) to their images (#1458). Coarse `prefilter` (ILIKE superset) narrows the scan; the precise
|
||||
`matcher` confirms. Idempotent-additive (ON CONFLICT DO NOTHING). Returns the row
|
||||
count newly applied."""
|
||||
from ..models import Post
|
||||
@@ -1082,26 +1097,17 @@ def _backfill_wip_tier(session, tag_id, prefilter, matcher, source) -> int:
|
||||
)
|
||||
def backfill_wip_title_tags() -> int:
|
||||
"""Scan EXISTING posts for WIP titles and apply the `wip` system tag to their
|
||||
images — the operator-triggered back-catalogue catch-up (task #1458 hard tier +
|
||||
#1474 soft tier). New imports are tagged live by the importer; this covers the
|
||||
existing library.
|
||||
|
||||
HARD tier ("WIP"/"work in progress") always runs (the operator triggered the
|
||||
scan); the SOFT tier (sketch/doodle, provisional source) runs only when
|
||||
wip_soft_title_tagging_enabled, AFTER hard so a title matching both keeps the
|
||||
trained hard tag (ON CONFLICT DO NOTHING). Keyset-paginated, restart-safe.
|
||||
images — the operator-triggered back-catalogue catch-up (task #1458). New
|
||||
imports are tagged live by the importer; this covers the existing library.
|
||||
Keyset-paginated, restart-safe.
|
||||
|
||||
Deliberately NOT scheduled as a beat: a periodic re-run would re-apply to matching
|
||||
posts and silently undo a manual WIP removal, so it stays an explicit operator
|
||||
action (Settings → "Scan existing posts for WIP titles"). Returns rows applied.
|
||||
"""
|
||||
from ..models import ImportSettings
|
||||
from ..services.wip_title import (
|
||||
SOFT_WIP_TITLE_SQL_PREFILTER,
|
||||
WIP_TITLE_SOFT_SOURCE,
|
||||
WIP_TITLE_SOURCE,
|
||||
WIP_TITLE_SQL_PREFILTER,
|
||||
matches_soft_wip_title,
|
||||
matches_wip_title,
|
||||
resolve_wip_tag_id,
|
||||
)
|
||||
@@ -1114,16 +1120,10 @@ def backfill_wip_title_tags() -> int:
|
||||
"backfill_wip_title_tags: no `wip` system tag present; nothing to do"
|
||||
)
|
||||
return 0
|
||||
settings = ImportSettings.load_sync(session)
|
||||
applied = _backfill_wip_tier(
|
||||
session, tag_id, WIP_TITLE_SQL_PREFILTER, matches_wip_title,
|
||||
WIP_TITLE_SOURCE,
|
||||
)
|
||||
if settings.wip_soft_title_tagging_enabled:
|
||||
applied += _backfill_wip_tier(
|
||||
session, tag_id, SOFT_WIP_TITLE_SQL_PREFILTER, matches_soft_wip_title,
|
||||
WIP_TITLE_SOFT_SOURCE,
|
||||
)
|
||||
if applied:
|
||||
log.info("backfill_wip_title_tags: applied wip to %d image(s)", applied)
|
||||
return applied
|
||||
|
||||
@@ -620,24 +620,6 @@ def scheduled_process_auto_apply() -> str:
|
||||
return f"applied={result['n_applied']} flagged={result['n_flagged']}"
|
||||
|
||||
|
||||
@celery.task(
|
||||
name="backend.app.tasks.ml.scheduled_soft_wip_conflict_audit",
|
||||
soft_time_limit=1800, time_limit=2100,
|
||||
)
|
||||
def scheduled_soft_wip_conflict_audit() -> str:
|
||||
"""Ring-loud audit over the SOFT WIP-title cohort (#1474) — flag sketch/doodle
|
||||
auto-tags that ALSO look like real content for review. No-op when there are no
|
||||
content heads; idempotent (already-flagged images skipped). Runs regardless of
|
||||
the process-sweep toggle, since soft-title tags come from the importer, not that
|
||||
sweep. Wall-clock bounded by the task time limits."""
|
||||
from ..services.ml.heads import soft_wip_conflict_audit
|
||||
|
||||
SessionLocal = _sync_session_factory()
|
||||
with SessionLocal() as session:
|
||||
result = soft_wip_conflict_audit(session)
|
||||
return f"scanned={result['n_scanned']} flagged={result['n_flagged']}"
|
||||
|
||||
|
||||
@celery.task(name="backend.app.tasks.ml.prune_presentation_reviews")
|
||||
def prune_presentation_reviews() -> str:
|
||||
"""Retention (rule 89): drop RESOLVED presentation-review flags older than 30
|
||||
|
||||
+4
-1
@@ -180,7 +180,10 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
|
||||
BUILD_REF` that every checkout in the file takes, rather than per job —
|
||||
otherwise `sign-extension` would derive dev's extension version while
|
||||
`build-web` bundled main's, and the release download would 404 on a version
|
||||
that exists perfectly well. Every job then ASSERTS its checkout is `main`
|
||||
that exists perfectly well. On every other trigger `BUILD_REF` is the
|
||||
triggering COMMIT (`github.sha`), not the branch: a branch is re-resolved
|
||||
per job, so a push landing mid-run used to move the publishing jobs onto a
|
||||
commit the run's lanes never tested (run 7499, #4427). Every job then ASSERTS its checkout is `main`
|
||||
before doing anything, because `env` inside `with:` is not a context this
|
||||
runner is known to evaluate — if it silently resolved to empty, checkout
|
||||
would fall back to the triggering ref and the refresh would publish dev's
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
<!-- System-tag auto-applies (chrome hides / process WIP tags) that ALSO looked
|
||||
like real content — surfaced PROACTIVELY atop the gallery whenever there's
|
||||
something to review (NOT gated on the Show-hidden toggle, so misfires can't
|
||||
go unnoticed), most-concerning first, with keep / remove (#141, #1464).
|
||||
go unnoticed), most-concerning first (#141, #1464). Each card asks one
|
||||
question — is this image a <tag>? — and its buttons answer it (#4424).
|
||||
Renders nothing when there's nothing to review. -->
|
||||
<section v-if="items.length" class="fc-review" aria-label="Auto-tagged images to review">
|
||||
<div class="fc-review__head">
|
||||
<v-icon size="18" color="warning">mdi-alert-outline</v-icon>
|
||||
<span class="fc-review__title">
|
||||
{{ items.length }} auto-tagged {{ items.length === 1 ? 'image' : 'images' }}
|
||||
may be real content — review
|
||||
{{ items.length }} {{ items.length === 1 ? 'auto-tag' : 'auto-tags' }} to check
|
||||
</span>
|
||||
</div>
|
||||
<div class="fc-review__cards">
|
||||
@@ -21,22 +21,23 @@
|
||||
class="fc-review-card__thumb" loading="lazy"
|
||||
>
|
||||
<div class="fc-review-card__body">
|
||||
<div class="fc-review-card__question">{{ question(it) }}</div>
|
||||
<div
|
||||
class="fc-review-card__conflict"
|
||||
:title="`Scored ${Math.round(it.conflict_score * 100)}% on “${it.conflict_name || 'a content tag'}”`"
|
||||
class="fc-review-card__reason"
|
||||
:title="reasonTitle(it)"
|
||||
>
|
||||
also looks like <strong>{{ it.conflict_name || 'content' }}</strong>
|
||||
also {{ Math.round(it.conflict_score * 100) }}%
|
||||
<strong>{{ it.conflict_name || 'content' }}</strong>
|
||||
</div>
|
||||
<div class="fc-review-card__tag">{{ tagLine(it) }}</div>
|
||||
<div class="fc-review-card__acts">
|
||||
<button
|
||||
type="button" class="fc-review-btn fc-review-btn--keep"
|
||||
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'keep')"
|
||||
>{{ keepLabel(it) }}</button>
|
||||
>Is {{ withArticle(it) }}</button>
|
||||
<button
|
||||
type="button" class="fc-review-btn fc-review-btn--unhide"
|
||||
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'unhide')"
|
||||
>{{ removeLabel(it) }}</button>
|
||||
>Is not {{ withArticle(it) }}</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -55,11 +56,18 @@ const items = ref([])
|
||||
const busy = ref([])
|
||||
|
||||
function keyOf(it) { return `${it.image_id}:${it.tag_id}` }
|
||||
// Chrome flags hide the image (keep-hidden / un-hide); process flags leave it
|
||||
// visible and just tagged (keep-tag / remove-tag). Same endpoints, different words.
|
||||
function tagLine(it) { return (it.mode === 'process' ? 'auto-tagged ' : 'hidden as ') + it.tag_name }
|
||||
function keepLabel(it) { return it.mode === 'process' ? 'Keep tag' : 'Keep hidden' }
|
||||
function removeLabel(it) { return it.mode === 'process' ? 'Remove tag' : 'Un-hide' }
|
||||
// The card asks whether the image IS the auto-applied system tag, and the buttons
|
||||
// answer that (operator, #4424: "is a <tag>" / "is not a <tag>"). "Is" keeps the
|
||||
// tag ('keep'); "Is not" removes it, un-hiding a chrome image ('unhide'). The
|
||||
// content tag it also scored on is the reason it was flagged, not the question.
|
||||
function noun(it) { return it.tag_name === 'wip' ? 'WIP' : it.tag_name }
|
||||
function withArticle(it) { return (/^[aeiou]/i.test(noun(it)) ? 'an ' : 'a ') + noun(it) }
|
||||
function question(it) { return `Is this ${withArticle(it)}?` }
|
||||
function reasonTitle(it) {
|
||||
const hidden = it.mode === 'process' ? '' : ' It is hidden from the gallery until you answer.'
|
||||
return `Auto-tagged “${it.tag_name}”, but it also scored ${Math.round(it.conflict_score * 100)}% `
|
||||
+ `on “${it.conflict_name || 'a content tag'}”, so it may be finished art.${hidden}`
|
||||
}
|
||||
|
||||
async function load() {
|
||||
// Fetched unconditionally on mount — the strip prompts for pending misfires
|
||||
@@ -77,14 +85,11 @@ async function resolve(it, action) {
|
||||
await api.post(`/api/gallery/hidden-review/${it.image_id}/${it.tag_id}/${action}`)
|
||||
items.value = items.value.filter((x) => keyOf(x) !== k)
|
||||
if (action === 'unhide') {
|
||||
const verb = it.mode === 'process' ? 'Removed' : 'Un-hidden'
|
||||
toast({ text: `${verb} — “${it.tag_name}” removed; it'll train the head`, type: 'success' })
|
||||
const shown = it.mode === 'process' ? '' : ', back in the gallery'
|
||||
toast({ text: `Not ${withArticle(it)} — “${it.tag_name}” removed${shown}; the tagger learns from it`, type: 'success' })
|
||||
}
|
||||
} catch (e) {
|
||||
toast({
|
||||
text: `Could not ${action === 'keep' ? 'keep hidden' : 'un-hide'}: ${e.message}`,
|
||||
type: 'error',
|
||||
})
|
||||
toast({ text: `Could not save your answer: ${e.message}`, type: 'error' })
|
||||
} finally {
|
||||
busy.value = busy.value.filter((x) => x !== k)
|
||||
}
|
||||
@@ -112,7 +117,7 @@ onMounted(load)
|
||||
display: flex; gap: 10px; overflow-x: auto; padding-bottom: 4px;
|
||||
}
|
||||
.fc-review-card {
|
||||
flex: 0 0 auto; width: 150px;
|
||||
flex: 0 0 auto; width: 170px;
|
||||
display: flex; flex-direction: column;
|
||||
border: 1px solid rgb(var(--v-theme-surface-light));
|
||||
border-radius: 6px; overflow: hidden;
|
||||
@@ -123,16 +128,16 @@ onMounted(load)
|
||||
background: rgb(var(--v-theme-surface-light));
|
||||
}
|
||||
.fc-review-card__body { padding: 6px 8px; }
|
||||
.fc-review-card__conflict {
|
||||
font-size: 11px; color: rgb(var(--v-theme-on-surface));
|
||||
.fc-review-card__question {
|
||||
font-size: 12px; font-weight: 600; color: rgb(var(--v-theme-on-surface));
|
||||
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
|
||||
}
|
||||
.fc-review-card__conflict strong { color: rgb(var(--v-theme-warning)); }
|
||||
.fc-review-card__tag {
|
||||
.fc-review-card__reason {
|
||||
font-size: 10px; color: rgb(var(--v-theme-on-surface-variant));
|
||||
margin: 1px 0 6px;
|
||||
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
|
||||
}
|
||||
.fc-review-card__reason strong { color: rgb(var(--v-theme-warning)); font-weight: 600; }
|
||||
.fc-review-card__acts { display: flex; gap: 4px; }
|
||||
.fc-review-btn {
|
||||
flex: 1; font-size: 11px; padding: 3px 4px; border-radius: 4px;
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
<v-icon start>mdi-folder-zip-outline</v-icon> Re-extract archives now
|
||||
</v-btn>
|
||||
<span v-if="queued" class="ml-3 text-caption text-success">Queued ✓</span>
|
||||
<QueueStatusBar queue="maintenance" queue-label="Maintenance" />
|
||||
<QueueStatusBar queue="maintenance_long" queue-label="Long maintenance" />
|
||||
</MaintenanceTile>
|
||||
</template>
|
||||
|
||||
|
||||
@@ -103,17 +103,6 @@
|
||||
the Explore browse. Applies to new imports; run the scan below to catch
|
||||
posts already in your library.
|
||||
</div>
|
||||
<v-switch
|
||||
v-model="local.wip_soft_title_tagging_enabled"
|
||||
label="Also tag “sketch” / “doodle” titles (lower precision)"
|
||||
density="compact" hide-details color="primary" @change="save"
|
||||
/>
|
||||
<div class="fc-help mb-3">
|
||||
Extends the above to softer cues (<code>sketch</code>, <code>doodle</code>,
|
||||
<code>scribble</code>). These stay <strong>visible</strong> and never train
|
||||
the tagging model — a daily audit flags any that actually look like finished
|
||||
art for review. Off by default.
|
||||
</div>
|
||||
<v-btn
|
||||
variant="tonal" color="primary" size="small"
|
||||
:loading="store.wipScanBusy" prepend-icon="mdi-magnify"
|
||||
@@ -156,7 +145,6 @@ const local = reactive({
|
||||
skip_single_color: false, single_color_threshold: 0.95,
|
||||
phash_threshold: 24,
|
||||
wip_title_tagging_enabled: true,
|
||||
wip_soft_title_tagging_enabled: false,
|
||||
})
|
||||
|
||||
watch(() => store.settings, (s) => { if (s) Object.assign(local, s) }, { immediate: true })
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
<v-icon start>mdi-file-remove-outline</v-icon> Repair missing-file records
|
||||
</v-btn>
|
||||
<span v-if="queued" class="ml-3 text-caption text-success">Queued ✓</span>
|
||||
<QueueStatusBar queue="maintenance" queue-label="Maintenance" />
|
||||
<QueueStatusBar queue="maintenance_long" queue-label="Long maintenance" />
|
||||
</MaintenanceTile>
|
||||
</template>
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@ const props = defineProps({
|
||||
|
||||
const QUEUE_NAMES = [
|
||||
'default', 'import', 'thumbnail', 'ml',
|
||||
'download', 'scan', 'maintenance',
|
||||
'download', 'scan', 'maintenance', 'maintenance_long',
|
||||
]
|
||||
|
||||
function formatDepth(name) {
|
||||
|
||||
@@ -119,7 +119,7 @@ export const useAdminStore = defineStore('admin', () => {
|
||||
// --- Task progress polling (taps FC-3i activity dashboard) --------
|
||||
|
||||
/**
|
||||
* Polls /api/system/activity/runs?queue=maintenance every 3s,
|
||||
* Polls /api/system/activity/runs?celery_task_id=<id> every 3s,
|
||||
* resolves when a task_run row with the given celery task_id
|
||||
* reaches a terminal status (ok / error / timeout). Returns the
|
||||
* row. Times out after 30 min by default.
|
||||
@@ -129,7 +129,9 @@ export const useAdminStore = defineStore('admin', () => {
|
||||
while (Date.now() < deadline) {
|
||||
const body = await api.get(
|
||||
'/api/system/activity/runs',
|
||||
{ params: { queue: 'maintenance', limit: 20 } },
|
||||
// By id, not by lane: these jobs run on `maintenance_long`, and a
|
||||
// lane filter here is one more copy of the routing table (#4432).
|
||||
{ params: { celery_task_id: taskId, limit: 1 } },
|
||||
)
|
||||
const row = (body.runs || []).find(r => r.celery_task_id === taskId)
|
||||
if (row && ['ok', 'error', 'timeout'].includes(row.status)) {
|
||||
|
||||
@@ -210,8 +210,8 @@ def render(
|
||||
)
|
||||
|
||||
parts.append(
|
||||
f"Built from `{short}`. The rollback unit is the immutable `:c-` tag "
|
||||
f"(rule 145) — these three move together:\n\n```\n"
|
||||
f"Built from `{short}`. To roll back to this release, pull these "
|
||||
f"immutable `:c-` tags — the images move together:\n\n```\n"
|
||||
+ "\n".join(f"{image}:c-{short}" for image in IMAGES)
|
||||
+ "\n```"
|
||||
)
|
||||
@@ -221,10 +221,10 @@ def render(
|
||||
# truncated to MAX_COMMITS, which is 200 lines of internal build-out
|
||||
# presented to someone who has never seen this project.
|
||||
parts.append(
|
||||
"---\n\n_First release under rule 148's `vYYYY.MM.DD.HHMM` shape, so "
|
||||
"there is no predecessor to diff against and no changelog to derive. "
|
||||
"The description above is README.md's, quoted at publish time. Later "
|
||||
"releases carry the commits since the previous one._"
|
||||
"---\n\n_The first release, so there is no earlier one to diff "
|
||||
"against and no changelog to derive. The description above is "
|
||||
"README.md's, quoted at publish time. Later releases carry the "
|
||||
"commits since the previous one._"
|
||||
)
|
||||
return "\n\n".join(parts)
|
||||
|
||||
@@ -257,8 +257,8 @@ def cross_checks(tag: str, sha: str) -> list[str]:
|
||||
|
||||
if not RULE_148.match(tag):
|
||||
notes.append(
|
||||
f"`{tag}` is not rule 148's `vYYYY.MM.DD.HHMM` shape. Published "
|
||||
f"anyway — the old `v26.*` tags predate the rule."
|
||||
f"`{tag}` is not the `vYYYY.MM.DD.HHMM` release-tag shape. "
|
||||
f"Published anyway — the old `v26.*` tags predate it."
|
||||
)
|
||||
else:
|
||||
derived = artifact_version("web")
|
||||
|
||||
@@ -295,3 +295,11 @@ async def test_failures_only_within_24h_window(client, _seed_failures):
|
||||
body = await resp.get_json()
|
||||
ids = {r["error_type"] for r in body["recent"]}
|
||||
assert "OldError" not in ids
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_runs_filter_by_celery_task_id(client, _seed_runs):
|
||||
# How a page follows a job it started, without knowing its lane (#4432).
|
||||
resp = await client.get("/api/system/activity/runs?celery_task_id=tid-3")
|
||||
body = await resp.get_json()
|
||||
assert [r["celery_task_id"] for r in body["runs"]] == ["tid-3"]
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
`maintenance_long` lane so a 30-min backup or a multi-chunk audit can't starve
|
||||
the quick recovery sweeps / vacuum on the concurrency-1 `maintenance` lane
|
||||
(operator-flagged 2026-06-07)."""
|
||||
import pytest
|
||||
|
||||
from backend.app.celery_app import celery
|
||||
|
||||
|
||||
@@ -20,23 +22,64 @@ def test_quick_maintenance_stays_on_maintenance():
|
||||
assert routes["backend.app.tasks.maintenance.*"]["queue"] == "maintenance"
|
||||
|
||||
|
||||
def test_queue_for_mirrors_external_to_download():
|
||||
"""celery_signals._queue_for is a hand-maintained mirror of task_routes
|
||||
that stamps TaskRun.queue. external.* routes to the download lane, so the
|
||||
mirror must agree — else TaskRun.queue lies 'default' for external fetches
|
||||
and per-queue dashboard filters / threshold overrides miss them
|
||||
(operator-flagged 2026-06-17)."""
|
||||
@pytest.mark.parametrize(("name", "queue"), [
|
||||
("backend.app.tasks.external.fetch_external_link", "download"),
|
||||
# The rows #4432 found recorded on the wrong lane:
|
||||
("backend.app.tasks.translation.translate_posts", "maintenance_long"),
|
||||
("backend.app.tasks.gpu_queue.enqueue_gpu_backfill", "maintenance"),
|
||||
("backend.app.tasks.maintenance.backfill_phash", "maintenance_long"),
|
||||
("backend.app.tasks.admin.normalize_tags_task", "maintenance_long"),
|
||||
("backend.app.tasks.backup.backup_db_task", "maintenance_long"),
|
||||
("backend.app.tasks.maintenance.vacuum_analyze", "maintenance"),
|
||||
("backend.app.tasks.not_routed.anything", "default"),
|
||||
])
|
||||
def test_task_run_records_the_queue_the_router_sends_to(name, queue):
|
||||
"""TaskRun.queue comes from the router, not a copy of task_routes (#4432):
|
||||
the stall sweep's per-queue thresholds and the System activity filters both
|
||||
key off it."""
|
||||
from backend.app.celery_signals import _queue_for
|
||||
|
||||
class _T:
|
||||
name = "backend.app.tasks.external.fetch_external_link"
|
||||
pass
|
||||
|
||||
assert _queue_for(_T()) == "download"
|
||||
assert (
|
||||
celery.conf.task_routes["backend.app.tasks.external.*"]["queue"]
|
||||
== "download"
|
||||
t = _T()
|
||||
t.name = name
|
||||
t.app = celery
|
||||
assert _queue_for(t) == queue
|
||||
|
||||
|
||||
def test_no_task_outlives_its_stall_threshold():
|
||||
"""A task whose hard time limit is longer than the stall sweep's threshold
|
||||
for it gets failed 'RecoverySweep' while it is still healthy — the class
|
||||
#4432 found on the ml, import and long-maintenance lanes. The threshold is
|
||||
resolved the way recover_stalled_task_runs resolves it: task-name override,
|
||||
then queue, then the default."""
|
||||
from backend.app.celery_signals import _queue_for
|
||||
from backend.app.tasks.maintenance import (
|
||||
QUEUE_STUCK_THRESHOLD_MINUTES,
|
||||
STUCK_THRESHOLD_MINUTES,
|
||||
TASK_STUCK_THRESHOLD_MINUTES,
|
||||
)
|
||||
|
||||
celery.loader.import_default_modules()
|
||||
checked = 0
|
||||
too_short = []
|
||||
for name, task in sorted(celery.tasks.items()):
|
||||
if name.startswith("celery."):
|
||||
continue
|
||||
limit = getattr(task, "time_limit", None)
|
||||
if not limit:
|
||||
continue
|
||||
threshold = TASK_STUCK_THRESHOLD_MINUTES.get(
|
||||
name,
|
||||
QUEUE_STUCK_THRESHOLD_MINUTES.get(_queue_for(task), STUCK_THRESHOLD_MINUTES),
|
||||
)
|
||||
checked += 1
|
||||
if limit / 60 > threshold:
|
||||
too_short.append(f"{name}: limit {limit / 60:.0f} min > sweep {threshold} min")
|
||||
assert checked > 20, "the task registry did not load — this guard checked nothing"
|
||||
assert not too_short, "\n".join(too_short)
|
||||
|
||||
|
||||
def test_backfill_phash_runs_on_the_long_lane():
|
||||
"""It lives in maintenance.py, so the quick-lane glob matches it too —
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
"""#4433: only the process consuming `download` clears a restart's leftovers.
|
||||
|
||||
The consolidated container boots four lanes side by side. If the scheduler or
|
||||
ml lane ran this too, a lane restarting on its own (supervisord restarts a
|
||||
crashed program) would close the events of walks that are still running.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from backend.app import celery_signals
|
||||
|
||||
|
||||
def _consumer(*queues: str):
|
||||
return SimpleNamespace(
|
||||
task_consumer=SimpleNamespace(queues=[SimpleNamespace(name=q) for q in queues])
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def calls(monkeypatch):
|
||||
seen: list[str] = []
|
||||
|
||||
class _Session:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
|
||||
def commit(self):
|
||||
seen.append("commit")
|
||||
|
||||
monkeypatch.setattr(celery_signals, "sync_session_factory", lambda: _Session)
|
||||
monkeypatch.setattr(
|
||||
"backend.app.services.download_recovery.interrupt_orphaned_download_events",
|
||||
lambda session, *, booted_at: seen.append("interrupt") or 0,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"backend.app.services.platform_lock.release_all_platform_locks",
|
||||
lambda: seen.append("release") or 0,
|
||||
)
|
||||
return seen
|
||||
|
||||
|
||||
def test_the_download_lane_clears_orphans_and_locks(calls):
|
||||
celery_signals._on_worker_ready(sender=_consumer("default", "download", "import"))
|
||||
assert calls == ["interrupt", "commit", "release"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("queues", [("maintenance", "scan"), ("ml",), ("maintenance_long",)])
|
||||
def test_other_lanes_leave_downloads_alone(calls, queues):
|
||||
celery_signals._on_worker_ready(sender=_consumer(*queues))
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_unreadable_queues_do_nothing_rather_than_guess(calls):
|
||||
celery_signals._on_worker_ready(sender=SimpleNamespace())
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_a_failure_does_not_stop_the_worker_starting(monkeypatch, calls):
|
||||
def boom(session, *, booted_at):
|
||||
raise RuntimeError("db down")
|
||||
|
||||
monkeypatch.setattr(
|
||||
"backend.app.services.download_recovery.interrupt_orphaned_download_events", boom
|
||||
)
|
||||
celery_signals._on_worker_ready(sender=_consumer("download")) # must not raise
|
||||
+70
-14
@@ -335,26 +335,30 @@ def test_recover_stalled_task_runs_ml_queue_uses_longer_threshold(db_sync):
|
||||
"""ml-queue tasks (embed_image video branch) legitimately run
|
||||
past the default 5-min threshold. The sweep must NOT flag an
|
||||
ml-queue task that's only been running 10 min — the override
|
||||
threshold (25 min via QUEUE_STUCK_THRESHOLD_MINUTES) protects
|
||||
in-flight video tagging. Operator-flagged 2026-05-28 after
|
||||
image 6288 (mp4) was marked failed at the 5-min tick mid-run."""
|
||||
threshold (QUEUE_STUCK_THRESHOLD_MINUTES["ml"]) protects in-flight
|
||||
video tagging. Operator-flagged 2026-05-28 after image 6288 (mp4)
|
||||
was marked failed at the 5-min tick mid-run. Read from the table
|
||||
rather than restated: the value moved 25 -> 40 in #4432."""
|
||||
from sqlalchemy import select
|
||||
|
||||
from backend.app.models import TaskRun
|
||||
from backend.app.tasks.maintenance import recover_stalled_task_runs
|
||||
from backend.app.tasks.maintenance import (
|
||||
QUEUE_STUCK_THRESHOLD_MINUTES,
|
||||
recover_stalled_task_runs,
|
||||
)
|
||||
|
||||
ml_threshold = QUEUE_STUCK_THRESHOLD_MINUTES["ml"]
|
||||
now = datetime.now(UTC)
|
||||
# 10-min-old ml-queue row: stale by the default 5-min rule but
|
||||
# fresh by the 25-min ml override. Must survive the sweep.
|
||||
# fresh by the ml override. Must survive the sweep.
|
||||
ml_fresh_id = _make_task_run(
|
||||
db_sync, status="running", queue="ml",
|
||||
started_at=now - timedelta(minutes=10),
|
||||
)
|
||||
# 30-min-old ml-queue row: past even the ml override. Must be
|
||||
# flagged.
|
||||
# Past even the ml override. Must be flagged.
|
||||
ml_stale_id = _make_task_run(
|
||||
db_sync, status="running", queue="ml",
|
||||
started_at=now - timedelta(minutes=30),
|
||||
started_at=now - timedelta(minutes=ml_threshold + 5),
|
||||
)
|
||||
db_sync.commit()
|
||||
|
||||
@@ -431,9 +435,10 @@ def test_download_stuck_threshold_exceeds_hard_time_limit():
|
||||
def test_recover_stalled_task_runs_archive_task_uses_longer_threshold(db_sync):
|
||||
"""import_archive_file shares the 'import' queue with fast
|
||||
single-file import_media_file, so it gets a per-task-name override
|
||||
(40 min) while the import queue stays at the 5-min default. A
|
||||
10-min-old archive task-run must survive; a 50-min-old one is
|
||||
flagged. Operator-flagged 2026-05-28."""
|
||||
(40 min) while the import queue keeps its short threshold (10 min
|
||||
since #4432; import_media_file's hard limit is 6). A 10-min-old
|
||||
archive task-run must survive; a 50-min-old one is flagged.
|
||||
Operator-flagged 2026-05-28."""
|
||||
from sqlalchemy import select
|
||||
|
||||
from backend.app.models import TaskRun
|
||||
@@ -441,12 +446,12 @@ def test_recover_stalled_task_runs_archive_task_uses_longer_threshold(db_sync):
|
||||
|
||||
archive_name = "backend.app.tasks.import_file.import_archive_file"
|
||||
now = datetime.now(UTC)
|
||||
# Fast single-file import on the same queue, 10 min old → flagged
|
||||
# by the default 5-min rule.
|
||||
# Fast single-file import on the same queue, 15 min old → flagged
|
||||
# by the import queue's 10-min threshold.
|
||||
media_id = _make_task_run(
|
||||
db_sync, status="running", queue="import",
|
||||
task_name="backend.app.tasks.import_file.import_media_file",
|
||||
started_at=now - timedelta(minutes=10),
|
||||
started_at=now - timedelta(minutes=15),
|
||||
)
|
||||
# Archive on the same queue, 10 min old → survives (40-min override).
|
||||
archive_fresh_id = _make_task_run(
|
||||
@@ -713,6 +718,57 @@ def test_recover_stalled_download_skips_fresh(db_sync):
|
||||
assert failures == 0
|
||||
|
||||
|
||||
def test_download_lane_boot_closes_pre_boot_events_without_blaming_the_source(db_sync):
|
||||
"""#4433: a restart SIGKILLs a walk past its stop grace and strands queued
|
||||
ones. At the download lane's boot, everything pending/running from before
|
||||
it ends as `skipped`/interrupted — and the source is left as it was, so it
|
||||
is due on the next tick instead of backed off as a failure."""
|
||||
from sqlalchemy import select
|
||||
|
||||
from backend.app.models import DownloadEvent, Source
|
||||
from backend.app.services.download_recovery import (
|
||||
DOWNLOAD_INTERRUPTED_MESSAGE,
|
||||
interrupt_orphaned_download_events,
|
||||
)
|
||||
|
||||
sid = _make_source(db_sync, slug="rebooted")
|
||||
booted_at = datetime.now(UTC)
|
||||
before = booted_at - timedelta(minutes=5)
|
||||
walking = DownloadEvent(
|
||||
source_id=sid, status="running", started_at=before,
|
||||
metadata_={"live": {"downloaded": 3}},
|
||||
)
|
||||
queued = DownloadEvent(source_id=sid, status="pending", started_at=before)
|
||||
finished = DownloadEvent(source_id=sid, status="ok", started_at=before)
|
||||
# Promoted to running after the boot: download_service resets started_at.
|
||||
fresh = DownloadEvent(
|
||||
source_id=sid, status="running", started_at=booted_at + timedelta(seconds=5),
|
||||
)
|
||||
db_sync.add_all([walking, queued, finished, fresh])
|
||||
db_sync.commit()
|
||||
|
||||
closed = interrupt_orphaned_download_events(db_sync, booted_at=booted_at)
|
||||
db_sync.commit()
|
||||
|
||||
assert closed == 2
|
||||
db_sync.expire_all()
|
||||
for ev in (walking, queued):
|
||||
assert ev.status == "skipped"
|
||||
assert ev.error == DOWNLOAD_INTERRUPTED_MESSAGE
|
||||
assert ev.finished_at is not None
|
||||
assert ev.metadata_["error_type"] == "interrupted"
|
||||
assert walking.metadata_["live"] == {"downloaded": 3}
|
||||
assert finished.status == "ok"
|
||||
assert fresh.status == "running"
|
||||
src = db_sync.execute(
|
||||
select(Source.consecutive_failures, Source.last_error, Source.last_checked_at)
|
||||
.where(Source.id == sid)
|
||||
).one()
|
||||
assert src.consecutive_failures == 0
|
||||
assert src.last_error is None
|
||||
assert src.last_checked_at is None
|
||||
|
||||
|
||||
def test_recover_stalled_download_flips_stale_pending(db_sync):
|
||||
"""A 2-hour-old pending event flips to error AND the source is bumped
|
||||
(consecutive_failures, last_error, last_checked_at) so the next scan
|
||||
|
||||
@@ -126,53 +126,17 @@ def test_process_sweep_flags_conflict_with_process_mode(db_sync):
|
||||
|
||||
|
||||
def test_process_auto_source_never_trains_head(db_sync):
|
||||
# The runaway break: provisional wip tags (process sweep 'process_auto', soft
|
||||
# title 'wip_title_soft') are NOT training positives; a HARD title-heuristic /
|
||||
# manual one IS. So the head learns only from trusted labels, never its own
|
||||
# output or the low-precision sketch/doodle tier (#1464 + #1474).
|
||||
# The runaway break: provisional wip tags (process sweep 'process_auto') are NOT
|
||||
# training positives; a title-heuristic / manual one IS. So the head learns only
|
||||
# from trusted labels, never its own output (#1464).
|
||||
wip = _system_tag(db_sync, "wip")
|
||||
auto_img = _img(db_sync, "f" * 64, _emb(0))
|
||||
soft_img = _img(db_sync, "9" * 64, _emb(2))
|
||||
title_img = _img(db_sync, "0" * 64, _emb(1))
|
||||
db_sync.execute(image_tag.insert().values(
|
||||
image_record_id=auto_img.id, tag_id=wip.id, source="process_auto"))
|
||||
db_sync.execute(image_tag.insert().values(
|
||||
image_record_id=soft_img.id, tag_id=wip.id, source="wip_title_soft"))
|
||||
db_sync.execute(image_tag.insert().values(
|
||||
image_record_id=title_img.id, tag_id=wip.id, source="wip_title"))
|
||||
db_sync.commit()
|
||||
positives = set(_ids_with_tag(db_sync, wip.id))
|
||||
assert title_img.id in positives # trusted HARD label trains the head
|
||||
assert auto_img.id not in positives # its own auto-applied output does NOT
|
||||
assert soft_img.id not in positives # low-precision soft tier does NOT
|
||||
|
||||
|
||||
def test_soft_wip_conflict_audit_flags_ring_loud(db_sync):
|
||||
# A soft-tagged image (sketch/doodle title) that ALSO scores high on a content
|
||||
# head is probably finished art mis-tagged — flagged for review; a quiet one is not.
|
||||
from backend.app.services.ml.heads import soft_wip_conflict_audit
|
||||
|
||||
s = db_sync.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
|
||||
s.process_conflict_threshold = 0.6
|
||||
wip = _system_tag(db_sync, "wip")
|
||||
content = Tag(name="looksreal", kind=TagKind.general)
|
||||
db_sync.add(content)
|
||||
db_sync.flush()
|
||||
_head(db_sync, content.id, 0, weight=1.0) # sigmoid(1)=0.73 > 0.6 conflict
|
||||
ring = _img(db_sync, "1" * 64, _emb(0)) # scores on the content head
|
||||
quiet = _img(db_sync, "2" * 64, _emb(5)) # orthogonal → 0.5 < 0.6
|
||||
for img in (ring, quiet):
|
||||
db_sync.execute(image_tag.insert().values(
|
||||
image_record_id=img.id, tag_id=wip.id, source="wip_title_soft"))
|
||||
db_sync.commit()
|
||||
|
||||
res = soft_wip_conflict_audit(db_sync)
|
||||
assert res["n_flagged"] == 1
|
||||
flag = db_sync.execute(
|
||||
select(PresentationReview).where(PresentationReview.image_record_id == ring.id)
|
||||
).scalar_one()
|
||||
assert flag.mode == "process"
|
||||
assert flag.conflict_tag_id == content.id
|
||||
assert db_sync.execute(
|
||||
select(PresentationReview).where(PresentationReview.image_record_id == quiet.id)
|
||||
).scalar_one_or_none() is None
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Migration 0113 (#4431): images linked to a post before its date arrived get
|
||||
the post's date back. Runs the migration's data step against real rows."""
|
||||
import importlib.util
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from backend.app.models import Artist, ImageProvenance, ImageRecord, Post
|
||||
from tests.factories import make_image as _img
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
_MIGRATION = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "alembic" / "versions" / "0113_redate_native_images.py"
|
||||
)
|
||||
|
||||
|
||||
def _redate():
|
||||
spec = importlib.util.spec_from_file_location("m0113", _MIGRATION)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod.redate_images
|
||||
|
||||
|
||||
def _post(db, artist, epid, when):
|
||||
p = Post(artist_id=artist.id, external_post_id=epid, post_date=when)
|
||||
db.add(p)
|
||||
db.flush()
|
||||
return p
|
||||
|
||||
|
||||
def _link(db, img, post, *, primary):
|
||||
db.add(ImageProvenance(image_record_id=img.id, post_id=post.id))
|
||||
if primary:
|
||||
img.primary_post_id = post.id
|
||||
db.flush()
|
||||
|
||||
|
||||
def test_redate_images_from_their_posts(db_sync):
|
||||
sent = datetime(2024, 3, 1, 18, 30, tzinfo=UTC)
|
||||
earlier = datetime(2023, 1, 5, tzinfo=UTC)
|
||||
artist = Artist(name="Alice", slug="alice")
|
||||
db_sync.add(artist)
|
||||
db_sync.flush()
|
||||
undated = _post(db_sync, artist, "u1", None)
|
||||
dated = _post(db_sync, artist, "d1", sent)
|
||||
repost = _post(db_sync, artist, "r1", earlier)
|
||||
|
||||
stale = _img(db_sync, "a" * 64) # primary post dated, image not
|
||||
reposted = _img(db_sync, "b" * 64) # also in an earlier post
|
||||
orphan = _img(db_sync, "c" * 64) # only an undated post
|
||||
_link(db_sync, stale, dated, primary=True)
|
||||
_link(db_sync, reposted, dated, primary=True)
|
||||
_link(db_sync, reposted, repost, primary=False)
|
||||
_link(db_sync, orphan, undated, primary=True)
|
||||
orphan_before = orphan.effective_date
|
||||
|
||||
_redate()(db_sync.connection())
|
||||
db_sync.expire_all()
|
||||
|
||||
stale = db_sync.get(ImageRecord, stale.id)
|
||||
reposted = db_sync.get(ImageRecord, reposted.id)
|
||||
orphan = db_sync.get(ImageRecord, orphan.id)
|
||||
assert stale.effective_date == sent
|
||||
assert stale.earliest_post_date == sent
|
||||
assert reposted.effective_date == sent # the primary post's date
|
||||
assert reposted.earliest_post_date == earlier # the earliest post's date
|
||||
assert orphan.effective_date == orphan_before # no date to take
|
||||
@@ -14,6 +14,7 @@ re-implementation of it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
@@ -162,6 +163,15 @@ def test_the_first_release_describes_the_product_instead_of_diffing(shaped_histo
|
||||
assert not [ln for ln in body.split("\n") if ln.startswith("- work landing")]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("tag", ["v2026.08.28.2208", "v2026.08.29.1000"])
|
||||
def test_the_release_page_cites_no_internal_rule_numbers(shaped_history, tag):
|
||||
"""The release page is read by strangers. "rule 145" names a record in the
|
||||
operator's own notes, which a reader cannot open — say what the rule means
|
||||
instead. Covers the first-release overview and the changelog body."""
|
||||
body = body_of(notes(tag, cwd=shaped_history))
|
||||
assert not re.search(r"\brule\s+\d+", body, re.IGNORECASE), body
|
||||
|
||||
|
||||
def test_the_overview_is_readmes_words_not_a_second_copy(shaped_history):
|
||||
"""Two hand-maintained descriptions of one product drift and nothing
|
||||
catches it. The release page quotes README.md so there is one source."""
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Migration 0112 (milestone 430, #4428): retiring the sketch/doodle WIP title tier.
|
||||
|
||||
Runs the migration's data step against real rows. The soft tags the operator stood
|
||||
behind (confirmed, or kept through the review strip) survive as `manual`; the rest go,
|
||||
and so do the unresolved review cards whose tag went with them. Hard title tags and
|
||||
their flags are untouched.
|
||||
"""
|
||||
import importlib.util
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
|
||||
from backend.app.models import PresentationReview, TagPositiveConfirmation
|
||||
from backend.app.models.tag import image_tag
|
||||
from backend.app.services.wip_title import resolve_wip_tag_id
|
||||
from tests.factories import make_image as _img
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
_MIGRATION = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "alembic" / "versions" / "0112_retire_soft_wip_title.py"
|
||||
)
|
||||
|
||||
|
||||
def _retire():
|
||||
spec = importlib.util.spec_from_file_location("m0112", _MIGRATION)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod.retire_soft_wip_tags
|
||||
|
||||
|
||||
def _source(db, image_id, tag_id):
|
||||
return db.execute(
|
||||
select(image_tag.c.source)
|
||||
.where(image_tag.c.image_record_id == image_id)
|
||||
.where(image_tag.c.tag_id == tag_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def _review(db, image_id, tag_id):
|
||||
return db.execute(
|
||||
select(PresentationReview)
|
||||
.where(PresentationReview.image_record_id == image_id)
|
||||
.where(PresentationReview.tag_id == tag_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def _flag(db, image_id, tag_id, *, resolved):
|
||||
db.add(PresentationReview(
|
||||
image_record_id=image_id, tag_id=tag_id, conflict_score=0.8, mode="process",
|
||||
resolved_at=datetime.now(UTC) if resolved else None,
|
||||
))
|
||||
|
||||
|
||||
def test_retire_soft_wip_keeps_human_judged_and_clears_the_rest(db_sync):
|
||||
wip = resolve_wip_tag_id(db_sync)
|
||||
plain = _img(db_sync, "a" * 64) # soft, never looked at → removed
|
||||
confirmed = _img(db_sync, "b" * 64) # soft + confirmed → kept as manual
|
||||
kept = _img(db_sync, "c" * 64) # soft + "Keep tag" in the strip → manual
|
||||
pending = _img(db_sync, "d" * 64) # soft + open card → tag and card removed
|
||||
hard = _img(db_sync, "e" * 64) # hard title tag + open card → untouched
|
||||
for img in (plain, confirmed, kept, pending):
|
||||
db_sync.execute(image_tag.insert().values(
|
||||
image_record_id=img.id, tag_id=wip, source="wip_title_soft"))
|
||||
db_sync.execute(image_tag.insert().values(
|
||||
image_record_id=hard.id, tag_id=wip, source="wip_title"))
|
||||
db_sync.add(TagPositiveConfirmation(image_record_id=confirmed.id, tag_id=wip))
|
||||
_flag(db_sync, kept.id, wip, resolved=True)
|
||||
_flag(db_sync, pending.id, wip, resolved=False)
|
||||
_flag(db_sync, hard.id, wip, resolved=False)
|
||||
db_sync.flush()
|
||||
|
||||
_retire()(db_sync.connection())
|
||||
db_sync.expire_all()
|
||||
|
||||
assert _source(db_sync, plain.id, wip) is None
|
||||
assert _source(db_sync, confirmed.id, wip) == "manual"
|
||||
assert _source(db_sync, kept.id, wip) == "manual"
|
||||
assert _source(db_sync, pending.id, wip) is None
|
||||
assert _source(db_sync, hard.id, wip) == "wip_title"
|
||||
|
||||
assert _review(db_sync, pending.id, wip) is None
|
||||
assert _review(db_sync, kept.id, wip) is not None # resolved history stays
|
||||
assert _review(db_sync, hard.id, wip) is not None # its tag is still on
|
||||
assert db_sync.execute(
|
||||
select(image_tag.c.image_record_id).where(image_tag.c.source == "wip_title_soft")
|
||||
).first() is None
|
||||
@@ -382,3 +382,36 @@ def test_external_links_not_duplicated_on_reimport(importer, import_layout):
|
||||
assert importer.session.execute(
|
||||
select(func.count()).select_from(ExternalLink)
|
||||
).scalar_one() == 1
|
||||
|
||||
|
||||
def test_post_record_redates_images_linked_before_it(importer, import_layout):
|
||||
"""#4431: the native ingesters import a message's media before its record,
|
||||
and only the record carries the date. The images start on their download
|
||||
time; when the record lands they take the post's date."""
|
||||
import_root, _ = import_layout
|
||||
artist = Artist(name="Alice", slug="alice")
|
||||
importer.session.add(artist)
|
||||
importer.session.flush()
|
||||
m = import_root / "Alice" / "20240301_123_01_art.jpg"
|
||||
_split(m, "v")
|
||||
_sidecar(m, {"category": "discord", "message_id": "123"})
|
||||
r = importer.import_one(m)
|
||||
assert r.status == "imported"
|
||||
rec = importer.session.get(ImageRecord, r.image_id)
|
||||
post = importer.session.execute(select(Post)).scalar_one()
|
||||
assert post.post_date is None
|
||||
download_time = rec.effective_date
|
||||
|
||||
sc = import_root / "Alice" / "20240301_123_post.json"
|
||||
sc.write_text(json.dumps({
|
||||
"category": "discord", "message_id": "123", "message": "",
|
||||
"date": "2024-03-01T18:30:00.000000+00:00",
|
||||
}))
|
||||
assert importer.upsert_post_record(sc, artist=artist) is True
|
||||
importer.session.expire_all()
|
||||
rec = importer.session.get(ImageRecord, r.image_id)
|
||||
post = importer.session.execute(select(Post)).scalar_one()
|
||||
assert post.post_date is not None
|
||||
assert post.post_date != download_time
|
||||
assert rec.effective_date == post.post_date
|
||||
assert rec.earliest_post_date == post.post_date
|
||||
|
||||
+1
-27
@@ -8,7 +8,7 @@ negative cases (substrings like ``swipe`` / ``wiped``) are the load-bearing ones
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from backend.app.services.wip_title import matches_soft_wip_title, matches_wip_title
|
||||
from backend.app.services.wip_title import matches_wip_title
|
||||
|
||||
|
||||
@pytest.mark.parametrize("title", [
|
||||
@@ -48,29 +48,3 @@ def test_matches_positive(title):
|
||||
])
|
||||
def test_matches_negative(title):
|
||||
assert matches_wip_title(title) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("title", [
|
||||
"quick sketch",
|
||||
"Nami sketch",
|
||||
"morning doodle",
|
||||
"some doodles",
|
||||
"sketches from today",
|
||||
"a little scribble",
|
||||
"SKETCH",
|
||||
])
|
||||
def test_soft_matches_positive(title):
|
||||
assert matches_soft_wip_title(title) is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("title", [
|
||||
None,
|
||||
"",
|
||||
"sketchbook tour", # 'sketch' inside sketchbook — must NOT match
|
||||
"kadoodle mascot", # 'doodle' mid-word
|
||||
"the final piece",
|
||||
"WIP", # a HARD cue is not a SOFT cue
|
||||
"prescribed colours", # 'scrib' inside prescribed — must NOT match
|
||||
])
|
||||
def test_soft_matches_negative(title):
|
||||
assert matches_soft_wip_title(title) is False
|
||||
|
||||
@@ -10,7 +10,6 @@ from backend.app.celery_app import celery
|
||||
from backend.app.models import Artist, ImageProvenance, ImageRecord, Post, Source
|
||||
from backend.app.models.tag import image_tag
|
||||
from backend.app.services.wip_title import (
|
||||
WIP_TITLE_SOFT_SOURCE,
|
||||
apply_wip_image_tags,
|
||||
resolve_wip_tag_id,
|
||||
)
|
||||
@@ -108,14 +107,3 @@ def test_backfill_tags_only_wip_titled_posts(db_sync):
|
||||
|
||||
# Idempotent: a second sweep finds the tag already present and applies nothing.
|
||||
assert backfill_wip_title_tags.apply().get() == 0
|
||||
|
||||
|
||||
def test_apply_soft_source_stamps_wip_title_soft(db_sync):
|
||||
# The soft tier (#1474) stamps a distinct provisional source.
|
||||
tag_id = resolve_wip_tag_id(db_sync)
|
||||
rec = _img(db_sync)
|
||||
db_sync.commit()
|
||||
assert apply_wip_image_tags(
|
||||
db_sync, [rec.id], tag_id, source=WIP_TITLE_SOFT_SOURCE
|
||||
) == 1
|
||||
assert _wip_source(db_sync, rec.id, tag_id) == "wip_title_soft"
|
||||
|
||||
Reference in New Issue
Block a user