Milestone 422 step 5. `docker compose -f docker-compose.single.yml up -d`
gives three containers — FabledCurator, Postgres, Redis — where the stack
previously needed seven.
THE MULTI-SERVICE STACK IS KEPT. docker-compose.yml still runs the five app
services separately and remains the right shape for a Swarm deployment spread
across hosts, where per-service rolling rollback and placement constraints
matter. This adds a compose file; it deletes none.
`entrypoint.sh all` GENERATES the supervisord config from worker_lanes.LANES
and execs it as PID 1. Generated rather than checked in because a static
.conf would spell out each lane's -Q list, making a FIFTH hand-kept copy of
the queue names — after celery_app.task_routes and the three collapsed in
steps 1, 2 and 4. Every one of those had already drifted when found.
Generating gives a stronger guarantee than "they match today": a lane added
to LANES gets a process, and a queue cannot end up with no consumer because
someone missed a file.
supervisord over s6-overlay: one pip dependency on an image already Python,
with per-program stop timeouts and stopasgroup. The process-group part is not
a detail — celery's prefork pool forks children, and a TERM reaching only the
parent leaves them orphaned holding tasks. s6's advantage (PID-1 signal and
zombie handling) comes from `init: true` instead. Nothing in FC talks to the
supervisor, so the choice is reversible without touching product code.
FOUR LANES, NOT FIVE. The ml lane is skipped: torch and the ML requirements
live only in Dockerfile.ml until step 6 merges the images, so an `ml` program
here would fail to import on every restart forever. `--with-ml` is the flag
step 6 turns on.
THREE BUGS FOUND BY READING IT BACK, none of which the first tests caught:
1. `environment=CELERY_QUEUES=default,import,thumbnail,download` — supervisord
parses that key as a COMMA-separated list, so it reads as
CELERY_QUEUES=default plus three malformed entries and the worker lane
would have consumed only `default`. Silent: the worker starts, reports
healthy, never picks up an import. Now quoted, and the test asserts the
quoted form rather than the bare substring, which passed either way.
2. The generator emitted `entrypoint.sh <lane.name>`, but `maintenance_long`
is not a role — compose runs it as the plain `worker` role with different
queues. Lane now carries `entrypoint_role`, and a test reads entrypoint.sh
to assert every role a lane names actually exists.
3. The `scheduler` role hardcoded --concurrency=1, ignoring CELERY_CONCURRENCY.
Harmless while only compose started it and set none; with a generated value
being passed, the lane would have sat at 1 until the reconcile noticed,
with nothing saying why.
The healthcheck asserts BOTH halves — hypercorn answers and every configured
lane is answering the broker. That is the failure mode consolidation creates:
docker can no longer see the lanes as separate services, so a web-only check
would report a healthy container with every lane inside it dead. It
deliberately ignores the `enabled` flag: a disabled lane still has a running
process with its consumers cancelled, and marking the container unhealthy for
turning tagging off would be wrong.
stop_grace_period 200s, sized to the slowest lane (maintenance_long at 180s)
rather than the average, with a test asserting no program's stopwaitsecs can
exceed what compose allows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
Hygiene follow-up to the Camie retirement (#1189) — these were left inert to
bound that change; nothing reads them now. Migration 0068 drops:
- ml_settings: tagger_store_floor, tagger_model_version, suggestion_threshold_
character/general (already dead pre-retirement — scoring uses per-head
thresholds), video_min_tag_frames (only the deleted video-prediction
aggregator used it).
- image_record: tagger_model_version (no writer), centroid_scores (dead JSON
cache, no reader).
Also: ml_admin _EDITABLE/GET/_validate pruned (dropped the store-floor invariant
+ video_min_tag_frames check); MLThresholdSliders trimmed to a video-embedding
card (interval + max frames only); importer no longer resets the dropped cols;
download_models drops the Camie fetch; stale CASCADE comments in cleanup_service
no longer name the removed tables. Tests updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The HF repo Camais03/camie-tagger-v2 has camie-tagger-v2.onnx (789 MB)
+ camie-tagger-v2-metadata.json (7.77 MB) at root, NOT model.onnx +
selected_tags.csv. Tags ship as nested JSON (dataset_info.tag_mapping)
not CSV. Per the published onnx_inference.py reference: input is NCHW
not NHWC, normalize with ImageNet mean/std, pad-square color (124,116,
104), sigmoid the second output (refined predictions) not the first.
Operator hit this during the IR migration ML backfill — download_models
silently fetched only 3 json files (allow_patterns matched nothing
useful), tagger.load() then raised RuntimeError. Fetched the actual
v2 layout via WebFetch, rewrote tagger to match published reference.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
import_media_file now enqueues tag_and_embed alongside generate_thumbnail
after a successful import. scripts/download_models.py snapshots Camie +
SigLIP into /models, idempotent (skips when present). The ml-worker
entrypoint runs it before starting the Celery worker so a fresh /models
volume self-heals on first boot. Downloader tests are pure-logic (no
network in CI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusenandClaude Opus 5