6 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	7a09dc3cda	build: the agent installs one CUDA-13 stack instead of two, the web image drops ML packages it never imported, and Redis moves to 8 (1451, 1452) ... Agent: - The image ran PyPI's CUDA-13 torch 2.14 and onnxruntime-gpu 1.30 on a CUDA 12.9 cudnn-runtime base. requirements.txt had silently replaced the Dockerfile's torch 2.6+cu124, because ultralytics pulls torchvision, which pulls its own torch. That left ~3 GB of base libraries and a ~3 GB torch nothing loaded: 10 GB compressed. - Now: an nvidia/cuda 13.0.3 `base` image, with torch and torchvision installed together from cu130. CUDA and cuDNN come from the nvidia-* pip packages; onnxruntime-gpu declares its [cuda,cudnn] extras. - fc_agent/accel.py preloads those libraries for onnxruntime. It then logs, and reports in /status, whether torch and the ONNX CUDA provider actually got the GPU, since both fall back to the CPU silently. Web image: - Drop opencv-python-headless and onnxruntime, plus the opencv-only apt libs. Both have been listed since the scaffold and nothing in backend/ imports them. - torch/torchvision move to 2.14/0.29, and the unexplained caps are lifted (rule 154). Redis: 8-alpine in both compose files and both CI service containers. That gives an AGPLv3 licence option, where 7.4 was RSAL/SSPL only. The client moves to >=8.1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-24 17:45:39 -04:00
bvandeusenandClaude Opus 5	ffcd13096a	feat: one image for every lane, with the model fetch gated on enabling (4296) ... Milestone 422 step 6. Dockerfile.ml is gone; the main image carries torch, torchvision, transformers, onnxruntime and opencv, and serves every lane. WHY IT HAD TO MERGE: step 5 runs every lane in one process tree, so a second image would mean the `ml` lane could never be enabled from the UI — there would be no worker in that container to enable. The switch needs something to switch. THE MODEL NO LONGER DOWNLOADS AT BOOT. `entrypoint.sh`'s ml-worker role ran download_models before celery started, so every boot of that role reached HuggingFace for ~3.5GB — a startup dependency on a third party for a feature the operator may never use. Rule 164 permits a runtime fetch only for something "optional and clearly off", so the fetch is now a TASK, enqueued the moment the lane is ENABLED. Being a task is what makes it visible: it gets a TaskRun row, so the download shows in Activity with a duration and a status, and a failure is something an operator can see and retry rather than a container that quietly never became useful. Idempotent, so re-enabling a provisioned lane costs one no-op. Enqueued only when the lane actually came ON (`enabled is True`, not the resolved value) so re-saving slots does not re-fetch, and only when the consumer change landed — a task queued onto a queue nothing consumes would sit pending with no explanation. `fabledcurator-ml` KEEPS PUBLISHING, from the merged Dockerfile. The operator's Swarm stack references that name and lives outside this repo; dropping it would not break their deploy, it would freeze it silently at the last publish — the exact failure class this milestone keeps finding. Retiring the NAME is its own task, gated on that stack moving. Same two-phase shape #406 used for pixiv. THREE LIVE BREAKAGES from deleting the file, found by grepping for it rather than assuming the build was the only consumer: - `docker-compose.override.yml` built the ml service from it (contributor path would have failed at `docker compose build`). - `tests/test_artifact_paths.py` pins the ml path set. - `scripts/artifacts.sh` ML_PATHS named it. A path set naming a deleted file silently stops contributing to the derived revision — which the reuse check and the version string both read. That is #3202's recorded shape. The `--with-ml` flag is gone from the generator and the healthcheck rather than left defaulting to true. One image carries every lane now, so a flag that can only be passed one way is a branch pretending to be a choice. The advisory shipped in ecbd325 is what makes this honest to an adopter: the lane says it is optional, names the model, and gives its download and per-slot RAM before the switch is thrown. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-22 08:51:44 -04:00
bvandeusenandClaude Opus 4.8	6e3c5f697f	feat(ml): tag-eval backend — head-vs-centroid learning-curve eval (persisted) ... Slice 1 of milestone #114 (tagging v2). Proves the frozen-embedding + trained- head spine on the operator's own data, reusing the SigLIP embeddings already stored on image_record — no re-embedding, no GPU. Per concept: train a logistic-regression HEAD (positives + negatives = explicit rejections + sampled unlabeled) vs the old single-CENTROID baseline; report cross-validated precision/recall/AP for both, a LEARNING CURVE (AP/F1 as tagged positives grow 10→30→100→300), and example image ids (head-would-suggest / head-doubts-positive) to eyeball. Persisted so the report SURVIVES navigation (operator-flagged): the run + full report live in a new tag_eval_run row (mirrors library_audit_run); the admin card will rehydrate from GET on mount, not transient state. - models.TagEvalRun + migration 0056; runs on the ml queue (only worker with numpy/sklearn) — numpy/sklearn lazy-imported so the API can still enqueue. - services/ml/tag_eval (compute + start helper, one-running guard), tasks.ml .tag_eval_run, api/tag-eval (POST create, GET history light / detail w/ report). - recover_stalled_tag_eval_runs sweep + retention (keep last 20) + 5-min beat (rule 89). scikit-learn added to requirements-ml. - tests: param normalization + the rehydrate read-path + create/conflict. Frontend admin card (trigger + render persisted report) follows next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-27 22:49:10 -04:00
bvandeusen	681d7777f3	fix(build): install CPU-only torch in ml image (drops ~5.6GB CUDA layer)	2026-05-15 21:04:26 -04:00
bvandeusenandClaude Opus 4.7	0f47c7485b	chore(fc2a): catch up on stale dep pins — current versions across the board ... Audit against PyPI revealed almost every pin in requirements.txt and requirements-ml.txt was significantly behind. Most bumps are minor-major catch-up; a handful are major-version jumps. requirements.txt: - quart 0.19 -> 0.20 - hypercorn 0.16 -> 0.18 (now declares 3.14 support) - asyncpg 0.30 -> 0.31 - psycopg 3.2 -> 3.3 - alembic 1.13 -> 1.18 (5 minors stale) - pgvector 0.2 -> 0.4 (2 minors stale) - celery 5.4 -> 5.6 - redis 5.0 -> 7.4 (major jump) - cryptography 44 -> 48 - pillow 11.1 -> 12 (major jump; 12.x has 3.14 wheels) - gallery-dl 1.27 -> 1.32 (5 minors stale) - python-dotenv 1.0 -> 1.2 - structlog 24.1 -> 25.5 (major jump) (sqlalchemy 2.0 line is current. imagehash 4.3.2 was already in range.) requirements-ml.txt (not exercised by CI yet; FC-2b territory): - torch 2.2 -> 2.12 (10 minors stale) - torchvision 0.17 -> 0.27 (CAVEAT: excludes Python 3.14.1 specifically — inline comment added) - transformers 4.40 -> 5.8 (major) - onnxruntime 1.17 -> 1.26 - huggingface-hub 0.22 -> 1.14 (major) - opencv-python-headless 4.9 -> 4.13 The torchvision 3.14.1 exclusion is a real footgun — the python-ci runner pulls python:3.14-bookworm (latest patch); if that ever resolves to 3.14.1, the ml-worker image build will fail. FC-2b will exercise this path for the first time, so the constraint is documented inline rather than worked around now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-14 13:26:37 -04:00
bvandeusenandClaude Opus 4.7	13eaa35f1c	feat: scaffold backend Python project (Quart + SQLAlchemy + Celery deps) ... Pins runtime and ML deps separately so the regular web image stays lean. Configures ruff for py312 with bugbear, async, and pyupgrade lints enabled. psycopg sync driver included up-front for alembic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-14 07:31:39 -04:00