13 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	86abaf0b94	docs: a new install could not start, and nothing told anyone why (#3422) ... The install path milestone 328 wrote produces a web container that exits on boot. entrypoint.sh runs alembic, then app construction raises: MissingCredentialKey: Fernet key file not found at /images/secrets/credential_key.b64. For first-time setup, set CURATOR_BOOTSTRAP_NEW_KEY=1. That variable appeared in no README, no .env.example and no compose file — only in backend/. So a stranger following the documented steps got an app that does not start and an error with no context. Found by the milestone-362 smoke gate on its first real run (#3422). The product behaviour stays exactly as it is. credential_crypto refuses to mint a key because the 2026-06-02 audit found a partial restore — database back, ./images/secrets lost — silently generating a fresh one and producing a healthy-looking instance where every authenticated download failed AUTH_ERROR. Failing fast is right; not saying so is the bug. So: .env.example carries the variable in its own FIRST BOOT ONLY section with the reasoning and an instruction to delete the line afterwards, and README's First run leads with it, because "the app will not start" belongs before "the ML worker downloads weights". Both say to back up ./images/secrets/ alongside the database, which is the part that costs real data if it is learned late. **compose had to change too, and this is the part that would have shipped a second broken instruction.** A variable in `.env` is only used for ${...} interpolation — it does not reach the container unless the service names it. Telling people to set it in .env, without that, would have documented a step that does nothing. Added to the shared app_env anchor, defaulted to empty so the refusal still stands for everyone who has not opted in. Not taken: auto-bootstrapping when the credential table is empty, which would remove the manual step entirely and keep the audit's protection for restores. That is the better product and it is a code change with a predicate that has to be exactly right; this is the smallest correct fix, and #3422 stays open for the other one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 16:19:28 -04:00
bvandeusenandClaude Opus 5	3590c478f5	docs+ci: folder import stays retired, and fix a readiness probe that never probed ... Two unrelated things, both found while closing out milestone 328. **Folder import (#3367).** The operator's call, this session: the import-from-file surface was abandoned on purpose and is not coming back — "it has its own complexities that we didn't need." The README and the compose comment both described the missing button as a rough edge with a tracking issue, which promised a fix that is not coming. Both now say the retirement is the decision, name Subscriptions as the supported way to fill a new install, and describe /api/import/trigger as an unsupported escape hatch for anyone who wants to script one. **The CI readiness probe.** ci.yml's integration job and baseline.yml both waited for Postgres with `(echo > /dev/tcp/$PG_IP/5432)`. Those steps run under `sh -e` — act's default shell — where /dev/tcp is not a magic path but a filename that does not exist. The probe could therefore never succeed: run 18035, a GREEN run, spends 05:20:53 → 05:22:53 in that loop and exits it by exhaustion, not by connecting. Every integration run has been paying a flat 120s for a check that established nothing, and proceeding regardless. Replaced with a socket connect in python (present in the image, no package needed), and exhausting the budget is now a named failure instead of a silent fall-through — rule 156. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 00:51:25 -04:00
bvandeusenandClaude Opus 5	8a4af589f1	docs: write the install path for someone who is not the operator (#3271) ... FC has no login — no User model, no session auth, nothing. That was a deliberate call for a single-operator tool and it stays (operator, this session), but it was nowhere in the docs, and the app stores live Patreon / SubscribeStar / Pixiv session cookies on accounts that carry a payment method. Anyone standing this up from the README could reasonably have put it behind a TLS-terminating proxy and considered it handled. So the no-auth posture is now stated three times, in the three places someone decides where to bind the port: README has a "Before you expose it" section above the install instructions, .env.example explains why there is no auth variable in it, and the compose header says it before the first service. SECURITY.md claimed the opposite. It listed "a multi-user sharing ACL — instances can be shared" among the things worth protecting; there are no accounts to share between. That was rule 47 applied to a codebase that does not implement it, and it would have told a researcher FC holds a boundary it does not. Replaced with the real posture, including that TLS without an authenticating layer in front changes nothing. Also corrected, all of it stale rather than wrong-at-the-time: - EXTENSION_API_KEY was dead config. config.py read it into a field nothing consumed; the real key is generated into app_setting on first use and managed in the UI. Removed from config.py, compose and .env.example. - .env.example pointed at docs/superpowers/specs/… — there is no docs/ dir — and described the extension key as "lands in FC-3", closed 2026-05-21. - The /import mount comment described an FC-5 ImageRepo migration run from "Settings → Maintenance → Legacy migration", a surface with no frontend. - README said the extension installs from Settings → Maintenance. It is on Subscriptions → Settings. README is now split: running FC above the line, developing FC below it, with requirements, first run, the extension, upgrading and troubleshooting on the running side. First run documents the one real gap it found — a new installer with a library on disk has no button to import it, only POST /api/import/trigger, because the manual-scan UI was retired 2026-07-02 when that stopped mattering for an established install. Filed as #3367. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-09-02 00:16:23 -04:00
bvandeusenandClaude Opus 5	bc4eba636d	fix: the documented install path pulled :dev, not :latest (#3270) ... docker-compose.yml pinned :dev on all five app services — web, worker, scheduler, maintenance-long, ml-worker. The README documents `docker compose -f docker-compose.yml up -d` as the production path, and -f means "use only this file", skipping the override and its build: directives. So Compose pulled image:, and image: was the rolling development channel. The documented way to install this product shipped development builds. It went unnoticed for a structural reason rather than a careless one: nobody who works on the project takes that path. The operator deploys from a swarm stack file; contributors get docker-compose.override.yml, which sets build: for all five services, and build: wins over image:. The broken path is reachable only by a stranger following the README — which is exactly the audience that did not exist until now. :latest, per rule 147: main IS production. It is also what the agent stack (agent/docker-compose.yml) already pinned, so this makes the two stacks agree rather than introducing a new convention. Both paths verified with `docker compose config`, which merges and prints without starting anything: dev path — build: present on all five, image: not pulled -f production — 0 build: directives, five :latest images resolved Also checked the base file for anything a stranger could not satisfy: no host-absolute volume paths, no operator-specific port bindings, no device mappings. The tag was the only defect in the consumer path. The comment on web.image is deliberately long (rule 32). A line reading :latest inside a file a developer is debugging with is exactly the line someone flips back to :dev to test something and then commits, and the consequence — strangers silently installing bleeding edge — is invisible to everyone who works here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 16:55:43 -04:00
bvandeusenandClaude Opus 4.8	40cc11be5b	feat(deploy): container healthchecks + Swarm rolling-update auto-rollback ... web gets a /api/health liveness check; workers a lenient celery-ping check. A shared deploy policy (update_config order=start-first, failure_action=rollback, monitor 90s; rollback_config; restart_policy) means a bad image that never goes healthy is rolled back automatically instead of taking the service down. Ignored by plain `docker compose up` (deploy: is swarm-only), so the dev override is unaffected. Assumes prod deploys from this file via docker stack deploy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 22:15:33 -04:00
bvandeusenandClaude Opus 4.8	c64261593d	feat(ops): graceful shutdown — worker stop-grace + Interpreter drain resilience ... Deploys (docker SIGTERM→SIGKILL, default 10s) were killing Celery jobs mid-flight. Give in-flight work room to drain and make interrupted work resume cleanly instead of stalling. - docker-compose.yml: stop_grace_period per lane (web 30s / worker 90s / scheduler 60s / maintenance-long 180s / ml-worker 120s) so warm shutdown can actually drain before SIGKILL. - celery_app.py: task_reject_on_worker_lost=True — a task killed past the grace window is re-queued (safe: idempotent + chunked, recovery sweeps re-drive stragglers). - interpreter_client.py: map 429/5xx (502/503/504) → InterpreterUnavailable and parse Retry-After (delta-seconds or HTTP-date); a draining Interpreter behind a reverse proxy no longer raises an opaque HTTPError. - translation.py: thread retry_after out of _translate_batch; retranslate_posts resumes after the Retry-After hint (or 60s default, capped 900s) on an interrupt with _reset_done=True, self-terminating via the health gate. - tests: 429/5xx mapping + Retry-After parse; interrupt-resume + default backoff. No migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 21:01:00 -04:00
bvandeusenandClaude Opus 4.8	43b02d79a4	fix(infra): size Postgres /dev/shm via tmpfs mount (shm_size ignored under Swarm) ... The prod stack runs under Docker Swarm (docker stack deploy), which SILENTLY IGNORES `shm_size` — container inspect showed ShmSize still 64MB after the a183be7 fix, and vacuum_analyze kept hitting DiskFull resizing a ~64MB POSIX DSM segment in /dev/shm (operator-flagged 2026-06-07). Replace the ignored `shm_size: 512m` with a tmpfs mount on /dev/shm (size 512MB), which Swarm AND plain Compose both honor. Requires a stack redeploy to take effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-07 16:22:09 -04:00
bvandeusenandClaude Opus 4.8	68cda6114d	chore(compose): maintenance-long needs only /images; drop dead /downloads mounts ... Operator-flagged: /downloads was never mapped in prod and everything worked — confirmed nothing in the app references a filesystem /downloads (only the unrelated /api/downloads route). Dropped the dead mount from web/worker/ scheduler, and scoped the new maintenance-long worker to just /images (backups write to /images/_backups; audits + admin tasks all operate on /images). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-07 10:26:36 -04:00
bvandeusenandClaude Opus 4.8	c217009425	feat(maintenance): dedicated maintenance_long lane for long one-shot tasks ... Even chunked, a single concurrency-1 maintenance lane is fragile — a 30-min DB backup or a multi-chunk library audit holds the slot and delays the quick self-healing recovery sweeps / vacuum (operator-flagged 2026-06-07: long runs must never block quick maintenance). Route the long one-shots — backup.*, admin.* (normalize/re-extract/cascade- delete), library_audit.* — to a new `maintenance_long` queue served by a dedicated worker (concurrency 1), added to docker-compose (+ dev override). The scheduler keeps the quick `maintenance` lane (sweeps, vacuum, cleanup) for itself, so a backup can no longer starve a 5-min vacuum. UI queue list + routing tests updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-07 09:00:03 -04:00
bvandeusenandClaude Opus 4.8	a183be7e6e	fix(infra): bump Postgres shm_size (vacuum DiskFull) + raise DB-backup time limit ... Two more maintenance-queue failures from the operator's 24h list: - vacuum_analyze died with "could not resize shared memory segment to 67MB: No space left on device" — Docker's default /dev/shm is 64MB, too small for VACUUM (ANALYZE)'s parallel-worker shared memory. Set the postgres service shm_size: 512m. - backup_db_task timed out at its 12-min limit once the DB grew; a pg_dump can't be chunked, so raise it to 30/35 min. (A long backup still briefly holds the concurrency-1 lane — the structural fix is a dedicated lane for long one-shots.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-07 00:08:19 -04:00
bvandeusenandClaude Opus 4.7	26fd3f9f2c	fc5: docker-compose comment — legacy IR images bind-mount example at /import/imagerepo ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-22 09:05:12 -04:00
bvandeusenandClaude Opus 4.7	4aff9c557d	chore: docker compose ergonomics — rename dev override, bake dev defaults ... - Renamed docker-compose.dev.yml → docker-compose.override.yml so Docker Compose auto-merges it. `docker compose up` (no -f) now Just Works for local development. - Removed the `env_file: .env` requirement from every service in the base file. Operators no longer need to create a .env to bring the stack up. - Baked sane dev defaults directly into docker-compose.yml via ${VAR:-default} interpolation: DB_USER=fabledcurator DB_PASSWORD=fabledcurator_dev DB_NAME=fabledcurator SECRET_KEY=dev_secret_key_not_for_production_change_me LOG_LEVEL=INFO (overridden to DEBUG by the dev override) Defaults are insecure but explicitly named so. For production, override via shell env vars or a .env file at the project root. - README quick-start simplified to a single `docker compose up -d`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-14 18:27:25 -04:00
bvandeusenandClaude Opus 4.7	a281620178	feat: add docker-compose stack (web/worker/scheduler/ml-worker + postgres + redis) ... Prod compose references the Forgejo registry; dev compose builds locally and exposes DB/Redis ports. Volume layout: ./images, ./import, ./downloads, ./models (all gitignored, anchored to repo root). Plain HTTP only — no TLS in-app (spec §2.1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-14 07:42:36 -04:00