feat: worker lanes become rows — slots, a settable cap, a derived ceiling (4291)
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 5s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Failing after 32s
Build images / build-web (push) Successful in 58s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m45s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m13s
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 5s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Failing after 32s
Build images / build-web (push) Successful in 58s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m45s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m13s
Milestone 422 step 1. The data model the rest of the milestone reads. No
behaviour change: nothing consumes these rows yet, and every lane still boots
at its CELERY_CONCURRENCY env value.
Three numbers, not two, per the operator's distinction — the derived value is
a cap ON the cap:
slots <= slots_cap <= derived_ceiling
(live) (operator) (computed)
They can always lower their own cap; they cannot raise it past what the
container can hold. The ceiling is never stored, so a row written on a 32GB
host and later run in a 4GB container is bounded by the 4GB.
`services/worker_lanes.py` is the one place that knows the lane set.
`models/worker_lane.py` holds only what an operator may change.
Two deviations from the step as written, both deliberate:
QUEUES ARE NOT A COLUMN. The step body said the row carries its `-Q` list,
but a lane's queues are decided by celery_app's task_routes, not by
preference — an operator cannot move a backup off maintenance_long. Storing
them would create a row that can contradict the routing table, with nothing
to notice until a queue had no consumer. So queues are code, slots are data.
`test_every_routed_queue_has_a_lane_that_serves_it` reads the real routing
table and fails if a route is ever added without a lane.
ROLE_NAMES IS NOW DERIVED, not left alone. It was a hand-kept second copy of
"queue set -> display name" and had already drifted: maintenance_long is a
live lane with four task routes and a dedicated worker in the operator's
stack, and the roster did not know its name — so the System tab labelled it
`Worker (maintenance_long)`. Adding a lane table beside it would have made
three copies.
The ceiling honours cgroup limits rather than the host's. `os.cpu_count()`
reports the HOST's cores from inside a container, so a 4-core quota on a
32-core host would otherwise offer 32 slots — and the operator's own stack
sets `cpus: '4.0'` on ml-worker, so that is real configuration, not a
hypothetical. Memory reads cgroup v2 then v1, and recognises v1's
PAGE_SIZE-aligned LONG_MAX sentinel by magnitude rather than treating it as
petabytes.
Every uncertain case fails LOW. An unreadable limit yields UNKNOWN_CEILING,
never unlimited — not knowing how much memory there is must not read as
plenty. A box too small to hold one model beside the web process gets an ML
ceiling of 0 rather than a floor of 1: offering a slot that OOMs the
container the first time it is used is exactly what this exists to prevent.
ML_BYTES_PER_SLOT is 4 GiB and is UNMEASURED — flagged as such in the code,
with the method for replacing it with a real figure. It decides whether a
stranger's server survives enabling tagging, so it errs toward refusing a
slot that would have fitted.
Seeded one-of-each with ml at 0 and disabled (alembic 0103). ML off is step
6's requirement arriving early: enabling the lane is what triggers the SigLIP
download, and rule 164 permits a runtime fetch only for a feature that is
optional and clearly off. The seed values are literals rather than an import
of LANES — a migration is a statement about one moment, and importing the
live defaults would silently change what this revision does on a fresh
database in 2027.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -44,6 +44,7 @@ from .tag_head import TagHead
|
||||
from .tag_positive_confirmation import TagPositiveConfirmation
|
||||
from .tag_suggestion_rejection import TagSuggestionRejection
|
||||
from .task_run import TaskRun
|
||||
from .worker_lane import WorkerLane
|
||||
|
||||
__all__ = [
|
||||
"Base",
|
||||
@@ -94,4 +95,5 @@ __all__ = [
|
||||
"TagPositiveConfirmation",
|
||||
"TagSuggestionRejection",
|
||||
"TaskRun",
|
||||
"WorkerLane",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""worker_lane — how many slots the operator wants each worker lane to have.
|
||||
|
||||
Milestone 422 step 1. One row per lane in `services/worker_lanes.LANES`.
|
||||
|
||||
## What is NOT in here
|
||||
|
||||
The lane's queues and its display name. Those are decided by `celery_app.py`'s
|
||||
`task_routes` — an operator cannot move a backup off `maintenance_long` — so
|
||||
storing them would be a row that can contradict the routing table, with
|
||||
nothing to notice until a queue had no consumer. They live in
|
||||
`services/worker_lanes.py`; this table holds only what an operator may change.
|
||||
|
||||
The DERIVED CEILING is also absent, and that is deliberate rather than an
|
||||
omission. It is computed from the container's cgroup limits on every read, so
|
||||
a row written on a 32GB host and later run in a 4GB container is bounded by
|
||||
the 4GB — a stored ceiling would quietly authorise what the box can no longer
|
||||
hold.
|
||||
|
||||
## The three numbers
|
||||
|
||||
slots <= slots_cap <= derived_ceiling
|
||||
(live) (this row) (computed)
|
||||
|
||||
Operator's distinction, 2026-09-22: the derived value is *a cap on the cap*.
|
||||
`slots_cap` is theirs and is always lowerable; it simply may not exceed what
|
||||
the container can hold. The CHECK constraint below enforces the left half,
|
||||
which is a fact about the row; the right half is enforced at write, because
|
||||
it depends on a value no database column holds.
|
||||
|
||||
## enabled
|
||||
|
||||
Whether the lane consumes its queues at all. This is how ML ships off
|
||||
(milestone 422 step 6): `enabled=false` with `slots=0`, so a fresh install
|
||||
never loads a model or reaches HuggingFace, and turning tagging on in
|
||||
Settings is what triggers the fetch.
|
||||
|
||||
Not a substitute for `slots=0`. A lane can be enabled with zero slots while
|
||||
it is being resized, and the two answer different questions: `enabled` is
|
||||
intent, `slots` is capacity.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
DateTime,
|
||||
Integer,
|
||||
String,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from .base import Base
|
||||
|
||||
|
||||
class WorkerLane(Base):
|
||||
__tablename__ = "worker_lane"
|
||||
__table_args__ = (
|
||||
# Bare names — Base.metadata's naming convention prepends
|
||||
# ck_worker_lane_. Pre-prefixing here doubles it, which is what
|
||||
# alembic 0088 had to rename four constraints for (#3275).
|
||||
CheckConstraint("slots >= 0", name="slots_non_negative"),
|
||||
CheckConstraint("slots_cap >= 0", name="cap_non_negative"),
|
||||
# The invariant that makes the cap mean anything. Enforced in the
|
||||
# database rather than only in the service, because a row that
|
||||
# violates it is not a rejected request — it is a lane that will be
|
||||
# reconciled UP to a value the operator capped.
|
||||
CheckConstraint("slots <= slots_cap", name="slots_within_cap"),
|
||||
)
|
||||
|
||||
# The lane name from services/worker_lanes.LANES — never a container
|
||||
# hostname. See models/service_seen.py for why: celery's worker names here
|
||||
# are `celery@<container id>`, minted fresh on every deploy.
|
||||
name: Mapped[str] = mapped_column(String(32), primary_key=True)
|
||||
|
||||
slots: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
slots_cap: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False)
|
||||
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=False,
|
||||
server_default=func.now(),
|
||||
onupdate=func.now(),
|
||||
)
|
||||
Reference in New Issue
Block a user