fix: each lane needs its own celery node name, or three of four vanish (4295)

Found by the all-role smoke on its very first execution (run 7319), which is
the whole argument for having added it one commit ago.

Celery's default node name is `celery@<hostname>`. In the single-container
layout all four lanes share one hostname, so all four registered as the SAME
node. Celery says so itself:

    DuplicateNodenameWarning: Received multiple replies from node name:
    celery@72adc5b706a7

`inspect` collapses four replies into one dict key and the last one wins, so
three lanes read as absent — and WHICH three varies between calls:

    lanes not answering: maintenance_long, ml, worker
    lanes not answering: maintenance_long, scheduler, worker

Fatal twice over:

  * The composite healthcheck can never pass. In Swarm that is a container
    that never goes healthy — restart loop, then an automatic rollback of a
    deploy whose image was fine.
  * `pool_grow`/`pool_shrink` take a `destination` of node names. The UI dial
    and the autoscaler would have resized whichever lane happened to answer
    rather than the one asked for — silently, and differently each time.

Every celery role now starts with `-n "${CELERY_NODENAME:-celery}@%h"`, and
the generated supervisord config sets that per lane. The lanes become
worker@<cid>, scheduler@<cid>, maintenance_long@<cid>, ml@<cid> — distinct,
so inspect keeps four entries and `destination` addresses what it names.
`inspect_lanes_sync` maps hostname to lane by QUEUES, so nothing there
changes; it just stops having three of its four entries overwritten.

Unset, it falls back to `celery` — exactly celery's own default — so every
service in the multi-service stack is byte-identical to before, including the
`celery@$HOSTNAME` healthcheck in docker-compose.yml and in the operator's
Swarm stack.

The test asserts DISTINCTNESS across the whole lane table rather than a fixed
string per lane. The property that broke is that no two collide, and stating
it that way keeps holding when a lane is added.

This is the bug I said a live deploy was needed to find, found in CI instead
for the price of one `docker run` — and it would have met the operator as a
rollback loop on their first consolidated deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR

entrypoint.sh

@@ -6,6 +6,31 @@ set -euo pipefail
 # disagreement between them would only show up as `docker run --entrypoint`
 # behaving differently from `docker run`.
 ROLE="${1:-all}"
+
+# CELERY NODE NAME. Every celery role below starts with `-n $CELERY_NODENAME@%h`.
+#
+# Celery's default node name is `celery@<hostname>`, and in the single-
+# container layout all four lanes share one hostname — so all four registered
+# as the SAME node. celery's own words for it, observed on run 7319:
+#
+#   DuplicateNodenameWarning: Received multiple replies from node name:
+#   celery@72adc5b706a7
+#
+# `inspect` then collapses four replies into one dict key and the last one
+# wins, so three lanes read as absent and WHICH three varies per call:
+#
+#   lanes not answering: maintenance_long, ml, worker
+#   lanes not answering: maintenance_long, scheduler, worker
+#
+# That is fatal twice over. The composite healthcheck can never pass, so the
+# container is permanently unhealthy; and `pool_grow(destination=[hostname])`
+# addresses a lane BY that name, so the UI dial and the autoscaler would have
+# resized whichever lane happened to answer rather than the one asked for.
+#
+# The generated supervisord config sets this per lane. Unset — which is every
+# service in the multi-service stack — it falls back to `celery`, exactly
+# celery's own default, so `celery@$HOSTNAME` healthchecks there still work.
+: "${CELERY_NODENAME:=celery}"
 shift || true
 
 case "$ROLE" in
@@ -32,6 +57,7 @@ case "$ROLE" in
     CONCURRENCY="${CELERY_CONCURRENCY:-2}"
     echo "[entrypoint] Starting Celery worker queues=$QUEUES concurrency=$CONCURRENCY"
     exec celery -A backend.app.celery_app:celery worker \
+      -n "${CELERY_NODENAME:-celery}@%h" \
       --loglevel=info \
       -Q "$QUEUES" \
       --concurrency="$CONCURRENCY"
@@ -47,6 +73,7 @@ case "$ROLE" in
     CONCURRENCY="${CELERY_CONCURRENCY:-1}"
     echo "[entrypoint] Starting Celery beat+worker queues=$QUEUES concurrency=$CONCURRENCY"
     exec celery -A backend.app.celery_app:celery worker \
+      -n "${CELERY_NODENAME:-celery}@%h" \
       --beat \
       --loglevel=info \
       -Q "$QUEUES" \
@@ -68,6 +95,7 @@ case "$ROLE" in
     CONCURRENCY="${CELERY_CONCURRENCY:-1}"
     echo "[entrypoint] Starting ML Celery worker queues=$QUEUES concurrency=$CONCURRENCY"
     exec celery -A backend.app.celery_app:celery worker \
+      -n "${CELERY_NODENAME:-celery}@%h" \
       --loglevel=info \
       -Q "$QUEUES" \
       --concurrency="$CONCURRENCY"