docs(6209): wide net, startup, stamps, slippage, miss route and parity tests
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m23s

Eight more test files state the property they pin as a present-tense
fact. Incident narratives (a dated storage stall, a week of zero tuning
events, a two-project poisoned ledger, a twenty-site owner audit) become
the failure shape each test defends; one-install figures that name a rule
become a general worked example with the calibration numbers kept. The
family-grouping regression guard is renamed for what it checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-21 21:27:17 -04:00
co-authored by Claude Opus 5.5
parent d41e72d26c
commit ff08500128
8 changed files with 154 additions and 196 deletions
+20 -25
View File
@@ -1,16 +1,13 @@
"""A sick database must not stop the app from starting — #4181, rule 156.
"""A sick database must not stop the app from starting.
THE INCIDENT THESE GUARD, so a later reader knows what is being defended:
THE FAILURE THESE GUARD. A storage stall can make Postgres answer in minutes
instead of milliseconds. If the app starts into one, a startup read with no
deadline hangs; Hypercorn kills the worker at its 60-second lifespan timeout;
and nothing retries a failed lifespan. A short disk hiccup becomes an outage
that only a human restart can clear.
On 2026-09-19 a host storage stall made one Postgres checkpoint of 14 buffers
take 281 seconds against a 1.3-second baseline. The app restarted into the tail
of it; `get_maintenance_hour()` — the first database read in `before_serving` —
hung with no deadline; Hypercorn killed the worker at its 60-second lifespan
timeout; and nothing retries a failed lifespan. A five-minute disk hiccup became
a three-hour outage that only a human restart could clear.
Every assertion here is about the SHAPE that made that possible, not about the
stall, which no test can reproduce and no code can prevent:
Every assertion here is about the SHAPE that makes that possible, not about
the stall, which no test can reproduce and no code can prevent:
- the startup read returns on a schedule of its own rather than the
database's;
@@ -48,9 +45,9 @@ async def test_a_hanging_settings_read_does_not_hang_startup():
@pytest.mark.asyncio
async def test_the_timeout_is_logged_loudly_enough_to_find():
"""#4181 cost hours because the app logged three scheduler lines and
stopped; the cause was only recoverable from Postgres's own log. A WARNING
here is the breadcrumb that would have named it in seconds."""
"""Without a WARNING the app logs a few scheduler lines and stops, and the
cause is only recoverable from Postgres's own log. This is the breadcrumb
that names it."""
from scribe.services import db_maintenance_scheduler as sched
async def _never_returns(*_a, **_kw):
@@ -60,7 +57,7 @@ async def test_the_timeout_is_logged_loudly_enough_to_find():
patch.object(sched, "_STARTUP_READ_TIMEOUT", 0.05), \
patch.object(sched.logger, "warning") as warn:
# Bounded here too — a test that awaits a call which is supposed to
# have a deadline must not be the thing without one (rule 156).
# have a deadline must not be the thing without one.
await asyncio.wait_for(sched.get_maintenance_hour(), timeout=2)
assert warn.called
@@ -105,11 +102,9 @@ def test_the_startup_deadline_leaves_room_inside_the_lifespan_budget():
@pytest.mark.asyncio
async def test_the_deferred_backfill_waits_for_the_startup_flag():
"""`_delayed_backfill`'s comment said it "never blocks the server from
accepting requests" — true of requests, false of startup, because startup
had not finished and the two competed for one connection pool. Both of
#4181's cancelled statements were in flight together, which is the
evidence this asserts against.
""""Never blocks the server from accepting requests" is true of requests
and false of startup: a backfill that starts before startup finishes
competes with it for one connection pool.
Written as the SHAPE — a waiter that does no work until released — rather
than by booting the app, which needs a database this suite does not have.
@@ -133,7 +128,7 @@ async def test_the_deferred_backfill_waits_for_the_startup_flag():
@pytest.mark.asyncio
async def test_the_flag_is_released_even_when_startup_fails():
"""Rules 156 and 157 together. The waiter has no deadline of its own, so
"""The waiter has no deadline of its own, so
the release has to be unmissable: a `finally`, never a last line. Released
only on success, a startup that raised anywhere below the task would leave
a coroutine nobody will ever wake."""
@@ -164,7 +159,7 @@ def test_startup_releases_the_flag_in_a_finally():
finally_at = body.index("finally:")
assert finally_at < body.index("_startup_finished.set()"), (
"the flag is released after the work rather than in a finally — a "
"startup that raises would strand the backfill forever (rule 157)"
"startup that raises would strand the backfill forever"
)
assert "await _startup_finished.wait()" in src
@@ -184,7 +179,7 @@ def test_the_engine_bounds_how_long_a_connect_may_take():
from scribe.models import _CONNECT_ARGS
timeout = _CONNECT_ARGS.get("timeout")
assert timeout is not None, "a connect with no deadline (rule 156)"
assert timeout is not None, "a connect with no deadline"
assert 0 < timeout < 60
@@ -192,7 +187,7 @@ def test_no_blanket_command_timeout_was_added_with_it():
"""The deliberate omission, guarded so nobody adds it as an obvious
follow-up. `command_timeout` applies to EVERY statement, and this app runs
long ones on purpose — the embedding backfills, VACUUM ANALYZE. It would
trade #4181's failure mode for a worse one."""
trade a hung startup for killed maintenance."""
from scribe.models import _CONNECT_ARGS
assert "command_timeout" not in _CONNECT_ARGS
@@ -211,7 +206,7 @@ def test_the_engine_actually_uses_those_args():
def test_the_startup_guards_can_fail():
"""Rule 167: each assertion above has to be able to bite."""
"""Each assertion above has to be able to bite."""
from scribe.services.db_maintenance_scheduler import _STARTUP_READ_TIMEOUT
# An unbounded read is the thing being prevented.