bvandeusenandClaude Opus 5.5
ad4f7d9232
Measured on dev (385a579), agent posts cut to five samples still dropped:
flush=8.8s for 990 rows, 6.6s for 680, 8.4s for 1000 — about 110 rows/s,
~9 ms a row, with rules and evaluation near zero. One random read per
row fits a random uuid4 primary key whose index no longer fits in memory:
every new row lands on a random page of it. (Inferred from the rate; the
table and index sizes have not been read.)
- steward.core.ids.time_ordered_id: a UUIDv7-layout string (48-bit ms
timestamp, then random), same 36-char shape as the stored uuid4 ids, so
no migration. Python's uuid7() arrives in 3.14; the image runs 3.13.
- Used for plugin_metrics and its hourly rollup, monitor_results,
audit_events, the docker metrics/hourly/events/logs tables, and the
traefik and unifi time series.
- MAX_SAMPLES_PER_INGEST 5 → 2 (~400 rows) for now: it fits the agent's
10 s even at the old rate, so hosts land data whichever way the new rate
comes out. Raise it once the new rate is measured.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
35 lines
1.3 KiB
Python
35 lines
1.3 KiB
Python
"""Time-series rows get time-ordered ids, so a new row lands at the right edge
|
|
of the primary-key index rather than on a random page of it (#5149)."""
|
|
import uuid
|
|
|
|
from steward.core import ids
|
|
from steward.core.ids import time_ordered_id
|
|
from steward.models.metrics import PluginMetric
|
|
|
|
|
|
def test_the_id_is_a_version_7_uuid_string():
|
|
raw = time_ordered_id()
|
|
u = uuid.UUID(raw)
|
|
assert len(raw) == 36 and str(u) == raw
|
|
assert u.version == 7 and u.variant == uuid.RFC_4122
|
|
|
|
|
|
def test_ids_sort_by_the_millisecond_they_were_made(monkeypatch):
|
|
made = []
|
|
for ms in (1_791_327_000_000, 1_791_327_000_001, 1_791_327_060_000):
|
|
monkeypatch.setattr(ids.time, "time_ns", lambda ms=ms: ms * 1_000_000)
|
|
made.append(time_ordered_id())
|
|
assert made == sorted(made)
|
|
assert len({m[:13] for m in made}) == 3 # the timestamp leads the string
|
|
|
|
|
|
def test_two_ids_in_one_millisecond_differ(monkeypatch):
|
|
monkeypatch.setattr(ids.time, "time_ns", lambda: 1_791_327_000_000 * 1_000_000)
|
|
assert time_ordered_id() != time_ordered_id()
|
|
|
|
|
|
def test_plugin_metrics_use_it():
|
|
# SQLAlchemy wraps a no-argument default as fn(context); call it as it would.
|
|
made = PluginMetric.__table__.c.id.default.arg(None)
|
|
assert uuid.UUID(made).version == 7
|