fix(docker): widen memory byte columns to BIGINT (int32 overflow on ingest)
CI / lint (push) Successful in 3s
CI / unit (push) Successful in 46s
CI / integration (push) Successful in 2m22s
CI / publish (push) Successful in 1m6s

docker_metrics.mem_usage_bytes and docker_containers.mem_usage_bytes /
mem_limit_bytes were int4 (max 2,147,483,647). A container using >2.1 GB of RAM
(e.g. 8.18 GB) overflowed the column, so asyncpg raised "value out of int32
range" and the entire docker ingest batch failed — no metrics stored for any
host with a large container.

The I/O counters and the milestone-77 rollup/disk tables already used
BigInteger; this trio (docker_001-era) was missed.

- models.py: DockerMetric.mem_usage_bytes, DockerContainer.mem_usage_bytes,
  DockerContainer.mem_limit_bytes → BigInteger.
- migration docker_008_bigint_mem: ALTER COLUMN ... TYPE BIGINT (safe in-place
  int4→int8 promotion).
- integration regression test: persist an ~8 GB container, assert the
  current-state and time-series rows round-trip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Jg27rgypiW2efULXJDtMC
This commit is contained in:
2026-06-19 22:50:32 -04:00
parent c8b6719b37
commit 2545e8c6ce
3 changed files with 79 additions and 3 deletions
+40
View File
@@ -151,6 +151,46 @@ def test_persist_scopes_containers_by_host(app):
@_NEEDS_DB
def test_large_memory_values_persist_as_bigint(app):
"""A container using >2^31 bytes of RAM must persist. Regression: mem_usage_bytes
/ mem_limit_bytes were int4 and overflowed (asyncpg 'value out of int32 range'),
failing the whole ingest batch for any host with a >2.1 GB container."""
from sqlalchemy import text
from steward.models.hosts import Host
persist = _persist_fn(app)
now = datetime.now(timezone.utc)
big_usage = 8_185_077_760 # ~8.18 GB — well past int4 max (2_147_483_647)
big_limit = 17_179_869_184 # 16 GB
snapshot = [(now, [{
"name": "bigmem", "container_id": "deadbeef", "image": "postgres",
"status": "running", "cpu_pct": 1.0, "mem_pct": 50.0,
"mem_usage_bytes": big_usage, "mem_limit_bytes": big_limit,
"ports": [], "started_at": None,
}])]
async def _go():
async with app.db_sessionmaker() as s:
async with s.begin():
await s.execute(text("DELETE FROM docker_metrics"))
await s.execute(text("DELETE FROM docker_containers"))
h = Host(id=str(uuid.uuid4()), name="bigmemhost", address="10.0.0.9")
s.add(h)
await s.flush()
await persist(s, h, snapshot)
row = (await s.execute(text(
"SELECT mem_usage_bytes, mem_limit_bytes FROM docker_containers "
"WHERE name = 'bigmem'"))).first()
metric = (await s.execute(text(
"SELECT mem_usage_bytes FROM docker_metrics "
"WHERE container_name = 'bigmem'"))).scalar()
return row, metric
row, metric = asyncio.run(_go())
assert row == (big_usage, big_limit) # current-state row round-trips
assert metric == big_usage # time-series row round-trips
def test_lifecycle_events_derived_across_snapshots(app):
from sqlalchemy import text
from steward.models.hosts import Host