Security values move into the Settings UI
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 9s
CI & Build / integration (push) Failing after 12s
CI & Build / Build & push image (push) Successful in 32s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 2m17s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 4m14s
Desktop (Tauri) / Update manifest (push) Successful in 5s
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 9s
CI & Build / integration (push) Failing after 12s
CI & Build / Build & push image (push) Successful in 32s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 2m17s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 4m14s
Desktop (Tauri) / Update manifest (push) Successful in 5s
Operator: *"proxy hops defaults to 1 and should be in the settings UI not in the envs, we need the security values to be in the UI."* Overrules the call I made yesterday, and rule 25 is on your side — I argued deployment-topology, but the operator has to be able to SEE what protects them, and reading a container's environment is not seeing. Six new settings in a **Security** group: trusted proxy hops (default 1), the per-account and per-address sign-in limits with their shared window, and the sign-up limit with its own. `THOUGHTSYNC_TRUSTED_PROXY_HOPS` is gone; the rate limits are no longer hardcoded constants. **The hard part was keeping the throttle cheap.** It consults these BEFORE opening a database connection — deliberately, because a refused attempt is meant to cost nothing, and the hop count is needed to know who is even asking. A query per attempt would undo both. So there is a small cache seeded from the registry defaults (the app works with no database at all, which is what the DB-free unit lane relies on), loaded at boot, and refreshed on every settings save — the same live-update contract `session_ttl_days` already had. `SlidingWindow` now takes its limit and window as SUPPLIERS rather than values, so a saved number applies to the next attempt instead of the next deploy. **Bounds are rejected, not clamped.** A hop count of 99 would trust anything a caller sent; a sign-in limit of 0 would lock every account out permanently. Both now fail validation with a message naming the range, and the number input carries min/max so the browser objects first. Silently storing a different number than the one typed is how somebody ends up believing a protection is set to something it is not. `MAX_BUCKETS` stays a constant on purpose: it protects the limiter from itself rather than the app from a caller, and there is no operator judgment to apply. Two integration tests, because the whole point is the round trip: a dangerous value refused, a legitimate one reaching the cache the throttle reads and persisting; and every Security row reaching the admin payload with bounds and a description that explains itself.
This commit is contained in:
@@ -21,7 +21,7 @@ from .notes import bp as notes_bp
|
||||
from .proxy import is_https
|
||||
from .retention import run_sweeper
|
||||
from .saved_filters import bp as saved_filters_bp
|
||||
from .settings import get_public_config, get_setting, load_or_create_secret_key
|
||||
from .settings import get_public_config, get_setting, load_or_create_secret_key, refresh_live
|
||||
from .settings_api import bp as settings_bp
|
||||
from .sync import bp as sync_bp, protocol_advertisement
|
||||
|
||||
@@ -94,6 +94,11 @@ def create_app() -> Quart:
|
||||
app.config["PERMANENT_SESSION_LIFETIME"] = timedelta(days=days)
|
||||
except (ValueError, TypeError, KeyError):
|
||||
pass
|
||||
# The security settings the throttle and the proxy trust read on hot
|
||||
# paths. Cached rather than queried per request; until this runs they
|
||||
# hold their registry defaults, which is the correct behaviour for a
|
||||
# server that has not finished starting.
|
||||
await refresh_live(db)
|
||||
# Expire old trash in the background (retention.py). One task per process is
|
||||
# correct because the image serves with a single hypercorn worker (Dockerfile);
|
||||
# if that ever gains `--workers`, this needs a lock so N workers don't each
|
||||
|
||||
@@ -15,9 +15,6 @@ class Config:
|
||||
If unset, a key is generated and persisted in the DB (see
|
||||
``thoughtsync.settings.load_or_create_secret_key``), so sessions survive
|
||||
restarts with no volume required.
|
||||
- ``THOUGHTSYNC_TRUSTED_PROXY_HOPS`` — how many proxies in front of this app may
|
||||
be believed when reading ``X-Forwarded-For`` / ``X-Forwarded-Proto``. Defaults
|
||||
to 1 (one reverse proxy terminating TLS). See ``trusted_proxy_hops``.
|
||||
|
||||
Uploaded media lives under ``DATA_DIR`` — a fixed, authoritative path
|
||||
(``/var/thoughtsync``), intentionally NOT configurable (a mutable data path only
|
||||
@@ -52,35 +49,3 @@ class Config:
|
||||
"""Optional break-glass override for the cookie-signing secret."""
|
||||
return os.environ.get("THOUGHTSYNC_SECRET_KEY") or None
|
||||
|
||||
@classmethod
|
||||
def trusted_proxy_hops(cls) -> int:
|
||||
"""How many proxies in front of this app may be believed.
|
||||
|
||||
`X-Forwarded-For` grows LEFT to RIGHT: each hop appends the address it saw.
|
||||
So the RIGHTMOST entry was written by our own proxy and is the address that
|
||||
actually connected to it, while anything a client sent arrives to the LEFT of
|
||||
that — which is why the leftmost entry, the "original client", is precisely
|
||||
the one a caller can forge.
|
||||
|
||||
With `n` trusted hops the real client is the nth entry from the right:
|
||||
|
||||
0 no proxy — ignore the header entirely, use the socket address
|
||||
1 one reverse proxy terminating TLS (the default, and this deployment)
|
||||
2 a CDN in front of that proxy — Cloudflare appended the client, our
|
||||
proxy appended Cloudflare
|
||||
|
||||
Set it to the number of proxies you actually run. Too HIGH and a caller can
|
||||
forge an address by padding the header; too low and everyone behind the CDN
|
||||
shares one bucket. Too low is the safe direction, so it is the fallback when
|
||||
the header is shorter than configured.
|
||||
|
||||
Env rather than the Settings UI (rule 25's "absolute bootstrap only" carve-
|
||||
out): it is a property of the deployment topology, not a preference, and the
|
||||
rate limiter consults it before opening a database connection — which is the
|
||||
whole point of checking a throttle before doing expensive work.
|
||||
"""
|
||||
raw = os.environ.get("THOUGHTSYNC_TRUSTED_PROXY_HOPS", "1")
|
||||
try:
|
||||
return max(0, int(raw))
|
||||
except (TypeError, ValueError):
|
||||
return 1
|
||||
|
||||
@@ -15,7 +15,7 @@ sent arrives to the LEFT of those.
|
||||
That inverts the intuitive reading. The leftmost entry is nominally "the original
|
||||
client" — and is exactly the one a caller can forge, by sending the header themselves.
|
||||
So we count in from the right by the number of proxies we actually run
|
||||
(`THOUGHTSYNC_TRUSTED_PROXY_HOPS`, default 1), and a forged prefix can never be
|
||||
(the **Trusted proxy hops** setting, default 1), and a forged prefix can never be
|
||||
selected no matter how much of it there is.
|
||||
|
||||
Too HIGH a hop count is the dangerous direction: it starts believing entries no proxy
|
||||
@@ -27,7 +27,7 @@ from __future__ import annotations
|
||||
|
||||
from quart import has_request_context, request
|
||||
|
||||
from .config import Config
|
||||
from .settings import live
|
||||
|
||||
|
||||
def trusted_entry(header: str, hops: int) -> str | None:
|
||||
@@ -54,7 +54,7 @@ def client_address() -> str:
|
||||
return forwarded_for(
|
||||
request.headers.get("X-Forwarded-For", ""),
|
||||
request.remote_addr,
|
||||
Config.trusted_proxy_hops(),
|
||||
live("trusted_proxy_hops"),
|
||||
)
|
||||
|
||||
|
||||
@@ -70,5 +70,5 @@ def is_https() -> bool:
|
||||
return False
|
||||
if request.is_secure:
|
||||
return True
|
||||
entry = trusted_entry(request.headers.get("X-Forwarded-Proto", ""), Config.trusted_proxy_hops())
|
||||
entry = trusted_entry(request.headers.get("X-Forwarded-Proto", ""), live("trusted_proxy_hops"))
|
||||
return (entry or "").lower() == "https"
|
||||
|
||||
@@ -23,7 +23,7 @@ came from, and either one can refuse it:
|
||||
addresses the attempts arrive from.
|
||||
- **The address** bounds the damage from one source spraying many accounts. It is
|
||||
read from ``X-Forwarded-For``, counting in from the RIGHT by
|
||||
``THOUGHTSYNC_TRUSTED_PROXY_HOPS`` so that only entries our own proxies wrote are
|
||||
the **Trusted proxy hops** setting so that only entries our own proxies wrote are
|
||||
believed — a forged header lands to the left of those and is never selected. It is
|
||||
still the weaker of the two keys, because it depends on that setting matching the
|
||||
deployment; the account key depends on nothing.
|
||||
@@ -36,29 +36,18 @@ from __future__ import annotations
|
||||
|
||||
import time
|
||||
from collections import deque
|
||||
from collections.abc import Callable
|
||||
|
||||
from .settings import live
|
||||
|
||||
|
||||
# Failed sign-ins tolerated per account before it stops answering, and for how long.
|
||||
# Ten is comfortably above a person mistyping a password and far below anything that
|
||||
# makes a dictionary worth running.
|
||||
ACCOUNT_LIMIT = 10
|
||||
ACCOUNT_WINDOW_S = 15 * 60
|
||||
|
||||
# Wider, because one address is legitimately many people: a household, an office
|
||||
# behind NAT, a phone on carrier-grade NAT.
|
||||
ADDRESS_LIMIT = 50
|
||||
ADDRESS_WINDOW_S = 15 * 60
|
||||
|
||||
# Registration is scarcer than a sign-in — it creates a row, and on a private
|
||||
# instance the honest number of accounts anyone needs to make is one.
|
||||
REGISTER_LIMIT = 5
|
||||
REGISTER_WINDOW_S = 60 * 60
|
||||
|
||||
# Never let the bookkeeping become the denial of service: an attacker rotating a
|
||||
# forged X-Forwarded-For could otherwise mint an unbounded number of buckets. Well
|
||||
# above any real deployment's distinct-caller count, so a legitimate instance never
|
||||
# reaches it; when it is reached the oldest buckets are dropped, which at worst
|
||||
# forgives some attempts.
|
||||
# Never let the bookkeeping become the denial of service: an attacker rotating an
|
||||
# address could otherwise mint an unbounded number of buckets. Well above any real
|
||||
# deployment's distinct-caller count, so a legitimate instance never reaches it; when
|
||||
# it is reached the oldest buckets are dropped, which at worst forgives some attempts.
|
||||
#
|
||||
# Not a setting: it protects the limiter from itself rather than the app from a
|
||||
# caller, and there is no operator judgment to apply to it.
|
||||
MAX_BUCKETS = 10_000
|
||||
|
||||
|
||||
@@ -68,13 +57,25 @@ class SlidingWindow:
|
||||
Sliding rather than a fixed window because a fixed one lets twice the limit
|
||||
through across a boundary — 10 at 14:59 and 10 at 15:00 — which for a login
|
||||
limiter is the difference between the number meaning something and not.
|
||||
|
||||
The limit and window are SUPPLIERS, not values, so an admin saving a new number in
|
||||
Settings takes effect on the next attempt instead of the next deploy. They are read
|
||||
per call, which is a dict lookup — the settings cache never touches the database.
|
||||
"""
|
||||
|
||||
def __init__(self, limit: int, window_s: float) -> None:
|
||||
self.limit = limit
|
||||
self.window_s = window_s
|
||||
def __init__(self, limit: Callable[[], int], window_s: Callable[[], float]) -> None:
|
||||
self._limit = limit
|
||||
self._window_s = window_s
|
||||
self._hits: dict[str, deque[float]] = {}
|
||||
|
||||
@property
|
||||
def limit(self) -> int:
|
||||
return self._limit()
|
||||
|
||||
@property
|
||||
def window_s(self) -> float:
|
||||
return self._window_s()
|
||||
|
||||
def _prune(self, key: str, now: float) -> deque[float]:
|
||||
hits = self._hits.get(key)
|
||||
if hits is None:
|
||||
@@ -112,9 +113,19 @@ class SlidingWindow:
|
||||
self._hits.clear()
|
||||
|
||||
|
||||
sign_in_by_account = SlidingWindow(ACCOUNT_LIMIT, ACCOUNT_WINDOW_S)
|
||||
sign_in_by_address = SlidingWindow(ADDRESS_LIMIT, ADDRESS_WINDOW_S)
|
||||
register_by_address = SlidingWindow(REGISTER_LIMIT, REGISTER_WINDOW_S)
|
||||
def _minutes(key: str) -> Callable[[], float]:
|
||||
return lambda: float(live(key)) * 60.0
|
||||
|
||||
|
||||
sign_in_by_account = SlidingWindow(
|
||||
lambda: live("signin_limit_per_account"), _minutes("signin_window_minutes")
|
||||
)
|
||||
sign_in_by_address = SlidingWindow(
|
||||
lambda: live("signin_limit_per_address"), _minutes("signin_window_minutes")
|
||||
)
|
||||
register_by_address = SlidingWindow(
|
||||
lambda: live("register_limit_per_address"), _minutes("register_window_minutes")
|
||||
)
|
||||
|
||||
|
||||
def reset_all() -> None:
|
||||
|
||||
+131
-1
@@ -19,6 +19,13 @@ class SettingDef:
|
||||
label: str
|
||||
description: str
|
||||
group: str
|
||||
# Ints only. Enforced server-side in validate_updates and passed to the UI so the
|
||||
# number input carries them too. These exist because several of the security
|
||||
# values have ranges where a typo is not merely wrong but dangerous — a proxy hop
|
||||
# count of 50 would trust anything a caller sent, and a sign-in limit of 0 would
|
||||
# lock every account out permanently.
|
||||
minimum: int | None = None
|
||||
maximum: int | None = None
|
||||
|
||||
|
||||
# The source of truth for every user-facing setting. Add a row here and it appears
|
||||
@@ -70,6 +77,80 @@ REGISTRY: list[SettingDef] = [
|
||||
"The server contacts the linked site; private/internal addresses are always blocked.",
|
||||
"Links",
|
||||
),
|
||||
# --- Security -----------------------------------------------------------------
|
||||
#
|
||||
# Read on paths too hot for a database round trip (the credential throttle checks
|
||||
# them BEFORE opening a connection, which is the point of checking a throttle
|
||||
# before doing expensive work), so they are cached — see `live()` below.
|
||||
SettingDef(
|
||||
"trusted_proxy_hops",
|
||||
"int",
|
||||
1,
|
||||
"Trusted proxy hops",
|
||||
"How many proxies sit in front of this server. 1 for a single reverse proxy "
|
||||
"terminating HTTPS; 2 if a CDN like Cloudflare sits in front of that; 0 if "
|
||||
"the app is exposed directly. This decides which entry of X-Forwarded-For is "
|
||||
"believed — set it TOO HIGH and a visitor can forge their own address and "
|
||||
"slip the sign-in limits below.",
|
||||
"Security",
|
||||
minimum=0,
|
||||
maximum=10,
|
||||
),
|
||||
SettingDef(
|
||||
"signin_limit_per_account",
|
||||
"int",
|
||||
10,
|
||||
"Failed sign-ins per account",
|
||||
"How many failures one account tolerates within the window before it stops "
|
||||
"answering. Comfortably above mistyping a password, far below anything that "
|
||||
"makes guessing worth attempting.",
|
||||
"Security",
|
||||
minimum=1,
|
||||
maximum=1000,
|
||||
),
|
||||
SettingDef(
|
||||
"signin_limit_per_address",
|
||||
"int",
|
||||
50,
|
||||
"Failed sign-ins per address",
|
||||
"The same, counted per visitor address instead of per account — it bounds one "
|
||||
"source trying many accounts. Wider, because one address is legitimately many "
|
||||
"people: a household, an office, a phone on carrier NAT.",
|
||||
"Security",
|
||||
minimum=1,
|
||||
maximum=10000,
|
||||
),
|
||||
SettingDef(
|
||||
"signin_window_minutes",
|
||||
"int",
|
||||
15,
|
||||
"Sign-in window (minutes)",
|
||||
"The trailing period both sign-in limits are counted over.",
|
||||
"Security",
|
||||
minimum=1,
|
||||
maximum=1440,
|
||||
),
|
||||
SettingDef(
|
||||
"register_limit_per_address",
|
||||
"int",
|
||||
5,
|
||||
"Sign-ups per address",
|
||||
"How many accounts one address may create within its window. Counted per "
|
||||
"attempt rather than per failure — each one is a row either way.",
|
||||
"Security",
|
||||
minimum=1,
|
||||
maximum=1000,
|
||||
),
|
||||
SettingDef(
|
||||
"register_window_minutes",
|
||||
"int",
|
||||
60,
|
||||
"Sign-up window (minutes)",
|
||||
"The trailing period the sign-up limit is counted over.",
|
||||
"Security",
|
||||
minimum=1,
|
||||
maximum=10080,
|
||||
),
|
||||
]
|
||||
|
||||
_BY_KEY: dict[str, SettingDef] = {d.key: d for d in REGISTRY}
|
||||
@@ -153,11 +234,52 @@ async def get_admin_settings(db) -> list[dict]:
|
||||
"label": d.label,
|
||||
"description": d.description,
|
||||
"group": d.group,
|
||||
"minimum": d.minimum,
|
||||
"maximum": d.maximum,
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
# Settings the app must be able to read WITHOUT awaiting a database.
|
||||
#
|
||||
# The credential throttle consults these before opening a connection — deliberately,
|
||||
# because a refused attempt is supposed to cost nothing, and the proxy hop count is
|
||||
# needed to know who is even asking. A per-request query would undo both.
|
||||
#
|
||||
# Seeded from the registry defaults so the app works before (and without) a database —
|
||||
# unit tests construct it with no Postgres at all — then refreshed from the DB at boot
|
||||
# and again whenever an admin saves. Same live-update contract `session_ttl_days`
|
||||
# already has in settings_api.py.
|
||||
_LIVE_KEYS = (
|
||||
"trusted_proxy_hops",
|
||||
"signin_limit_per_account",
|
||||
"signin_limit_per_address",
|
||||
"signin_window_minutes",
|
||||
"register_limit_per_address",
|
||||
"register_window_minutes",
|
||||
)
|
||||
|
||||
_live: dict[str, Any] = {k: _BY_KEY[k].default for k in _LIVE_KEYS}
|
||||
|
||||
|
||||
def live(key: str) -> Any:
|
||||
"""The cached value of a hot setting. Synchronous, never touches the database."""
|
||||
return _live[key]
|
||||
|
||||
|
||||
async def refresh_live(db) -> None:
|
||||
"""Re-read the hot settings into the cache. Called at boot and after every save."""
|
||||
for key in _LIVE_KEYS:
|
||||
_live[key] = await get_setting(db, key)
|
||||
|
||||
|
||||
def reset_live() -> None:
|
||||
"""Back to registry defaults. For tests — nothing in the app calls this."""
|
||||
for key in _LIVE_KEYS:
|
||||
_live[key] = _BY_KEY[key].default
|
||||
|
||||
|
||||
def validate_updates(updates: dict) -> tuple[dict, str | None]:
|
||||
"""Coerce/validate a {key: value} dict against the registry. Returns
|
||||
(clean_values, error_message). An unknown key or a bad int is rejected."""
|
||||
@@ -168,9 +290,17 @@ def validate_updates(updates: dict) -> tuple[dict, str | None]:
|
||||
return {}, f"unknown setting: {key}"
|
||||
if defn.type == "int":
|
||||
try:
|
||||
clean[key] = int(val)
|
||||
n = int(val)
|
||||
except (ValueError, TypeError):
|
||||
return {}, f"{defn.label} must be a whole number"
|
||||
# Rejected rather than clamped: silently accepting a number and storing a
|
||||
# different one is how somebody ends up believing a protection is set to
|
||||
# something it is not.
|
||||
if defn.minimum is not None and n < defn.minimum:
|
||||
return {}, f"{defn.label} must be at least {defn.minimum}"
|
||||
if defn.maximum is not None and n > defn.maximum:
|
||||
return {}, f"{defn.label} must be at most {defn.maximum}"
|
||||
clean[key] = n
|
||||
elif defn.type == "bool":
|
||||
clean[key] = _coerce_bool(val)
|
||||
else:
|
||||
|
||||
@@ -6,7 +6,7 @@ from quart import Blueprint, current_app, jsonify, request
|
||||
|
||||
from .auth import require_admin
|
||||
from .db import session_scope
|
||||
from .settings import get_admin_settings, set_settings, validate_updates
|
||||
from .settings import get_admin_settings, refresh_live, set_settings, validate_updates
|
||||
|
||||
bp = Blueprint("settings", __name__, url_prefix="/api/settings")
|
||||
|
||||
@@ -35,6 +35,10 @@ async def update_settings():
|
||||
async with session_scope() as db:
|
||||
await set_settings(db, clean)
|
||||
await db.commit()
|
||||
# Re-read the cached security values so a saved limit or hop count applies to
|
||||
# the very next request. Unconditional: cheap, and a conditional here would be
|
||||
# one more place that has to know which keys are hot.
|
||||
await refresh_live(db)
|
||||
result = await get_admin_settings(db)
|
||||
|
||||
# Apply the live-tunable knob without a restart (rule 25).
|
||||
|
||||
Reference in New Issue
Block a user