Security values move into the Settings UI
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 9s
CI & Build / integration (push) Failing after 12s
CI & Build / Build & push image (push) Successful in 32s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 2m17s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 4m14s
Desktop (Tauri) / Update manifest (push) Successful in 5s
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 9s
CI & Build / integration (push) Failing after 12s
CI & Build / Build & push image (push) Successful in 32s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 2m17s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 4m14s
Desktop (Tauri) / Update manifest (push) Successful in 5s
Operator: *"proxy hops defaults to 1 and should be in the settings UI not in the envs, we need the security values to be in the UI."* Overrules the call I made yesterday, and rule 25 is on your side — I argued deployment-topology, but the operator has to be able to SEE what protects them, and reading a container's environment is not seeing. Six new settings in a **Security** group: trusted proxy hops (default 1), the per-account and per-address sign-in limits with their shared window, and the sign-up limit with its own. `THOUGHTSYNC_TRUSTED_PROXY_HOPS` is gone; the rate limits are no longer hardcoded constants. **The hard part was keeping the throttle cheap.** It consults these BEFORE opening a database connection — deliberately, because a refused attempt is meant to cost nothing, and the hop count is needed to know who is even asking. A query per attempt would undo both. So there is a small cache seeded from the registry defaults (the app works with no database at all, which is what the DB-free unit lane relies on), loaded at boot, and refreshed on every settings save — the same live-update contract `session_ttl_days` already had. `SlidingWindow` now takes its limit and window as SUPPLIERS rather than values, so a saved number applies to the next attempt instead of the next deploy. **Bounds are rejected, not clamped.** A hop count of 99 would trust anything a caller sent; a sign-in limit of 0 would lock every account out permanently. Both now fail validation with a message naming the range, and the number input carries min/max so the browser objects first. Silently storing a different number than the one typed is how somebody ends up believing a protection is set to something it is not. `MAX_BUCKETS` stays a constant on purpose: it protects the limiter from itself rather than the app from a caller, and there is no operator judgment to apply. Two integration tests, because the whole point is the round trip: a dangerous value refused, a legitimate one reaching the cache the throttle reads and persisting; and every Security row reaching the admin payload with bounds and a description that explains itself.
This commit is contained in:
+4
-12
@@ -39,18 +39,10 @@ POSTGRES_PASSWORD=
|
|||||||
# 127.0.0.1 so only the proxy can talk to it.
|
# 127.0.0.1 so only the proxy can talk to it.
|
||||||
#THOUGHTSYNC_BIND=0.0.0.0
|
#THOUGHTSYNC_BIND=0.0.0.0
|
||||||
|
|
||||||
# How many proxies sit in front of this app. This is a SECURITY setting, not a
|
# NOTE: how many proxies sit in front of this app is a SETTING, not an env var —
|
||||||
# preference: it decides which entry of X-Forwarded-For is believed, and therefore
|
# Settings → Security → "Trusted proxy hops" in the admin UI. It defaults to 1 (one
|
||||||
# whether a caller can forge their own address and slip the per-address rate limit.
|
# reverse proxy terminating HTTPS) and belongs there because it is something you may
|
||||||
#
|
# need to change while the server is running, alongside the sign-in limits.
|
||||||
# 0 nothing in front — the app is directly exposed
|
|
||||||
# 1 one reverse proxy terminating TLS (the default, and the usual case)
|
|
||||||
# 2 a CDN in front of that proxy, e.g. Cloudflare
|
|
||||||
#
|
|
||||||
# Set it to the number you actually run. Too HIGH is the dangerous direction — it
|
|
||||||
# starts trusting entries no proxy of yours wrote. Too low just means callers share
|
|
||||||
# a rate-limit bucket.
|
|
||||||
#THOUGHTSYNC_TRUSTED_PROXY_HOPS=1
|
|
||||||
|
|
||||||
# How much the app says. Credential events (sign-ins, failures, throttles, new
|
# How much the app says. Credential events (sign-ins, failures, throttles, new
|
||||||
# accounts, device tokens issued) are logged at INFO and read with
|
# accounts, device tokens issued) are logged at INFO and read with
|
||||||
|
|||||||
@@ -41,9 +41,10 @@ Once a browser has seen HSTS from your hostname it will refuse plain HTTP there
|
|||||||
year, even if the header stops. That is the point of it, but it is worth knowing
|
year, even if the header stops. That is the point of it, but it is worth knowing
|
||||||
before you put a hostname behind TLS temporarily.
|
before you put a hostname behind TLS temporarily.
|
||||||
|
|
||||||
**3. Tell it how many proxies are in front of it.** `THOUGHTSYNC_TRUSTED_PROXY_HOPS`
|
**3. Tell it how many proxies are in front of it.** **Settings → Security → Trusted
|
||||||
defaults to `1` — one reverse proxy terminating TLS. Behind a CDN as well (Cloudflare
|
proxy hops**, which defaults to `1` — one reverse proxy terminating TLS. Behind a CDN
|
||||||
in front of your proxy) set it to `2`.
|
as well (Cloudflare in front of your proxy) set it to `2`. It applies immediately; no
|
||||||
|
restart.
|
||||||
|
|
||||||
This decides which entry of `X-Forwarded-For` is believed, and it is a security
|
This decides which entry of `X-Forwarded-For` is believed, and it is a security
|
||||||
setting rather than a preference. The header grows left to right as a request
|
setting rather than a preference. The header grows left to right as a request
|
||||||
@@ -74,9 +75,10 @@ docker run --rm -v thoughtsync-data:/d -v "$PWD":/out alpine tar czf /out/media.
|
|||||||
|
|
||||||
- **The credential endpoints are throttled.** `/api/auth/login`, `/api/auth/register`
|
- **The credential endpoints are throttled.** `/api/auth/login`, `/api/auth/register`
|
||||||
and `/api/auth/device-login` count attempts against both the account and the calling
|
and `/api/auth/device-login` count attempts against both the account and the calling
|
||||||
address, and answer `429` with a `Retry-After` once either is over budget — ten
|
address, and answer `429` with a `Retry-After` once either is over budget. The
|
||||||
failed sign-ins per account per fifteen minutes, five registrations per address per
|
numbers live in **Settings → Security** — ten failed sign-ins per account per
|
||||||
hour. The account-keyed limit is the one that holds when the address is forged.
|
fifteen minutes and five sign-ups per address per hour by default — and a change
|
||||||
|
applies to the next attempt rather than the next deploy. The account-keyed limit is the one that holds when the address is forged.
|
||||||
Checked *before* the password is verified, so a throttled attempt costs no bcrypt:
|
Checked *before* the password is verified, so a throttled attempt costs no bcrypt:
|
||||||
hashing is deliberately slow, and an unauthenticated caller who can trigger it
|
hashing is deliberately slow, and an unauthenticated caller who can trigger it
|
||||||
without limit has a CPU-exhaustion primitive as well as a guessing one.
|
without limit has a CPU-exhaustion primitive as well as a guessing one.
|
||||||
|
|||||||
@@ -12,6 +12,10 @@ interface SettingItem {
|
|||||||
label: string;
|
label: string;
|
||||||
description: string;
|
description: string;
|
||||||
group: string;
|
group: string;
|
||||||
|
// Ints only, and nullable: the server sends the registry's bounds so the number
|
||||||
|
// input can refuse an out-of-range value before the round trip.
|
||||||
|
minimum: number | null;
|
||||||
|
maximum: number | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
const config = useConfigStore();
|
const config = useConfigStore();
|
||||||
@@ -131,10 +135,16 @@ onMounted(load);
|
|||||||
:checked="Boolean(it.value)"
|
:checked="Boolean(it.value)"
|
||||||
@change="it.value = ($event.target as HTMLInputElement).checked"
|
@change="it.value = ($event.target as HTMLInputElement).checked"
|
||||||
/>
|
/>
|
||||||
|
<!-- min/max come from the registry. The server rejects out-of-range
|
||||||
|
values regardless — this is so the browser says so first, rather than
|
||||||
|
letting someone type a hop count that would disable a protection and
|
||||||
|
only learn about it from an error banner. -->
|
||||||
<input
|
<input
|
||||||
v-else-if="it.type === 'int'"
|
v-else-if="it.type === 'int'"
|
||||||
:id="it.key"
|
:id="it.key"
|
||||||
type="number"
|
type="number"
|
||||||
|
:min="it.minimum ?? undefined"
|
||||||
|
:max="it.maximum ?? undefined"
|
||||||
class="w-28 rounded-lg border border-neutral-300 bg-white px-3 py-2 text-sm text-neutral-900 shadow-sm focus:outline-none focus-visible:ring-2 focus-visible:ring-brand dark:border-neutral-700 dark:bg-neutral-800 dark:text-neutral-100"
|
class="w-28 rounded-lg border border-neutral-300 bg-white px-3 py-2 text-sm text-neutral-900 shadow-sm focus:outline-none focus-visible:ring-2 focus-visible:ring-brand dark:border-neutral-700 dark:bg-neutral-800 dark:text-neutral-100"
|
||||||
:value="Number(it.value)"
|
:value="Number(it.value)"
|
||||||
@input="it.value = Number(($event.target as HTMLInputElement).value)"
|
@input="it.value = Number(($event.target as HTMLInputElement).value)"
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ from .notes import bp as notes_bp
|
|||||||
from .proxy import is_https
|
from .proxy import is_https
|
||||||
from .retention import run_sweeper
|
from .retention import run_sweeper
|
||||||
from .saved_filters import bp as saved_filters_bp
|
from .saved_filters import bp as saved_filters_bp
|
||||||
from .settings import get_public_config, get_setting, load_or_create_secret_key
|
from .settings import get_public_config, get_setting, load_or_create_secret_key, refresh_live
|
||||||
from .settings_api import bp as settings_bp
|
from .settings_api import bp as settings_bp
|
||||||
from .sync import bp as sync_bp, protocol_advertisement
|
from .sync import bp as sync_bp, protocol_advertisement
|
||||||
|
|
||||||
@@ -94,6 +94,11 @@ def create_app() -> Quart:
|
|||||||
app.config["PERMANENT_SESSION_LIFETIME"] = timedelta(days=days)
|
app.config["PERMANENT_SESSION_LIFETIME"] = timedelta(days=days)
|
||||||
except (ValueError, TypeError, KeyError):
|
except (ValueError, TypeError, KeyError):
|
||||||
pass
|
pass
|
||||||
|
# The security settings the throttle and the proxy trust read on hot
|
||||||
|
# paths. Cached rather than queried per request; until this runs they
|
||||||
|
# hold their registry defaults, which is the correct behaviour for a
|
||||||
|
# server that has not finished starting.
|
||||||
|
await refresh_live(db)
|
||||||
# Expire old trash in the background (retention.py). One task per process is
|
# Expire old trash in the background (retention.py). One task per process is
|
||||||
# correct because the image serves with a single hypercorn worker (Dockerfile);
|
# correct because the image serves with a single hypercorn worker (Dockerfile);
|
||||||
# if that ever gains `--workers`, this needs a lock so N workers don't each
|
# if that ever gains `--workers`, this needs a lock so N workers don't each
|
||||||
|
|||||||
@@ -15,9 +15,6 @@ class Config:
|
|||||||
If unset, a key is generated and persisted in the DB (see
|
If unset, a key is generated and persisted in the DB (see
|
||||||
``thoughtsync.settings.load_or_create_secret_key``), so sessions survive
|
``thoughtsync.settings.load_or_create_secret_key``), so sessions survive
|
||||||
restarts with no volume required.
|
restarts with no volume required.
|
||||||
- ``THOUGHTSYNC_TRUSTED_PROXY_HOPS`` — how many proxies in front of this app may
|
|
||||||
be believed when reading ``X-Forwarded-For`` / ``X-Forwarded-Proto``. Defaults
|
|
||||||
to 1 (one reverse proxy terminating TLS). See ``trusted_proxy_hops``.
|
|
||||||
|
|
||||||
Uploaded media lives under ``DATA_DIR`` — a fixed, authoritative path
|
Uploaded media lives under ``DATA_DIR`` — a fixed, authoritative path
|
||||||
(``/var/thoughtsync``), intentionally NOT configurable (a mutable data path only
|
(``/var/thoughtsync``), intentionally NOT configurable (a mutable data path only
|
||||||
@@ -52,35 +49,3 @@ class Config:
|
|||||||
"""Optional break-glass override for the cookie-signing secret."""
|
"""Optional break-glass override for the cookie-signing secret."""
|
||||||
return os.environ.get("THOUGHTSYNC_SECRET_KEY") or None
|
return os.environ.get("THOUGHTSYNC_SECRET_KEY") or None
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def trusted_proxy_hops(cls) -> int:
|
|
||||||
"""How many proxies in front of this app may be believed.
|
|
||||||
|
|
||||||
`X-Forwarded-For` grows LEFT to RIGHT: each hop appends the address it saw.
|
|
||||||
So the RIGHTMOST entry was written by our own proxy and is the address that
|
|
||||||
actually connected to it, while anything a client sent arrives to the LEFT of
|
|
||||||
that — which is why the leftmost entry, the "original client", is precisely
|
|
||||||
the one a caller can forge.
|
|
||||||
|
|
||||||
With `n` trusted hops the real client is the nth entry from the right:
|
|
||||||
|
|
||||||
0 no proxy — ignore the header entirely, use the socket address
|
|
||||||
1 one reverse proxy terminating TLS (the default, and this deployment)
|
|
||||||
2 a CDN in front of that proxy — Cloudflare appended the client, our
|
|
||||||
proxy appended Cloudflare
|
|
||||||
|
|
||||||
Set it to the number of proxies you actually run. Too HIGH and a caller can
|
|
||||||
forge an address by padding the header; too low and everyone behind the CDN
|
|
||||||
shares one bucket. Too low is the safe direction, so it is the fallback when
|
|
||||||
the header is shorter than configured.
|
|
||||||
|
|
||||||
Env rather than the Settings UI (rule 25's "absolute bootstrap only" carve-
|
|
||||||
out): it is a property of the deployment topology, not a preference, and the
|
|
||||||
rate limiter consults it before opening a database connection — which is the
|
|
||||||
whole point of checking a throttle before doing expensive work.
|
|
||||||
"""
|
|
||||||
raw = os.environ.get("THOUGHTSYNC_TRUSTED_PROXY_HOPS", "1")
|
|
||||||
try:
|
|
||||||
return max(0, int(raw))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return 1
|
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ sent arrives to the LEFT of those.
|
|||||||
That inverts the intuitive reading. The leftmost entry is nominally "the original
|
That inverts the intuitive reading. The leftmost entry is nominally "the original
|
||||||
client" — and is exactly the one a caller can forge, by sending the header themselves.
|
client" — and is exactly the one a caller can forge, by sending the header themselves.
|
||||||
So we count in from the right by the number of proxies we actually run
|
So we count in from the right by the number of proxies we actually run
|
||||||
(`THOUGHTSYNC_TRUSTED_PROXY_HOPS`, default 1), and a forged prefix can never be
|
(the **Trusted proxy hops** setting, default 1), and a forged prefix can never be
|
||||||
selected no matter how much of it there is.
|
selected no matter how much of it there is.
|
||||||
|
|
||||||
Too HIGH a hop count is the dangerous direction: it starts believing entries no proxy
|
Too HIGH a hop count is the dangerous direction: it starts believing entries no proxy
|
||||||
@@ -27,7 +27,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from quart import has_request_context, request
|
from quart import has_request_context, request
|
||||||
|
|
||||||
from .config import Config
|
from .settings import live
|
||||||
|
|
||||||
|
|
||||||
def trusted_entry(header: str, hops: int) -> str | None:
|
def trusted_entry(header: str, hops: int) -> str | None:
|
||||||
@@ -54,7 +54,7 @@ def client_address() -> str:
|
|||||||
return forwarded_for(
|
return forwarded_for(
|
||||||
request.headers.get("X-Forwarded-For", ""),
|
request.headers.get("X-Forwarded-For", ""),
|
||||||
request.remote_addr,
|
request.remote_addr,
|
||||||
Config.trusted_proxy_hops(),
|
live("trusted_proxy_hops"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -70,5 +70,5 @@ def is_https() -> bool:
|
|||||||
return False
|
return False
|
||||||
if request.is_secure:
|
if request.is_secure:
|
||||||
return True
|
return True
|
||||||
entry = trusted_entry(request.headers.get("X-Forwarded-Proto", ""), Config.trusted_proxy_hops())
|
entry = trusted_entry(request.headers.get("X-Forwarded-Proto", ""), live("trusted_proxy_hops"))
|
||||||
return (entry or "").lower() == "https"
|
return (entry or "").lower() == "https"
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ came from, and either one can refuse it:
|
|||||||
addresses the attempts arrive from.
|
addresses the attempts arrive from.
|
||||||
- **The address** bounds the damage from one source spraying many accounts. It is
|
- **The address** bounds the damage from one source spraying many accounts. It is
|
||||||
read from ``X-Forwarded-For``, counting in from the RIGHT by
|
read from ``X-Forwarded-For``, counting in from the RIGHT by
|
||||||
``THOUGHTSYNC_TRUSTED_PROXY_HOPS`` so that only entries our own proxies wrote are
|
the **Trusted proxy hops** setting so that only entries our own proxies wrote are
|
||||||
believed — a forged header lands to the left of those and is never selected. It is
|
believed — a forged header lands to the left of those and is never selected. It is
|
||||||
still the weaker of the two keys, because it depends on that setting matching the
|
still the weaker of the two keys, because it depends on that setting matching the
|
||||||
deployment; the account key depends on nothing.
|
deployment; the account key depends on nothing.
|
||||||
@@ -36,29 +36,18 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import time
|
import time
|
||||||
from collections import deque
|
from collections import deque
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
from .settings import live
|
||||||
|
|
||||||
|
|
||||||
# Failed sign-ins tolerated per account before it stops answering, and for how long.
|
# Never let the bookkeeping become the denial of service: an attacker rotating an
|
||||||
# Ten is comfortably above a person mistyping a password and far below anything that
|
# address could otherwise mint an unbounded number of buckets. Well above any real
|
||||||
# makes a dictionary worth running.
|
# deployment's distinct-caller count, so a legitimate instance never reaches it; when
|
||||||
ACCOUNT_LIMIT = 10
|
# it is reached the oldest buckets are dropped, which at worst forgives some attempts.
|
||||||
ACCOUNT_WINDOW_S = 15 * 60
|
#
|
||||||
|
# Not a setting: it protects the limiter from itself rather than the app from a
|
||||||
# Wider, because one address is legitimately many people: a household, an office
|
# caller, and there is no operator judgment to apply to it.
|
||||||
# behind NAT, a phone on carrier-grade NAT.
|
|
||||||
ADDRESS_LIMIT = 50
|
|
||||||
ADDRESS_WINDOW_S = 15 * 60
|
|
||||||
|
|
||||||
# Registration is scarcer than a sign-in — it creates a row, and on a private
|
|
||||||
# instance the honest number of accounts anyone needs to make is one.
|
|
||||||
REGISTER_LIMIT = 5
|
|
||||||
REGISTER_WINDOW_S = 60 * 60
|
|
||||||
|
|
||||||
# Never let the bookkeeping become the denial of service: an attacker rotating a
|
|
||||||
# forged X-Forwarded-For could otherwise mint an unbounded number of buckets. Well
|
|
||||||
# above any real deployment's distinct-caller count, so a legitimate instance never
|
|
||||||
# reaches it; when it is reached the oldest buckets are dropped, which at worst
|
|
||||||
# forgives some attempts.
|
|
||||||
MAX_BUCKETS = 10_000
|
MAX_BUCKETS = 10_000
|
||||||
|
|
||||||
|
|
||||||
@@ -68,13 +57,25 @@ class SlidingWindow:
|
|||||||
Sliding rather than a fixed window because a fixed one lets twice the limit
|
Sliding rather than a fixed window because a fixed one lets twice the limit
|
||||||
through across a boundary — 10 at 14:59 and 10 at 15:00 — which for a login
|
through across a boundary — 10 at 14:59 and 10 at 15:00 — which for a login
|
||||||
limiter is the difference between the number meaning something and not.
|
limiter is the difference between the number meaning something and not.
|
||||||
|
|
||||||
|
The limit and window are SUPPLIERS, not values, so an admin saving a new number in
|
||||||
|
Settings takes effect on the next attempt instead of the next deploy. They are read
|
||||||
|
per call, which is a dict lookup — the settings cache never touches the database.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, limit: int, window_s: float) -> None:
|
def __init__(self, limit: Callable[[], int], window_s: Callable[[], float]) -> None:
|
||||||
self.limit = limit
|
self._limit = limit
|
||||||
self.window_s = window_s
|
self._window_s = window_s
|
||||||
self._hits: dict[str, deque[float]] = {}
|
self._hits: dict[str, deque[float]] = {}
|
||||||
|
|
||||||
|
@property
|
||||||
|
def limit(self) -> int:
|
||||||
|
return self._limit()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def window_s(self) -> float:
|
||||||
|
return self._window_s()
|
||||||
|
|
||||||
def _prune(self, key: str, now: float) -> deque[float]:
|
def _prune(self, key: str, now: float) -> deque[float]:
|
||||||
hits = self._hits.get(key)
|
hits = self._hits.get(key)
|
||||||
if hits is None:
|
if hits is None:
|
||||||
@@ -112,9 +113,19 @@ class SlidingWindow:
|
|||||||
self._hits.clear()
|
self._hits.clear()
|
||||||
|
|
||||||
|
|
||||||
sign_in_by_account = SlidingWindow(ACCOUNT_LIMIT, ACCOUNT_WINDOW_S)
|
def _minutes(key: str) -> Callable[[], float]:
|
||||||
sign_in_by_address = SlidingWindow(ADDRESS_LIMIT, ADDRESS_WINDOW_S)
|
return lambda: float(live(key)) * 60.0
|
||||||
register_by_address = SlidingWindow(REGISTER_LIMIT, REGISTER_WINDOW_S)
|
|
||||||
|
|
||||||
|
sign_in_by_account = SlidingWindow(
|
||||||
|
lambda: live("signin_limit_per_account"), _minutes("signin_window_minutes")
|
||||||
|
)
|
||||||
|
sign_in_by_address = SlidingWindow(
|
||||||
|
lambda: live("signin_limit_per_address"), _minutes("signin_window_minutes")
|
||||||
|
)
|
||||||
|
register_by_address = SlidingWindow(
|
||||||
|
lambda: live("register_limit_per_address"), _minutes("register_window_minutes")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def reset_all() -> None:
|
def reset_all() -> None:
|
||||||
|
|||||||
+131
-1
@@ -19,6 +19,13 @@ class SettingDef:
|
|||||||
label: str
|
label: str
|
||||||
description: str
|
description: str
|
||||||
group: str
|
group: str
|
||||||
|
# Ints only. Enforced server-side in validate_updates and passed to the UI so the
|
||||||
|
# number input carries them too. These exist because several of the security
|
||||||
|
# values have ranges where a typo is not merely wrong but dangerous — a proxy hop
|
||||||
|
# count of 50 would trust anything a caller sent, and a sign-in limit of 0 would
|
||||||
|
# lock every account out permanently.
|
||||||
|
minimum: int | None = None
|
||||||
|
maximum: int | None = None
|
||||||
|
|
||||||
|
|
||||||
# The source of truth for every user-facing setting. Add a row here and it appears
|
# The source of truth for every user-facing setting. Add a row here and it appears
|
||||||
@@ -70,6 +77,80 @@ REGISTRY: list[SettingDef] = [
|
|||||||
"The server contacts the linked site; private/internal addresses are always blocked.",
|
"The server contacts the linked site; private/internal addresses are always blocked.",
|
||||||
"Links",
|
"Links",
|
||||||
),
|
),
|
||||||
|
# --- Security -----------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# Read on paths too hot for a database round trip (the credential throttle checks
|
||||||
|
# them BEFORE opening a connection, which is the point of checking a throttle
|
||||||
|
# before doing expensive work), so they are cached — see `live()` below.
|
||||||
|
SettingDef(
|
||||||
|
"trusted_proxy_hops",
|
||||||
|
"int",
|
||||||
|
1,
|
||||||
|
"Trusted proxy hops",
|
||||||
|
"How many proxies sit in front of this server. 1 for a single reverse proxy "
|
||||||
|
"terminating HTTPS; 2 if a CDN like Cloudflare sits in front of that; 0 if "
|
||||||
|
"the app is exposed directly. This decides which entry of X-Forwarded-For is "
|
||||||
|
"believed — set it TOO HIGH and a visitor can forge their own address and "
|
||||||
|
"slip the sign-in limits below.",
|
||||||
|
"Security",
|
||||||
|
minimum=0,
|
||||||
|
maximum=10,
|
||||||
|
),
|
||||||
|
SettingDef(
|
||||||
|
"signin_limit_per_account",
|
||||||
|
"int",
|
||||||
|
10,
|
||||||
|
"Failed sign-ins per account",
|
||||||
|
"How many failures one account tolerates within the window before it stops "
|
||||||
|
"answering. Comfortably above mistyping a password, far below anything that "
|
||||||
|
"makes guessing worth attempting.",
|
||||||
|
"Security",
|
||||||
|
minimum=1,
|
||||||
|
maximum=1000,
|
||||||
|
),
|
||||||
|
SettingDef(
|
||||||
|
"signin_limit_per_address",
|
||||||
|
"int",
|
||||||
|
50,
|
||||||
|
"Failed sign-ins per address",
|
||||||
|
"The same, counted per visitor address instead of per account — it bounds one "
|
||||||
|
"source trying many accounts. Wider, because one address is legitimately many "
|
||||||
|
"people: a household, an office, a phone on carrier NAT.",
|
||||||
|
"Security",
|
||||||
|
minimum=1,
|
||||||
|
maximum=10000,
|
||||||
|
),
|
||||||
|
SettingDef(
|
||||||
|
"signin_window_minutes",
|
||||||
|
"int",
|
||||||
|
15,
|
||||||
|
"Sign-in window (minutes)",
|
||||||
|
"The trailing period both sign-in limits are counted over.",
|
||||||
|
"Security",
|
||||||
|
minimum=1,
|
||||||
|
maximum=1440,
|
||||||
|
),
|
||||||
|
SettingDef(
|
||||||
|
"register_limit_per_address",
|
||||||
|
"int",
|
||||||
|
5,
|
||||||
|
"Sign-ups per address",
|
||||||
|
"How many accounts one address may create within its window. Counted per "
|
||||||
|
"attempt rather than per failure — each one is a row either way.",
|
||||||
|
"Security",
|
||||||
|
minimum=1,
|
||||||
|
maximum=1000,
|
||||||
|
),
|
||||||
|
SettingDef(
|
||||||
|
"register_window_minutes",
|
||||||
|
"int",
|
||||||
|
60,
|
||||||
|
"Sign-up window (minutes)",
|
||||||
|
"The trailing period the sign-up limit is counted over.",
|
||||||
|
"Security",
|
||||||
|
minimum=1,
|
||||||
|
maximum=10080,
|
||||||
|
),
|
||||||
]
|
]
|
||||||
|
|
||||||
_BY_KEY: dict[str, SettingDef] = {d.key: d for d in REGISTRY}
|
_BY_KEY: dict[str, SettingDef] = {d.key: d for d in REGISTRY}
|
||||||
@@ -153,11 +234,52 @@ async def get_admin_settings(db) -> list[dict]:
|
|||||||
"label": d.label,
|
"label": d.label,
|
||||||
"description": d.description,
|
"description": d.description,
|
||||||
"group": d.group,
|
"group": d.group,
|
||||||
|
"minimum": d.minimum,
|
||||||
|
"maximum": d.maximum,
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# Settings the app must be able to read WITHOUT awaiting a database.
|
||||||
|
#
|
||||||
|
# The credential throttle consults these before opening a connection — deliberately,
|
||||||
|
# because a refused attempt is supposed to cost nothing, and the proxy hop count is
|
||||||
|
# needed to know who is even asking. A per-request query would undo both.
|
||||||
|
#
|
||||||
|
# Seeded from the registry defaults so the app works before (and without) a database —
|
||||||
|
# unit tests construct it with no Postgres at all — then refreshed from the DB at boot
|
||||||
|
# and again whenever an admin saves. Same live-update contract `session_ttl_days`
|
||||||
|
# already has in settings_api.py.
|
||||||
|
_LIVE_KEYS = (
|
||||||
|
"trusted_proxy_hops",
|
||||||
|
"signin_limit_per_account",
|
||||||
|
"signin_limit_per_address",
|
||||||
|
"signin_window_minutes",
|
||||||
|
"register_limit_per_address",
|
||||||
|
"register_window_minutes",
|
||||||
|
)
|
||||||
|
|
||||||
|
_live: dict[str, Any] = {k: _BY_KEY[k].default for k in _LIVE_KEYS}
|
||||||
|
|
||||||
|
|
||||||
|
def live(key: str) -> Any:
|
||||||
|
"""The cached value of a hot setting. Synchronous, never touches the database."""
|
||||||
|
return _live[key]
|
||||||
|
|
||||||
|
|
||||||
|
async def refresh_live(db) -> None:
|
||||||
|
"""Re-read the hot settings into the cache. Called at boot and after every save."""
|
||||||
|
for key in _LIVE_KEYS:
|
||||||
|
_live[key] = await get_setting(db, key)
|
||||||
|
|
||||||
|
|
||||||
|
def reset_live() -> None:
|
||||||
|
"""Back to registry defaults. For tests — nothing in the app calls this."""
|
||||||
|
for key in _LIVE_KEYS:
|
||||||
|
_live[key] = _BY_KEY[key].default
|
||||||
|
|
||||||
|
|
||||||
def validate_updates(updates: dict) -> tuple[dict, str | None]:
|
def validate_updates(updates: dict) -> tuple[dict, str | None]:
|
||||||
"""Coerce/validate a {key: value} dict against the registry. Returns
|
"""Coerce/validate a {key: value} dict against the registry. Returns
|
||||||
(clean_values, error_message). An unknown key or a bad int is rejected."""
|
(clean_values, error_message). An unknown key or a bad int is rejected."""
|
||||||
@@ -168,9 +290,17 @@ def validate_updates(updates: dict) -> tuple[dict, str | None]:
|
|||||||
return {}, f"unknown setting: {key}"
|
return {}, f"unknown setting: {key}"
|
||||||
if defn.type == "int":
|
if defn.type == "int":
|
||||||
try:
|
try:
|
||||||
clean[key] = int(val)
|
n = int(val)
|
||||||
except (ValueError, TypeError):
|
except (ValueError, TypeError):
|
||||||
return {}, f"{defn.label} must be a whole number"
|
return {}, f"{defn.label} must be a whole number"
|
||||||
|
# Rejected rather than clamped: silently accepting a number and storing a
|
||||||
|
# different one is how somebody ends up believing a protection is set to
|
||||||
|
# something it is not.
|
||||||
|
if defn.minimum is not None and n < defn.minimum:
|
||||||
|
return {}, f"{defn.label} must be at least {defn.minimum}"
|
||||||
|
if defn.maximum is not None and n > defn.maximum:
|
||||||
|
return {}, f"{defn.label} must be at most {defn.maximum}"
|
||||||
|
clean[key] = n
|
||||||
elif defn.type == "bool":
|
elif defn.type == "bool":
|
||||||
clean[key] = _coerce_bool(val)
|
clean[key] = _coerce_bool(val)
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ from quart import Blueprint, current_app, jsonify, request
|
|||||||
|
|
||||||
from .auth import require_admin
|
from .auth import require_admin
|
||||||
from .db import session_scope
|
from .db import session_scope
|
||||||
from .settings import get_admin_settings, set_settings, validate_updates
|
from .settings import get_admin_settings, refresh_live, set_settings, validate_updates
|
||||||
|
|
||||||
bp = Blueprint("settings", __name__, url_prefix="/api/settings")
|
bp = Blueprint("settings", __name__, url_prefix="/api/settings")
|
||||||
|
|
||||||
@@ -35,6 +35,10 @@ async def update_settings():
|
|||||||
async with session_scope() as db:
|
async with session_scope() as db:
|
||||||
await set_settings(db, clean)
|
await set_settings(db, clean)
|
||||||
await db.commit()
|
await db.commit()
|
||||||
|
# Re-read the cached security values so a saved limit or hop count applies to
|
||||||
|
# the very next request. Unconditional: cheap, and a conditional here would be
|
||||||
|
# one more place that has to know which keys are hot.
|
||||||
|
await refresh_live(db)
|
||||||
result = await get_admin_settings(db)
|
result = await get_admin_settings(db)
|
||||||
|
|
||||||
# Apply the live-tunable knob without a restart (rule 25).
|
# Apply the live-tunable knob without a restart (rule 25).
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ from thoughtsync.db import dispose_engine, session_scope
|
|||||||
from thoughtsync.models.note import Note
|
from thoughtsync.models.note import Note
|
||||||
from thoughtsync.models.note_item import NoteItem
|
from thoughtsync.models.note_item import NoteItem
|
||||||
from thoughtsync.models.user import User
|
from thoughtsync.models.user import User
|
||||||
from thoughtsync.settings import get_setting, set_settings
|
from thoughtsync.settings import get_setting, live, refresh_live, reset_live, set_settings
|
||||||
from thoughtsync.notes.helpers import derive_display_title
|
from thoughtsync.notes.helpers import derive_display_title
|
||||||
from thoughtsync.models.note_link_preview import NoteLinkPreview
|
from thoughtsync.models.note_link_preview import NoteLinkPreview
|
||||||
from thoughtsync.sync import _apply_note_items
|
from thoughtsync.sync import _apply_note_items
|
||||||
@@ -337,3 +337,80 @@ async def test_registration_closes_itself_once_an_admin_exists(app_client, db):
|
|||||||
)
|
)
|
||||||
assert third.status_code == 201
|
assert third.status_code == 201
|
||||||
assert (await third.get_json())["is_admin"] is False
|
assert (await third.get_json())["is_admin"] is False
|
||||||
|
|
||||||
|
|
||||||
|
async def test_security_settings_are_live_and_bounded(app_client, db):
|
||||||
|
"""The security values are settings now, not constants — so saving one has to take
|
||||||
|
effect without a restart, and a dangerous value has to be refused.
|
||||||
|
|
||||||
|
Real database because the whole point is the round trip: write through the admin
|
||||||
|
API, re-read into the cache the throttle consults, observe the new number.
|
||||||
|
"""
|
||||||
|
# An admin to authenticate as. First account, so it is allowed and becomes admin.
|
||||||
|
reset_live()
|
||||||
|
created = await app_client.post(
|
||||||
|
"/api/auth/register",
|
||||||
|
json={"email": "admin@example.test", "password": "a-long-enough-password"},
|
||||||
|
)
|
||||||
|
assert created.status_code == 201
|
||||||
|
|
||||||
|
# Defaults are what the registry says.
|
||||||
|
async with session_scope() as fresh:
|
||||||
|
await refresh_live(fresh)
|
||||||
|
assert live("trusted_proxy_hops") == 1
|
||||||
|
assert live("signin_limit_per_account") == 10
|
||||||
|
|
||||||
|
# A value that would disable the protection is REFUSED, not clamped — storing a
|
||||||
|
# different number than the one typed is how somebody ends up believing a limit
|
||||||
|
# is set to something it is not.
|
||||||
|
bad = await app_client.patch("/api/settings", json={"signin_limit_per_account": 0})
|
||||||
|
assert bad.status_code == 400
|
||||||
|
assert "at least" in (await bad.get_json())["error"]
|
||||||
|
|
||||||
|
# …and so is a hop count that would trust anything a caller sent.
|
||||||
|
bad_hops = await app_client.patch("/api/settings", json={"trusted_proxy_hops": 99})
|
||||||
|
assert bad_hops.status_code == 400
|
||||||
|
|
||||||
|
# A legitimate change applies to the cache the throttle reads, immediately.
|
||||||
|
ok = await app_client.patch(
|
||||||
|
"/api/settings", json={"signin_limit_per_account": 3, "trusted_proxy_hops": 2}
|
||||||
|
)
|
||||||
|
assert ok.status_code == 200
|
||||||
|
assert live("signin_limit_per_account") == 3
|
||||||
|
assert live("trusted_proxy_hops") == 2
|
||||||
|
|
||||||
|
# And it is persisted, not just cached.
|
||||||
|
async with session_scope() as fresh:
|
||||||
|
assert await get_setting(fresh, "trusted_proxy_hops") == 2
|
||||||
|
|
||||||
|
reset_live()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_the_security_group_reaches_the_admin_ui(app_client, db):
|
||||||
|
"""Every security value has to be visible and editable, which is the whole reason
|
||||||
|
they moved out of the environment."""
|
||||||
|
created = await app_client.post(
|
||||||
|
"/api/auth/register",
|
||||||
|
json={"email": "admin2@example.test", "password": "a-long-enough-password"},
|
||||||
|
)
|
||||||
|
assert created.status_code == 201
|
||||||
|
|
||||||
|
resp = await app_client.get("/api/settings")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
rows = (await resp.get_json())["settings"]
|
||||||
|
security = {r["key"]: r for r in rows if r["group"] == "Security"}
|
||||||
|
|
||||||
|
assert set(security) == {
|
||||||
|
"trusted_proxy_hops",
|
||||||
|
"signin_limit_per_account",
|
||||||
|
"signin_limit_per_address",
|
||||||
|
"signin_window_minutes",
|
||||||
|
"register_limit_per_address",
|
||||||
|
"register_window_minutes",
|
||||||
|
}
|
||||||
|
# The UI renders a number input from these, and it cannot offer a safe range it
|
||||||
|
# was never told about.
|
||||||
|
for row in security.values():
|
||||||
|
assert row["type"] == "int"
|
||||||
|
assert row["minimum"] is not None and row["maximum"] is not None
|
||||||
|
assert row["description"], f"{row['key']} has no description to explain itself"
|
||||||
|
|||||||
+17
-9
@@ -13,10 +13,18 @@ import time
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from thoughtsync import ratelimit
|
from thoughtsync import ratelimit
|
||||||
|
from thoughtsync.settings import live
|
||||||
from thoughtsync.app import create_app
|
from thoughtsync.app import create_app
|
||||||
from thoughtsync.ratelimit import SlidingWindow
|
from thoughtsync.ratelimit import SlidingWindow
|
||||||
|
|
||||||
|
|
||||||
|
def window(limit: int, window_s: float) -> SlidingWindow:
|
||||||
|
"""A fixed-value window. The real ones read their numbers from the settings cache
|
||||||
|
so an admin's change applies immediately; these tests are about the counting, not
|
||||||
|
about where the numbers come from."""
|
||||||
|
return SlidingWindow(lambda: limit, lambda: window_s)
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
def _clean_counters():
|
def _clean_counters():
|
||||||
ratelimit.reset_all()
|
ratelimit.reset_all()
|
||||||
@@ -30,7 +38,7 @@ def app():
|
|||||||
|
|
||||||
|
|
||||||
def test_under_the_limit_is_not_blocked():
|
def test_under_the_limit_is_not_blocked():
|
||||||
w = SlidingWindow(limit=3, window_s=60)
|
w = window(3, 60)
|
||||||
for i in range(3):
|
for i in range(3):
|
||||||
assert w.retry_after("k", now=i) is None
|
assert w.retry_after("k", now=i) is None
|
||||||
w.record("k", now=i)
|
w.record("k", now=i)
|
||||||
@@ -38,7 +46,7 @@ def test_under_the_limit_is_not_blocked():
|
|||||||
|
|
||||||
|
|
||||||
def test_window_slides_rather_than_resetting():
|
def test_window_slides_rather_than_resetting():
|
||||||
w = SlidingWindow(limit=2, window_s=60)
|
w = window(2, 60)
|
||||||
w.record("k", now=0)
|
w.record("k", now=0)
|
||||||
w.record("k", now=30)
|
w.record("k", now=30)
|
||||||
assert w.retry_after("k", now=31) is not None
|
assert w.retry_after("k", now=31) is not None
|
||||||
@@ -50,7 +58,7 @@ def test_window_slides_rather_than_resetting():
|
|||||||
|
|
||||||
|
|
||||||
def test_retry_after_points_past_the_oldest_hit():
|
def test_retry_after_points_past_the_oldest_hit():
|
||||||
w = SlidingWindow(limit=1, window_s=100)
|
w = window(1, 100)
|
||||||
w.record("k", now=10)
|
w.record("k", now=10)
|
||||||
wait = w.retry_after("k", now=40)
|
wait = w.retry_after("k", now=40)
|
||||||
# The hit at t=10 leaves the window at t=110, i.e. 70s away. Rounded up, never
|
# The hit at t=10 leaves the window at t=110, i.e. 70s away. Rounded up, never
|
||||||
@@ -62,14 +70,14 @@ def test_retry_after_points_past_the_oldest_hit():
|
|||||||
|
|
||||||
|
|
||||||
def test_keys_are_counted_separately():
|
def test_keys_are_counted_separately():
|
||||||
w = SlidingWindow(limit=1, window_s=60)
|
w = window(1, 60)
|
||||||
w.record("a", now=0)
|
w.record("a", now=0)
|
||||||
assert w.retry_after("a", now=1) is not None
|
assert w.retry_after("a", now=1) is not None
|
||||||
assert w.retry_after("b", now=1) is None
|
assert w.retry_after("b", now=1) is None
|
||||||
|
|
||||||
|
|
||||||
def test_forget_clears_one_key():
|
def test_forget_clears_one_key():
|
||||||
w = SlidingWindow(limit=1, window_s=60)
|
w = window(1, 60)
|
||||||
w.record("a", now=0)
|
w.record("a", now=0)
|
||||||
w.record("b", now=0)
|
w.record("b", now=0)
|
||||||
w.forget("a")
|
w.forget("a")
|
||||||
@@ -81,7 +89,7 @@ def test_bucket_count_is_bounded(monkeypatch):
|
|||||||
# An attacker rotating a forged X-Forwarded-For must not be able to grow this
|
# An attacker rotating a forged X-Forwarded-For must not be able to grow this
|
||||||
# dict without limit — the limiter cannot become the exhaustion it prevents.
|
# dict without limit — the limiter cannot become the exhaustion it prevents.
|
||||||
monkeypatch.setattr(ratelimit, "MAX_BUCKETS", 8)
|
monkeypatch.setattr(ratelimit, "MAX_BUCKETS", 8)
|
||||||
w = SlidingWindow(limit=5, window_s=60)
|
w = window(5, 60)
|
||||||
for i in range(50):
|
for i in range(50):
|
||||||
w.record(f"addr-{i}", now=i)
|
w.record(f"addr-{i}", now=i)
|
||||||
assert len(w._hits) <= 8
|
assert len(w._hits) <= 8
|
||||||
@@ -97,7 +105,7 @@ async def test_login_starts_refusing(app):
|
|||||||
# at t=0..9 are fifteen minutes stale the moment the route reads
|
# at t=0..9 are fifteen minutes stale the moment the route reads
|
||||||
# `time.monotonic()` and get pruned before they can refuse anything.
|
# `time.monotonic()` and get pruned before they can refuse anything.
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
for _ in range(ratelimit.ACCOUNT_LIMIT):
|
for _ in range(live("signin_limit_per_account")):
|
||||||
ratelimit.sign_in_by_account.record("someone@example.com", now=now)
|
ratelimit.sign_in_by_account.record("someone@example.com", now=now)
|
||||||
resp = await client.post("/api/auth/login", json=body)
|
resp = await client.post("/api/auth/login", json=body)
|
||||||
assert resp.status_code == 429
|
assert resp.status_code == 429
|
||||||
@@ -109,7 +117,7 @@ async def test_login_starts_refusing(app):
|
|||||||
async def test_device_login_shares_the_account_counter(app):
|
async def test_device_login_shares_the_account_counter(app):
|
||||||
client = app.test_client()
|
client = app.test_client()
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
for _ in range(ratelimit.ACCOUNT_LIMIT):
|
for _ in range(live("signin_limit_per_account")):
|
||||||
ratelimit.sign_in_by_account.record("someone@example.com", now=now)
|
ratelimit.sign_in_by_account.record("someone@example.com", now=now)
|
||||||
resp = await client.post(
|
resp = await client.post(
|
||||||
"/api/auth/device-login",
|
"/api/auth/device-login",
|
||||||
@@ -123,7 +131,7 @@ async def test_device_login_shares_the_account_counter(app):
|
|||||||
async def test_register_is_throttled_by_address(app):
|
async def test_register_is_throttled_by_address(app):
|
||||||
client = app.test_client()
|
client = app.test_client()
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
for _ in range(ratelimit.REGISTER_LIMIT):
|
for _ in range(live("register_limit_per_address")):
|
||||||
ratelimit.register_by_address.record("203.0.113.9", now=now)
|
ratelimit.register_by_address.record("203.0.113.9", now=now)
|
||||||
resp = await client.post(
|
resp = await client.post(
|
||||||
"/api/auth/register",
|
"/api/auth/register",
|
||||||
|
|||||||
Reference in New Issue
Block a user