clients: the server hands out five platforms, not "the Android client"
Android / Build, or is the channel already serving this? (push) Successful in 3s
CI & Build / Build now, or wait for Android? (push) Successful in 3s
Android / Kotlin + Rust (APK) (push) Skipped
CI & Build / Python lint (push) Successful in 2s
Desktop (Tauri) / Build, or is the channel already serving this? (push) Successful in 3s
Desktop (Tauri) / Tauri desktop (Linux) (push) Skipped
Desktop (Tauri) / Windows installer (cross-compiled) (push) Skipped
Desktop (Tauri) / Update manifest (push) Skipped
CI & Build / TypeScript typecheck (push) Successful in 8s
CI & Build / Python tests (push) Successful in 11s
CI & Build / integration (push) Successful in 15s
CI & Build / Build & push image (push) Successful in 30s

`client_dist.py` was written for one platform and everything structural in it was
already right — drop-in beats baked, the pair must describe one build, absence is
an ordinary answer, metadata public and bytes authenticated. This widens it to a
table rather than building beside it. Its own docstring made the argument years
before there was a second platform: a self-hoster should not need an account on
someone else's forge to get the app for their own notes.

Server side only. CI bakes nothing new until step 3 and the UI reads nothing new
until step 4, so this lands green and inert.

Five rows — android, linux-deb, linux-pacman, linux-appimage, windows — each
naming its artifact, sidecar and mimetype. Fixed filenames, version only in the
sidecar: a version-stamped name would force a glob, and a glob over a directory an
operator drops files into is how you serve the older of two builds, which is the
failure write-manifest.sh already carries a comment about.

THE ANDROID NAMES AND ROUTE DO NOT MOVE. The lane publishes those exact filenames,
clients in the field poll /api/client/android, and `android_client` stays on
/api/config beside the new `clients` map. Renaming them to match the pattern would
buy tidiness and strand every installed phone; retiring the key belongs to a later
change made when nothing polls it, not to the change introducing its replacement.
Fields were added, not moved — `ClientRelease` in core is a plain serde struct and
ignores what it does not know.

PRECEDENCE IS PER PLATFORM, which is the trap the table introduces. "First
directory holding anything wins" would mean dropping in an APK silently retracts
the four desktop downloads. Pinned by a test.

The AppImage needs a third file. It is the only bundle that replaces itself in
place, so the updater verifies a minisign signature before it does — and a bundle
that cannot be verified cannot be offered. A missing or empty `.sig` therefore
makes it absent rather than merely unsigned, and the signature travels WITH the
version so an updater can never pair one build's version with another's signature.

The tests parametrize over the table instead of testing Android and trusting the
rest. The bugs this module can have are not platform-specific, and a suite that
only exercised one platform is how the other four would ship untested.
This commit is contained in:
Bryan Van Deusen
2026-08-30 12:52:40 -04:00
parent f992439588
commit ef8aa9340f
5 changed files with 544 additions and 146 deletions
+4 -3
View File
@@ -204,9 +204,10 @@ def create_app() -> Quart:
# linking — while it still has no token and possibly no account — to decide
# whether it can talk to this server, and which optional features to offer.
data.update(protocol_advertisement())
# Which Android client this server can hand out, if any. Absent rather than
# null when it has none, so the web UI hides the download instead of
# offering a button that 404s.
# Which CLIENTS this server can hand out, if any — the whole set under
# `clients`, plus the older `android_client` key that phones in the field
# still read. Absent rather than null when it has none, so the web UI hides
# a download instead of offering a button that 404s.
data.update(client_advertisement())
return jsonify(data)
+255 -69
View File
@@ -1,4 +1,4 @@
"""The server hands out the Android client it is in step with.
"""The server hands out the clients it is in step with.
## Why the server, and not a release page
@@ -11,43 +11,69 @@ It also keeps the two in step by construction. Client and server already
negotiate a sync protocol version before linking, so a server that also serves
the client cannot hand out a phone it is unable to talk to.
## Where the file comes from
That argument was never Android-specific, which is why this module now serves a
TABLE of platforms rather than the one it was written for.
Two places, checked in that order:
## Where the files come from
Two places, checked in that order, per platform:
1. `DATA_DIR/client/` — the mounted volume that already holds attachments. An
operator who wants a SPECIFIC build drops it there and it wins.
2. the copy baked into the image at build time — CI fetches the newest published
Android build into every image, so `:dev`, `:latest` and `:<version>` all
carry a client and `docker compose pull` delivers a new one with nothing
copied by hand.
build of each client into every image, so `:dev` and `:latest` both carry a
full set and `docker compose pull` delivers new ones with nothing copied by
hand.
The precedence is the point: the image is the default, and a person who wants to
override it should not have to fight it. The baked copy sits inside the package
rather than under DATA_DIR because DATA_DIR is a volume mount, and anything the
image wrote there would be hidden the moment one is attached.
**Precedence is decided per platform, not for the set.** A drop-in `.deb` does
not shadow the baked APK. The alternative — first directory holding anything
wins — would mean replacing one client silently retracts the other four.
## What a platform needs on disk
Two files, and both must be present:
- `thoughtsync.apk` — the client
- `thoughtsync-android.json` — `{version_name, version_code, size, sha256}`
- the artifact, at a FIXED name (`thoughtsync.deb`, not
`ThoughtSync_2026.08.30.0307_amd64.deb`)
- its sidecar, `{version_name, version_code, size, sha256}`
The sidecar exists because an APK's version lives in a binary AXML manifest that
Python cannot read without the Android build tools. CI writes it beside the APK
at publish time, where the real values are already known.
**Fixed names, and the version only in the sidecar.** A version-stamped filename
would force this module to glob, and a glob over a directory an operator can drop
files into is how you serve the older of two builds — `write-manifest.sh` carries
a comment about exactly that, from the time it advertised a new version while
pointing at an old binary.
The sidecar exists because a version is not reliably readable from the artifact:
an APK keeps it in a binary AXML manifest Python cannot parse without the Android
build tools, and a `.deb` or an AppImage would each need a different unpacker. CI
writes the sidecar where the real value is already known.
The AppImage needs a THIRD file, `thoughtsync.AppImage.sig`. That is the minisign
signature the desktop updater verifies before replacing the running binary, and a
bundle that cannot be verified cannot be offered as an update — so a missing
signature makes the AppImage absent rather than merely unsigned.
## Absence is normal
A server with no APK advertises nothing, and the web UI hides the download
rather than offering a button that 404s. Same for a mismatched pair: if the
sidecar's recorded size does not match the file on disk, the two did not arrive
together and the server says it has nothing rather than serving one build while
describing another.
A server with no client for a platform advertises none, and the web UI hides that
download rather than offering a button that 404s. Same for a mismatched pair: if
the sidecar's recorded size does not match the file on disk, the two did not
arrive together, and the server says it has nothing rather than serving one build
while describing another.
This is the ONLY state available before a platform's first build has ever
published, so it is an ordinary answer and never an error.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
from quart import Blueprint, jsonify, send_from_directory
@@ -55,10 +81,90 @@ from quart import Blueprint, jsonify, send_from_directory
from .auth import login_required
from .config import Config
APK_NAME = "thoughtsync.apk"
MANIFEST_NAME = "thoughtsync-android.json"
DOWNLOAD_PATH = "/api/client/android/download"
APK_MIMETYPE = "application/vnd.android.package-archive"
@dataclass(frozen=True)
class Platform:
"""One installable client, and where its files sit."""
id: str
# What a person calls it. Named for the DISTRO rather than the package format
# ("Debian / Ubuntu", not ".deb") — someone knows which system they run and
# does not necessarily know which packaging it uses.
label: str
artifact: str
sidecar: str
mimetype: str
# An updater-verifiable bundle: `<artifact>.sig` must be present too, and its
# contents are published with the metadata. Only the AppImage, because it is
# the only bundle that can replace itself in place — a package-manager install
# cannot, by design (see the desktop's update.rs).
signed: bool = False
@property
def signature(self) -> str:
return f"{self.artifact}.sig"
@property
def download_path(self) -> str:
return f"/api/client/{self.id}/download"
# ONE definition of what this server can hand out. Every route, the /api/config
# advertisement and the tests all read this table; adding a platform is adding a
# row.
PLATFORMS: tuple[Platform, ...] = (
Platform(
id="android",
label="Android",
# UNCHANGED, and it must stay unchanged: the Android lane publishes these
# exact names, CI bakes them in under them, and clients in the field poll
# `/api/client/android`. Renaming them to match the pattern below would
# buy tidiness and strand every installed phone.
artifact="thoughtsync.apk",
sidecar="thoughtsync-android.json",
mimetype="application/vnd.android.package-archive",
),
Platform(
id="linux-deb",
label="Debian / Ubuntu",
artifact="thoughtsync.deb",
sidecar="thoughtsync-linux-deb.json",
mimetype="application/vnd.debian.binary-package",
),
Platform(
id="linux-pacman",
label="Arch / CachyOS",
artifact="thoughtsync.pkg.tar.zst",
sidecar="thoughtsync-linux-pacman.json",
mimetype="application/zstd",
),
Platform(
id="linux-appimage",
label="Other Linux (AppImage)",
artifact="thoughtsync.AppImage",
sidecar="thoughtsync-linux-appimage.json",
# No registered type for an AppImage, and guessing one buys nothing: it is
# served as an attachment either way, and octet-stream is the answer that
# cannot be wrong.
mimetype="application/octet-stream",
signed=True,
),
Platform(
id="windows",
label="Windows",
artifact="thoughtsync-setup.exe",
sidecar="thoughtsync-windows.json",
mimetype="application/vnd.microsoft.portable-executable",
),
)
BY_ID: dict[str, Platform] = {p.id: p for p in PLATFORMS}
# The Android names, still importable under their old spellings because docs and
# the CI lane refer to them. Derived from the table rather than restated, so the
# two cannot drift.
APK_NAME = BY_ID["android"].artifact
MANIFEST_NAME = BY_ID["android"].sidecar
# The copy CI bakes into the image. Inside the package, NOT under DATA_DIR: that
# is a volume mount, and a file the image wrote there would vanish behind it.
@@ -67,17 +173,17 @@ BAKED_ROOT = Path(__file__).resolve().parent / "client"
bp = Blueprint("client_dist", __name__)
def _read(root: Path) -> dict | None:
"""The build in one directory, or None.
def _read(root: Path, platform: Platform) -> dict | None:
"""One platform's build in one directory, or None.
Never raises. A missing directory, an unreadable sidecar, malformed JSON and a
sidecar that describes a different file are all the same answer to the only
question being asked — "is there a client here I can honestly offer?" — and
that answer is no.
Never raises. A missing directory, an unreadable sidecar, malformed JSON, a
sidecar that describes a different file and — for a signed bundle — a missing
signature are all the same answer to the only question being asked, "is there
a client here I can honestly offer?", and that answer is no.
"""
try:
size = (root / APK_NAME).stat().st_size
meta = json.loads((root / MANIFEST_NAME).read_text(encoding="utf-8"))
size = (root / platform.artifact).stat().st_size
meta = json.loads((root / platform.sidecar).read_text(encoding="utf-8"))
version = str(meta["version_name"])
code = int(meta["version_code"])
recorded = int(meta["size"])
@@ -87,83 +193,163 @@ def _read(root: Path) -> dict | None:
# The pair has to describe one build. A sidecar left behind by a previous
# release would otherwise advertise a version this server cannot serve, and the
# phone would download something other than what it was promised.
# client would download something other than what it was promised.
if recorded != size:
return None
return {
release = {
"platform": platform.id,
"label": platform.label,
"version": version,
# What Android actually compares. `version` is for people; a name is a
# string and sorts like one, which is not how "is this newer" works.
# What a comparator reads. `version` is for people; a name is a string and
# sorts like one, which is not how "is this newer" works.
"version_code": code,
"size": size,
# Computed by CI over the same bytes it uploaded, so a client can tell a
# truncated download from a complete one BEFORE handing it to the
# installer. Not a trust anchor — the signature is that.
# truncated download from a complete one BEFORE handing it to an installer.
# Not a trust anchor — the signature is that.
"sha256": digest,
"url": DOWNLOAD_PATH,
# A PATH, never an absolute URL: the client joins it to the base it is
# already linked to, so a compromised or misconfigured server cannot
# redirect the download somewhere else. `core/src/sync/client.rs` relies on
# this and says so.
"url": platform.download_path,
}
if platform.signed:
# The signature travels WITH the metadata rather than behind its own route.
# It is ~100 bytes, it is public wherever these bundles are published, and
# the updater needs the version and the signature in the same breath — one
# request that cannot return a signature belonging to a different build.
try:
sig = (root / platform.signature).read_text(encoding="utf-8").strip()
except OSError:
return None
if not sig:
return None
release["signature"] = sig
def _resolve() -> tuple[Path, dict] | None:
"""Which directory this server serves from, and what is in it.
return release
def _resolve(platform: Platform) -> tuple[Path, dict] | None:
"""Which directory this server serves a platform from, and what is in it.
The operator's drop-in beats the baked copy — someone who deliberately put a
build on the volume wants that build, not whatever the image happened to ship
with. A directory holding a broken or half-copied pair does NOT shadow the
image: it simply is not a client, so the search moves on.
build on the volume wants that build, older or not. A directory holding a
broken or half-copied pair does NOT shadow the image: it simply is not a
client, so the search moves on.
"""
for root in (Path(Config.client_root()), BAKED_ROOT):
release = _read(root)
release = _read(root, platform)
if release is not None:
return root, release
return None
def android_release() -> dict | None:
"""What Android build this server holds, or None if it holds none."""
resolved = _resolve()
def release(platform_id: str) -> dict | None:
"""What build of one client this server holds, or None if it holds none."""
platform = BY_ID.get(platform_id)
if platform is None:
return None
resolved = _resolve(platform)
return resolved[1] if resolved else None
def advertisement() -> dict:
"""The `/api/config` fragment describing this server's Android client.
def releases() -> dict[str, dict]:
"""Every client this server can hand out, keyed by platform id.
An empty dict when there is none, so the key is ABSENT rather than null — a
client testing for the key gets one unambiguous answer instead of having to
distinguish "no client" from "old server that never had this field".
Platforms it holds nothing for are ABSENT rather than present-and-null, so a
caller can test for the key instead of distinguishing "no build" from "a
server that never had this platform".
"""
release = android_release()
return {"android_client": release} if release else {}
found = {p.id: _resolve(p) for p in PLATFORMS}
return {pid: r[1] for pid, r in found.items() if r is not None}
@bp.get("/api/client/android")
async def android_metadata():
"""Version and digest without the 55 MiB. What an updater polls."""
release = android_release()
if release is None:
return jsonify({"error": "this server has no Android client"}), 404
return jsonify(release)
def android_release() -> dict | None:
"""What Android build this server holds, or None.
Kept as its own name because the back-compatible `/api/config` key below is
about Android specifically, and because saying so reads better than
`release("android")` at the two call sites that mean the phone.
"""
return release("android")
@bp.get(DOWNLOAD_PATH)
def advertisement() -> dict:
"""The `/api/config` fragment describing this server's clients.
Two keys, deliberately, and the older one is not deprecated here:
`clients` is the whole table, which is what the web UI renders the downloads
section from.
`android_client` is what phones in the field already read. It costs one
duplicated dict to not strand every installed Android client, and retiring it
is a later decision made when nothing polls it — not a tidy-up done in the
change that introduces its replacement.
Both are ABSENT rather than null when empty, so a client testing for a key gets
one unambiguous answer instead of having to distinguish "no client" from "an
old server that never had this field".
"""
data: dict = {}
found = releases()
if found:
data["clients"] = found
if "android" in found:
data["android_client"] = found["android"]
return data
@bp.get("/api/client")
async def client_index():
"""Everything this server holds, in one request.
The downloads UI needs all five to decide what to lead with, and five requests
to answer one question is five chances to render half a page.
"""
return jsonify({"clients": releases()})
@bp.get("/api/client/<platform_id>")
async def client_metadata(platform_id: str):
"""Version and digest without the payload. What an updater polls.
Public, because a client has to be able to ask "is there something newer?"
cheaply — before it has a token, in the case of a first pairing.
An unknown platform and a platform with no build both 404. They are the same
answer to the caller ("not here"), and distinguishing them would only tell an
unauthenticated stranger which platforms this build of the server knows about.
"""
found = release(platform_id)
if found is None:
return jsonify({"error": f"this server has no {platform_id} client"}), 404
return jsonify(found)
@bp.get("/api/client/<platform_id>/download")
@login_required
async def android_download():
"""The APK itself.
async def client_download(platform_id: str):
"""The bytes themselves.
Authenticated — by session cookie from a browser, or by device bearer token
from a client updating itself; `login_required` accepts either. The metadata
above is public because a client has to be able to ask "is there something
newer?" cheaply, but the bytes are not for anyone who can reach the port.
above is public; the bytes are not for anyone who can reach the port.
"""
resolved = _resolve()
platform = BY_ID.get(platform_id)
if platform is None:
return jsonify({"error": f"unknown client platform '{platform_id}'"}), 404
resolved = _resolve(platform)
if resolved is None:
return jsonify({"error": "this server has no Android client"}), 404
# From the SAME directory the advertisement came from, or a drop-in appearing
# between the two calls would serve bytes the metadata does not describe.
return jsonify({"error": f"this server has no {platform_id} client"}), 404
# From the SAME directory the metadata came from, or a drop-in appearing between
# the two calls would serve bytes the metadata does not describe.
root, _ = resolved
response = await send_from_directory(root, APK_NAME, mimetype=APK_MIMETYPE)
response = await send_from_directory(root, platform.artifact, mimetype=platform.mimetype)
# Without this some browsers try to render it, and Android's download handler
# wants a filename to hand to the package installer.
response.headers["Content-Disposition"] = f'attachment; filename="{APK_NAME}"'
response.headers["Content-Disposition"] = f'attachment; filename="{platform.artifact}"'
return response
+8 -5
View File
@@ -35,12 +35,15 @@ class Config:
@classmethod
def client_root(cls) -> Path:
"""Where the Android APK this server hands out lives.
"""Where an operator DROPS IN clients for this server to hand out.
Under DATA_DIR rather than baked into the image: the APK is ~55 MiB and an
install that never touches Android should not carry it. Being on the same
mounted volume as uploads also means an operator drops a build there once
and container recreation does not lose it. See client_dist.py.
One directory for every platform; `client_dist.py` picks files out of it by
name. Under DATA_DIR because it is a mounted volume: a build placed here
survives container recreation, and it beats the copy baked into the image,
which is the whole point of the directory existing.
Empty is the ordinary case — the image ships its own set and most operators
never touch this.
"""
return Path(cls.DATA_DIR) / "client"