bvandeusen
43ebb6eceb
Run 4092 published `thoughtsync.apk` to the `dev` release. Run 4098 removed it, four minutes later, and both runs were green. `write-manifest.sh` prunes the rolling channel to stop ~100 MB AppImages accumulating forever, keeping `latest.json` and anything whose name contains the current `$APP_VERSION`. The Android assets deliberately have no version in their names — a fixed name is the only addressable URL on a tag that never moves, which is the entire reason the `dev` release exists — so they matched neither rule and were swept. They would have been swept even if they HAD carried a version: Android is a different workflow with its own run number, so its version never equals the desktop's `$APP_VERSION` in this script. The keep-list is now about fixed names rather than about `latest.json` specifically, which is what the rule always meant. A fixed-name asset is self-limiting — each publish replaces the same name — so the accumulation this prune exists to prevent cannot happen to one. Worth noting how this presented: two green runs and a missing file. Nothing failed, and the only way to see it was to ask the release what it actually held rather than trusting that a step named "Publish" had published.
193 lines
9.9 KiB
Bash
193 lines
9.9 KiB
Bash
#!/usr/bin/env bash
|
|
#
|
|
# Write the updater manifest (`latest.json`) for one channel and attach it to that
|
|
# channel's release.
|
|
#
|
|
# WHY A SEPARATE STEP: the Linux and Windows bundles are built by two jobs in two
|
|
# workspaces, and neither can see the other's output — but ONE manifest has to
|
|
# describe both platforms. So this runs after both, reads what actually landed on
|
|
# the release, and writes the manifest from that. Building it inside either job
|
|
# would produce a manifest that silently omits the other platform, and a missing
|
|
# platform reads to a user as "no update available" rather than as a broken feed.
|
|
#
|
|
# WHAT IT READS: the release's own asset list. The signature for each bundle is a
|
|
# `.sig` asset published beside it (see publish-release.sh); its CONTENT is what
|
|
# goes in the manifest, which is why each one is downloaded rather than linked.
|
|
#
|
|
# Tauri's expected shape:
|
|
# { "version": "0.1.0", "pub_date": "...", "notes": "...",
|
|
# "platforms": { "<target>-<arch>": { "signature": "...", "url": "..." } } }
|
|
set -euo pipefail
|
|
|
|
: "${GITHUB_TOKEN:?GITHUB_TOKEN is required}"
|
|
: "${GITHUB_SERVER_URL:?GITHUB_SERVER_URL is required}"
|
|
: "${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is required (owner/repo)}"
|
|
: "${RELEASE_TAG:?RELEASE_TAG is required (the release holding the bundles)}"
|
|
: "${APP_VERSION:?APP_VERSION is required (the version the bundles carry)}"
|
|
|
|
# Where the manifest is PUBLISHED, which need not be where the bundles live.
|
|
#
|
|
# That split is what makes the stable channel work at all. A versioned release
|
|
# (`v0.2.0`) holds the real assets, but the app can only read a URL that never
|
|
# changes — so the same manifest is also attached to a `stable` release whose tag is
|
|
# permanent and whose only content is this file. It points back at the versioned
|
|
# assets, so nothing is duplicated.
|
|
MANIFEST_TAG="${MANIFEST_TAG:-$RELEASE_TAG}"
|
|
|
|
API="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY"
|
|
AUTH=(-H "Authorization: token $GITHUB_TOKEN")
|
|
NOTES="${RELEASE_NOTES:-}"
|
|
|
|
work="$(mktemp -d)"
|
|
trap 'rm -rf "$work"' EXIT INT TERM
|
|
|
|
echo "==> Reading assets on release $RELEASE_TAG"
|
|
release="$(curl -sS "${AUTH[@]}" "$API/releases/tags/$RELEASE_TAG")"
|
|
release_id="$(printf '%s' "$release" | grep -oE '"id"[[:space:]]*:[[:space:]]*[0-9]+' | head -1 | grep -oE '[0-9]+')"
|
|
[ -n "$release_id" ] || { echo "ERROR: no release tagged $RELEASE_TAG" >&2; exit 1; }
|
|
assets="$(curl -sS "${AUTH[@]}" "$API/releases/$release_id/assets")"
|
|
|
|
# Asset names, one per line. The API returns them in a single JSON blob; this is
|
|
# the only field needed, and grep beats adding a jq dependency to the CI image.
|
|
names="$(printf '%s' "$assets" | grep -oE '"name"[[:space:]]*:[[:space:]]*"[^"]+"' | sed -E 's/.*"([^"]+)"$/\1/')"
|
|
|
|
download_url() { printf '%s/%s/releases/download/%s/%s' "$GITHUB_SERVER_URL" "$GITHUB_REPOSITORY" "$RELEASE_TAG" "$1"; }
|
|
|
|
# One platform entry, or nothing if that platform's bundle or signature is absent.
|
|
# Emitting a partial entry would be worse than emitting none: the app would try to
|
|
# install something it can't verify.
|
|
platform_entry() {
|
|
local target="$1" pattern="$2" bundle sig_name
|
|
# Matched on THIS build's version, not just the file extension.
|
|
#
|
|
# The rolling dev release accumulates every build's assets, and picking the first
|
|
# extension match returned the OLDEST one — so the manifest advertised the new
|
|
# version while pointing at an old binary. The client would install the older
|
|
# build, still be told the newer version was available, and update forever. The
|
|
# signature check couldn't catch it either: the old bundle's signature is
|
|
# perfectly valid FOR THE OLD BUNDLE.
|
|
bundle="$(printf '%s\n' "$names" | grep -F "_${APP_VERSION}_" | grep -E "$pattern" | head -1 || true)"
|
|
# A hard failure, not a skip: reaching here means this platform's build didn't
|
|
# upload, and the whole point is to never advertise a bundle that isn't there.
|
|
[ -n "$bundle" ] || { echo " no $APP_VERSION bundle matching $pattern — skipping $target" >&2; return; }
|
|
sig_name="$bundle.sig"
|
|
if ! printf '%s\n' "$names" | grep -qxF "$sig_name"; then
|
|
echo " $bundle has no $sig_name — skipping $target (was the build signed?)" >&2
|
|
return
|
|
fi
|
|
curl -fsSL "${AUTH[@]}" -o "$work/sig" "$(download_url "$sig_name")"
|
|
# The signature is base64 on one line already; strip any stray newline so it
|
|
# can't break the JSON string it's about to become.
|
|
local signature
|
|
signature="$(tr -d '\r\n' < "$work/sig")"
|
|
printf ' "%s": { "signature": "%s", "url": "%s" }' "$target" "$signature" "$(download_url "$bundle")"
|
|
}
|
|
|
|
echo "==> Building the manifest"
|
|
entries=()
|
|
# `.AppImage` only on Linux: the updater replaces the running bundle in place, which
|
|
# a package-manager install (deb/pacman) must never have done to it.
|
|
if entry="$(platform_entry "linux-x86_64" '\.AppImage$')" && [ -n "$entry" ]; then entries+=("$entry"); fi
|
|
if entry="$(platform_entry "windows-x86_64" '\.exe$')" && [ -n "$entry" ]; then entries+=("$entry"); fi
|
|
|
|
if [ ${#entries[@]} -eq 0 ]; then
|
|
echo "ERROR: no signed bundle on $RELEASE_TAG — refusing to publish an empty manifest." >&2
|
|
echo " (An empty manifest would tell every client it is up to date.)" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# No `date -u -Is` — busybox date in the CI image doesn't take it.
|
|
pub_date="$(date -u '+%Y-%m-%dT%H:%M:%SZ')"
|
|
{
|
|
printf '{\n'
|
|
printf ' "version": "%s",\n' "$APP_VERSION"
|
|
printf ' "pub_date": "%s",\n' "$pub_date"
|
|
printf ' "notes": "%s",\n' "$NOTES"
|
|
printf ' "platforms": {\n'
|
|
for i in "${!entries[@]}"; do
|
|
[ "$i" -eq 0 ] || printf ',\n'
|
|
printf '%s' "${entries[$i]}"
|
|
done
|
|
printf '\n }\n'
|
|
printf '}\n'
|
|
} > "$work/latest.json"
|
|
|
|
echo "==> Manifest:"
|
|
cat "$work/latest.json"
|
|
|
|
# --- resolve the release the manifest is published TO ------------------------
|
|
if [ "$MANIFEST_TAG" = "$RELEASE_TAG" ]; then
|
|
target_id="$release_id"
|
|
target_assets="$assets"
|
|
else
|
|
echo "==> Resolving the $MANIFEST_TAG channel release"
|
|
target="$(curl -sS "${AUTH[@]}" "$API/releases/tags/$MANIFEST_TAG")"
|
|
target_id="$(printf '%s' "$target" | grep -oE '"id"[[:space:]]*:[[:space:]]*[0-9]+' | head -1 | grep -oE '[0-9]+' || true)"
|
|
if [ -z "${target_id:-}" ]; then
|
|
# First publish to this channel. A pointer release: no bundles of its own, just
|
|
# a permanent tag for the manifest to live under.
|
|
echo " creating it (pointer release, manifest only)"
|
|
body="{\"tag_name\":\"$MANIFEST_TAG\",\"name\":\"ThoughtSync ($MANIFEST_TAG channel)\",\"draft\":false,\"prerelease\":false,\"body\":\"Update channel pointer. The installable builds live on the versioned releases; this holds only the updater manifest.\"}"
|
|
target="$(curl -sS -X POST "${AUTH[@]}" -H "Content-Type: application/json" -d "$body" "$API/releases")"
|
|
target_id="$(printf '%s' "$target" | grep -oE '"id"[[:space:]]*:[[:space:]]*[0-9]+' | head -1 | grep -oE '[0-9]+')"
|
|
fi
|
|
[ -n "${target_id:-}" ] || { echo "ERROR: could not resolve the $MANIFEST_TAG release" >&2; exit 1; }
|
|
target_assets="$(curl -sS "${AUTH[@]}" "$API/releases/$target_id/assets")"
|
|
fi
|
|
|
|
# Replace rather than duplicate: Forgejo rejects a second asset with the same name,
|
|
# and this file is rewritten on every publish by design.
|
|
old_id="$(printf '%s' "$target_assets" \
|
|
| grep -oE "\"id\"[[:space:]]*:[[:space:]]*[0-9]+[^}]*\"name\"[[:space:]]*:[[:space:]]*\"latest\.json\"" \
|
|
| head -1 | grep -oE '[0-9]+' | head -1 || true)"
|
|
if [ -n "${old_id:-}" ]; then
|
|
echo "==> Removing the previous latest.json (id $old_id)"
|
|
curl -fsS -X DELETE "${AUTH[@]}" "$API/releases/$target_id/assets/$old_id" >/dev/null
|
|
fi
|
|
|
|
echo "==> Uploading latest.json to $MANIFEST_TAG"
|
|
curl -fsS -X POST "${AUTH[@]}" "$API/releases/$target_id/assets?name=latest.json" \
|
|
-F "attachment=@$work/latest.json" >/dev/null
|
|
|
|
echo "==> Done. $MANIFEST_TAG now advertises $APP_VERSION for ${#entries[@]} platform(s)."
|
|
|
|
# --- prune superseded builds from a rolling channel ---------------------------
|
|
#
|
|
# The dev release is republished on every push and its assets otherwise accumulate
|
|
# forever — an AppImage alone is ~100 MB, so a week of pushes is gigabytes on the
|
|
# Git host for builds nobody can reach (the manifest only ever names the newest).
|
|
#
|
|
# Only for a rolling channel. A versioned release must keep its assets: that IS the
|
|
# archive, and the stable pointer's URLs aim at it.
|
|
if [ "${PRUNE_OLD_ASSETS:-false}" = "true" ]; then
|
|
echo "==> Pruning superseded assets from $RELEASE_TAG"
|
|
# Re-read: the manifest upload above changed the asset list.
|
|
current="$(curl -sS "${AUTH[@]}" "$API/releases/$release_id/assets")"
|
|
printf '%s' "$current" \
|
|
| grep -oE '"id"[[:space:]]*:[[:space:]]*[0-9]+[^}]*"name"[[:space:]]*:[[:space:]]*"[^"]+"' \
|
|
| while IFS= read -r row; do
|
|
asset_id="$(printf '%s' "$row" | grep -oE '[0-9]+' | head -1)"
|
|
asset_name="$(printf '%s' "$row" | sed -E 's/.*"name"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')"
|
|
# Keep FIXED-NAME assets and everything belonging to the current build.
|
|
#
|
|
# A fixed name is self-limiting: each publish replaces that same name, so
|
|
# it cannot accumulate and the reason this prune exists does not apply to
|
|
# it. It is also the only kind of URL that stays addressable on a rolling
|
|
# tag, which is the whole point of having one — deleting it breaks
|
|
# whatever was pointing at it.
|
|
#
|
|
# The Android client is on that list for a second reason too: it is built
|
|
# by a DIFFERENT workflow with its own run number, so its version never
|
|
# matches $APP_VERSION here and a version-stamped name would be pruned on
|
|
# every desktop push regardless. That is exactly what happened on run
|
|
# 4098, which swept the APK run 4092 had just published.
|
|
case "$asset_name" in
|
|
latest.json|thoughtsync.apk|thoughtsync-android.json) continue ;;
|
|
*"$APP_VERSION"*) continue ;;
|
|
esac
|
|
echo " removing $asset_name"
|
|
curl -fsS -X DELETE "${AUTH[@]}" "$API/releases/$release_id/assets/$asset_id" >/dev/null || \
|
|
echo " (couldn't remove $asset_name — leaving it)" >&2
|
|
done
|
|
fi
|