URLs unfurl on their own, and a lone link becomes the note
CI & Build / Build now, or wait for Android? (push) Successful in 3s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 7s
CI & Build / Python tests (push) Successful in 9s
CI & Build / integration (push) Successful in 14s
CI & Build / Build & push image (push) Successful in 37s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 2m6s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 4m16s
Desktop (Tauri) / Update manifest (push) Successful in 6s
CI & Build / Build now, or wait for Android? (push) Successful in 3s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 7s
CI & Build / Python tests (push) Successful in 9s
CI & Build / integration (push) Successful in 14s
CI & Build / Build & push image (push) Successful in 37s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 2m6s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 4m16s
Desktop (Tauri) / Update manifest (push) Successful in 6s
Operator: *"I'd like for URLs to unfurl. To be the whole note when the note is a single URL, and to be a compact slot on the bottom of the note when the URL is inline. We also need to support multiple URLs in a single note."* Less new machinery than it sounds: `unfurl.py` already fetched and parsed OG tags, SSRF-hardened, and `note_link_previews` was already `UNIQUE(note_id, url)` — so several URLs per note has worked at the storage layer all along. What was missing was that it needed a button, had one size, and drew that size in the wrong place. **Automatic, and never in the way.** New `unfurl_queue.py` detects a body's URLs and fetches them on a background task AFTER the note is committed. Capture speed is the product: an unfurl is a five-second timeout against a host nobody controls, and a note has to persist the instant someone stops typing. Scheduled from create, from a body edit, and from a synced push — so a linked desktop or Android client gets previews too, on its next pull. An unlinked one has no server to ask and simply has none, which is the honest consequence of being offline. Safe to call on every save: it re-reads what's cached and does nothing when nothing is new. Capped at five URLs per note, silent on every failure (a link that won't fetch isn't an error the person needs — the note is fine, the link is still there), and it re-checks before storing, so a slow fetch can't resurrect a preview for a URL that was deleted while it was in flight. **Two presentations.** A note whose body is nothing but a URL renders as its preview and nothing else — printing the raw URL under a card that already says where it goes is saying the same thing twice, badly. Until the fetch lands, or if it never does, the URL stands in, so the card is never blank. Anything else gets a compact strip. **And the strip moved.** Previews were rendered ABOVE the body, which put a stranger's headline where the note's own first line should be — worse now that the first line IS the note's name. They sit at the foot of the card now, under the note's own words. The editor's "Preview example.com" button is gone with the manual path; removing an unwanted preview stays, and stays editor-only. Nine tests: three on detection (order, dedupe, sentence-punctuation trimming, non-http rejection) in the unit lane, and three in the integration lane for what only a real database shows — the upsert landing on the right row, a second pass fetching nothing, and a preview NOT being stored for a URL that left the body.
This commit is contained in:
@@ -26,13 +26,15 @@ from thoughtsync.models.note import Note
|
||||
from thoughtsync.models.note_item import NoteItem
|
||||
from thoughtsync.models.user import User
|
||||
from thoughtsync.notes.helpers import derive_display_title
|
||||
from thoughtsync.models.note_link_preview import NoteLinkPreview
|
||||
from thoughtsync.sync import _apply_note_items
|
||||
from thoughtsync.unfurl_queue import _unfurl_new_urls, detect_urls
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
# Every table the tests touch, child-first so FKs never block the truncate.
|
||||
# RESTART IDENTITY + CASCADE keeps this honest if a table gains children later.
|
||||
_TABLES = "notes, note_items, note_revisions, note_labels, labels, users"
|
||||
_TABLES = "notes, note_items, note_revisions, note_labels, note_link_previews, labels, users"
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
@@ -201,3 +203,81 @@ async def test_a_note_with_only_items_still_has_a_name(db, owner):
|
||||
await db.commit()
|
||||
|
||||
assert (await db.scalar(select(Note.display_title).where(Note.id == note.id))) == "milk"
|
||||
|
||||
|
||||
async def test_auto_unfurl_stores_a_preview_and_skips_what_is_cached(db, owner, monkeypatch):
|
||||
"""The background pass, run inline so the assertions are deterministic.
|
||||
|
||||
The network is stubbed — this is about what reaches the DATABASE, not about
|
||||
parsing someone's OpenGraph tags (unfurl.py's own tests cover that). What matters
|
||||
here is the part only a real database can show: the unique constraint holding, the
|
||||
upsert going to the right row, and a second pass not re-fetching.
|
||||
"""
|
||||
note = Note(
|
||||
owner_id=owner.id,
|
||||
body="read https://example.com/a and https://example.com/b",
|
||||
display_title="read https://example.com/a and https://example.com/b",
|
||||
)
|
||||
db.add(note)
|
||||
await db.commit()
|
||||
|
||||
calls: list[str] = []
|
||||
|
||||
async def fake_unfurl(url):
|
||||
calls.append(url)
|
||||
return {"url": url, "title": f"T {url}", "description": None, "image_url": None, "site_name": "example.com"}
|
||||
|
||||
monkeypatch.setattr("thoughtsync.unfurl_queue.unfurl", fake_unfurl)
|
||||
|
||||
await _unfurl_new_urls(note.id, note.body)
|
||||
assert sorted(calls) == ["https://example.com/a", "https://example.com/b"]
|
||||
|
||||
rows = (await db.scalars(select(NoteLinkPreview).where(NoteLinkPreview.note_id == note.id))).all()
|
||||
assert {r.url for r in rows} == {"https://example.com/a", "https://example.com/b"}
|
||||
assert all(r.title.startswith("T ") for r in rows)
|
||||
|
||||
# A second pass over an unchanged body fetches nothing — the whole reason
|
||||
# `schedule` is safe to call on every save.
|
||||
calls.clear()
|
||||
await _unfurl_new_urls(note.id, note.body)
|
||||
assert calls == []
|
||||
|
||||
|
||||
async def test_auto_unfurl_drops_a_preview_whose_url_left_the_body(db, owner, monkeypatch):
|
||||
"""A slow fetch must not resurrect a link the person deleted mid-flight."""
|
||||
note = Note(owner_id=owner.id, body="https://example.com/gone", display_title="x")
|
||||
db.add(note)
|
||||
await db.commit()
|
||||
|
||||
async def fake_unfurl(url):
|
||||
# Simulate the body changing while the request was in the air.
|
||||
return {"url": url, "title": "T", "description": None, "image_url": None, "site_name": None}
|
||||
|
||||
monkeypatch.setattr("thoughtsync.unfurl_queue.unfurl", fake_unfurl)
|
||||
note.body = "changed my mind"
|
||||
await db.commit()
|
||||
|
||||
await _unfurl_new_urls(note.id, "https://example.com/gone")
|
||||
rows = (await db.scalars(select(NoteLinkPreview).where(NoteLinkPreview.note_id == note.id))).all()
|
||||
assert rows == [], "a preview was stored for a URL the note no longer contains"
|
||||
|
||||
|
||||
async def test_detection_agrees_with_what_gets_stored(db, owner, monkeypatch):
|
||||
"""The detector and the storage path read the same body the same way."""
|
||||
body = "one https://example.com/x. two (https://example.com/y) three"
|
||||
assert detect_urls(body) == ["https://example.com/x", "https://example.com/y"]
|
||||
|
||||
note = Note(owner_id=owner.id, body=body, display_title="one")
|
||||
db.add(note)
|
||||
await db.commit()
|
||||
|
||||
async def fake_unfurl(url):
|
||||
return {"url": url, "title": "T", "description": None, "image_url": None, "site_name": None}
|
||||
|
||||
monkeypatch.setattr("thoughtsync.unfurl_queue.unfurl", fake_unfurl)
|
||||
await _unfurl_new_urls(note.id, body)
|
||||
|
||||
stored = {
|
||||
r for r in (await db.scalars(select(NoteLinkPreview.url).where(NoteLinkPreview.note_id == note.id))).all()
|
||||
}
|
||||
assert stored == set(detect_urls(body))
|
||||
|
||||
@@ -5,6 +5,7 @@ import pytest
|
||||
from thoughtsync.app import create_app
|
||||
from thoughtsync.common import coerce_bool, parse_dt
|
||||
from thoughtsync.models.note import NOTE_COLORS, Note
|
||||
from thoughtsync.unfurl_queue import detect_urls
|
||||
from thoughtsync.notes import (
|
||||
_attachment_ext,
|
||||
_header_filename,
|
||||
@@ -386,3 +387,22 @@ def test_native_spec_roundtrip_fields():
|
||||
assert spec["created_at"].year == 2026
|
||||
assert spec["labels"] == ["x"]
|
||||
assert spec["attachments"] == [{"file": "attachments/ab/img.png", "mime": "image/png"}]
|
||||
|
||||
|
||||
def test_detect_urls_finds_each_link_once_in_order():
|
||||
body = "see https://example.com/a and https://example.com/b\nand https://example.com/a again"
|
||||
assert detect_urls(body) == ["https://example.com/a", "https://example.com/b"]
|
||||
|
||||
|
||||
def test_detect_urls_trims_sentence_punctuation():
|
||||
# A URL can end in most punctuation; a SENTENCE containing one usually doesn't.
|
||||
assert detect_urls("read https://example.com/page.") == ["https://example.com/page"]
|
||||
assert detect_urls("(see https://example.com/x)") == ["https://example.com/x"]
|
||||
# …but a path that legitimately ends in a slash or a dash keeps it.
|
||||
assert detect_urls("https://example.com/dir/") == ["https://example.com/dir/"]
|
||||
|
||||
|
||||
def test_detect_urls_ignores_non_http():
|
||||
assert detect_urls("ftp://example.com and mailto:a@b.c and bare example.com") == []
|
||||
assert detect_urls(None) == []
|
||||
assert detect_urls("") == []
|
||||
|
||||
Reference in New Issue
Block a user