docs(6205): backup comments state the reason, not the instance that taught it
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 1m11s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 22s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 1m11s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 22s
The format-version log keeps what each version adds, because restore depends on what an older archive can carry; dates, milestone and issue ids, and the stories of what a missing column once cost are removed. Compatibility comments now name the archive version instead of a milestone. Two leftover history-phrased comments in embeddings and shape_ledger are restated as present-tense reasons. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+188
-225
@@ -42,92 +42,75 @@ from scribe.models.user_profile import UserProfile
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
# Backup format version. v3 (2026-06) added rulebooks/topics/rules + their
|
# Backup format version. Bump when the serialized schema changes. What each
|
||||||
# project subscription/suppression join tables. v4 (2026-07) dropped events
|
# version added, so a restore knows what an older archive can carry:
|
||||||
# when the calendar surface was retired — old v3 events are skipped on restore.
|
#
|
||||||
# v5 (2026-08) added the six tables that had accumulated outside the backup
|
# v3 rulebooks, topics and rules.
|
||||||
# entirely (#2293), and the coverage guard that stops the seventh.
|
# v4 drops calendar events; a v3 archive's events are skipped on restore.
|
||||||
# v6 (2026-08) added note_supersessions — and the guard did stop the seventh:
|
# v5 systems, record_systems, design systems and tokens, note usage events,
|
||||||
# the table shipped without a backup section and the coverage test failed the
|
# repo bindings — and the coverage guard below.
|
||||||
# build, which is the whole reason that list was written.
|
# v6 note_supersessions.
|
||||||
# v7 (2026-08) added code_shapes — the shape ledger (#2787). Classifications
|
# v7 code_shapes, the shape ledger. A restore keeps a classification only
|
||||||
# are judgment data worth carrying; a restore keeps a judgment only when its
|
# when its snippet target survives the id re-mapping, else the row rejoins
|
||||||
# snippet target survives the id re-mapping, else the row rejoins the todo.
|
# the todo.
|
||||||
# v8 (2026-08) added code_shape_events — the ledger's history (#2793): what
|
# v8 code_shape_events, the ledger's history: what was used where, when and
|
||||||
# was used where, when, and why is not recomputable, so it travels.
|
# why is not recomputable, so it travels.
|
||||||
# v9 (2026-08) added code_shape_uses — the ledger's consumption edges (#2870):
|
# v9 code_shape_uses, the ledger's consumption edges. Judgment-grade edges
|
||||||
# judgment-grade edges (agent/audit/import) are operator records; mechanical
|
# (agent/audit/import) are operator records; mechanical ones
|
||||||
# ones (reference/hook) travel too, cheaply, and the next refresh refreshes them.
|
# (reference/hook) travel too, cheaply, and the next refresh refreshes
|
||||||
# v10 (2026-08) added projects.inception + project_rulebook_exclusions
|
# them.
|
||||||
# (milestone 297): the WHY a project inherits what it does, and its opt-outs.
|
# v10 projects.inception and project_rulebook_exclusions: why a project
|
||||||
# v11 (2026-08) added the note verification trio — notes.verify_with /
|
# inherits what it does, and its opt-outs.
|
||||||
# expires_when / verified_at (milestone 317).
|
# v11 the note verification trio — verify_with / expires_when / verified_at.
|
||||||
# v12 (2026-08) closed #3182: the nine Note columns that had been missing for
|
# v12 the Note typing and provenance columns (note_type, task_kind,
|
||||||
# years (note_type, task_kind, arose_from_id, data, description, the
|
# arose_from_id, data, description, the recurrence pair, started_at,
|
||||||
# recurrence pair, started_at, completed_at), plus milestones.body — which IS
|
# completed_at), milestones.body — which IS the plan — and
|
||||||
# the plan — and repo_bindings.ref. Until v12 a restore reported success and
|
# repo_bindings.ref; and the column guard below. An archive older than
|
||||||
# handed back a corpus with every snippet and process flattened into a plain
|
# v12 restores every snippet and process as a plain note, every issue and
|
||||||
# note, every issue and spike into `work`, and every plan reduced to a title.
|
# spike as `work`, and every plan as a title.
|
||||||
# _COLUMN_EXCLUSIONS and its guard landed with it, so the next such column
|
# v13 rule_versions, a rule's edit history.
|
||||||
# fails the build instead.
|
# v14 rule_usage_events, the rule twin of note_usage_events. Carrying it is the
|
||||||
# v13 (2026-08) added rule_versions — a rule's edit history (milestone 323).
|
# WHOLE REASON the table is separate: the note importer maps note_id
|
||||||
# v14 (2026-09) added rule_usage_events — the rule twin of note_usage_events
|
# through note_id_map, so a rule id parked there would restore attached to
|
||||||
# (milestone 333). Carrying it is the WHOLE REASON the table is separate: the
|
# whatever note took that number.
|
||||||
# note importer maps note_id through note_id_map, so a rule id parked there
|
# v15 drops the rule subscription and suppression tables: a rule's scope is
|
||||||
# would restore attached to whatever note took that number.
|
# its home. Older archives carrying those sections still restore — the
|
||||||
# v15 (2026-09) dropped rulebook_subscriptions / rule_suppressions /
|
# keys are simply not read.
|
||||||
# topic_suppressions with their tables (milestone 414): a rule's scope is its
|
# v16 retrieval_tuning_events: the REASON each retrieval floor and budget is
|
||||||
# home now. Older archives carrying those sections still restore — the keys are
|
# where it is. `settings` carries the numbers; without this a restore
|
||||||
# simply not read — as do the subscribe_rulebooks inception choices they hold.
|
# keeps moved dials and drops the argument for them, which an operator
|
||||||
# v16 (2026-09) added retrieval_tuning_events (milestone 416): the REASON each
|
# needs to review a dial the model moved.
|
||||||
# retrieval floor and budget is where it is. `settings` already carried the
|
# v17 retrieval_tuning_events.embedding_model / shape_version: a floor is a
|
||||||
# numbers, so leaving this behind would restore six moved dials with the
|
# distance in ONE embedding model's geometry over documents cut one
|
||||||
# argument for them silently dropped — and from this step on those dials are
|
# particular way. Both travel NULLABLE — an unstamped row restores
|
||||||
# moved by the model, which is exactly the case where the operator needs the
|
# unstamped, because inventing the model it was measured under would turn
|
||||||
# argument to review.
|
# "unknown" into a stated fact.
|
||||||
# v17 (2026-09) added retrieval_tuning_events.embedding_model / shape_version
|
# v18 lesson_rule_links: which rule each lesson is an instance of, and the
|
||||||
# (milestone 416 step 6): a floor is a distance in ONE embedding model's
|
# judgments on each pair. A confirmed link is a judgment nothing else
|
||||||
# geometry over documents cut one particular way, so the number alone cannot
|
# records, and a rejected one stops the pair being proposed again.
|
||||||
# say whether it still measures anything. Both travel NULLABLE and unfilled —
|
# v19 lesson_no_rule: the "no rule fits" answer and its reason. Without it a
|
||||||
# a row written before the stamp existed restores unstamped, because inventing
|
# lesson judged to stand alone lands back on the unjudged list.
|
||||||
# the model it was measured under would turn "unknown" into a stated fact.
|
# v20 systems.path_patterns and system_usage_events. The usage rows restore
|
||||||
# v18 (2026-10) added lesson_rule_links (milestone 440): which rule each lesson
|
# through the SYSTEM map, for the reason the rule twin restores through the
|
||||||
# is an instance of, and the judgments that confirmed or rejected each pair. A
|
# rule map.
|
||||||
# confirmed link is a judgment nothing else records, and a rejected one is
|
# v21 moment_mappings: which of this install's actions reach which moment, and
|
||||||
# what stops the pair being proposed again — losing either undoes work.
|
# the shipped defaults it switched off. Each row is a correction an
|
||||||
# v19 (2026-10) added lesson_no_rule (#4631): the "no rule fits" answer and its
|
# operator made; dropping them puts every misfire back.
|
||||||
# reason. Without it a restored lesson that was judged to stand alone reads as
|
# v22 rule_moments: the moments each rule arrives at — a judgment about when a
|
||||||
# never judged, and lands back on the unjudged list.
|
# rule applies that nothing in the rule's text records.
|
||||||
# v20 (2026-10) added systems.path_patterns and system_usage_events (milestone
|
# v23 rule_moment_judgments: proposals waiting on a mount, and the rejections
|
||||||
# 444): the files that are each area, and whether an area's rulings were read
|
# and "no moment fits" answers that stop the same pair being proposed
|
||||||
# once shown. The usage rows restore through the SYSTEM map, for the reason
|
# again.
|
||||||
# the rule twin restores through the rule map.
|
# v24 rule_moment_judgments.misfire: reports that a mounted rule arrived where
|
||||||
# v21 (2026-10) added moment_mappings (milestone 458): which of this install's
|
# it did not apply, and the operator's "keep it" that stops them being
|
||||||
# actions reach which moment, and the shipped defaults it switched off. Each
|
# proposed as an unmount again.
|
||||||
# row is a correction an operator made in-session; a restore that dropped them
|
# v25 family canon: the platform catalog, each project's platforms, family
|
||||||
# would silently put every misfire back.
|
# ideas with their platforms and reference implementations, the adoption
|
||||||
# v22 (2026-10) added rule_moments (milestone 458): the moments each rule
|
# ledger and the decision log. Lose the ledger or the log and every
|
||||||
# arrives at. A mount is a judgment about when a rule applies that nothing in
|
# assessment is owed again, with nothing to keep it consistent with the
|
||||||
# the rule's text records, and losing it would put back exactly the misses the
|
# last.
|
||||||
# mounts were made to fix.
|
# v26 user_profiles: what each user wrote on their Profile page. Every field
|
||||||
# v23 (2026-10) added rule_moment_judgments (milestone 458 step 7): the
|
# is typed by its user, and none is derived or a credential.
|
||||||
# proposals waiting on a mount, and the rejections and "no moment fits"
|
|
||||||
# answers that stop a pass or the open-after-moment signal proposing the same
|
|
||||||
# pair again. Losing them puts every answered question back on the list.
|
|
||||||
# v24 (2026-10) added rule_moment_judgments.misfire (milestone 458 step 7b):
|
|
||||||
# the reports that a mounted rule arrived where it did not apply, and the
|
|
||||||
# operator's "keep it" that stops them being proposed as an unmount again.
|
|
||||||
# v25 (2026-10) added family canon (milestone 463): the platform catalog, each
|
|
||||||
# project's platforms, family ideas with their platforms and reference
|
|
||||||
# implementations, the adoption ledger and the decision log. The ledger is
|
|
||||||
# every project's recorded answer to a shared idea, and the log is the
|
|
||||||
# precedent each later answer follows. Lose either and every assessment is
|
|
||||||
# owed again, made fresh, with nothing to keep it consistent with the last.
|
|
||||||
# v26 (2026-10) added user_profiles (#5879): the Profile page — name, role,
|
|
||||||
# expertise, response style, tone, interests, working hours. Every field is
|
|
||||||
# typed by its user and none is derived or a credential, so leaving it out made
|
|
||||||
# a restore drop it without a word.
|
|
||||||
# Bump when the serialized schema changes.
|
|
||||||
BACKUP_VERSION = 26
|
BACKUP_VERSION = 26
|
||||||
|
|
||||||
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
|
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
|
||||||
@@ -137,55 +120,49 @@ BACKUP_VERSION = 26
|
|||||||
# The point is the ABSENCE case. A new table gets a model and a migration, both
|
# The point is the ABSENCE case. A new table gets a model and a migration, both
|
||||||
# of which fail loudly if wrong, and then silently never gets a backup section:
|
# of which fail loudly if wrong, and then silently never gets a backup section:
|
||||||
# no error, no warning, and a restore that reports success. Naming the coverage
|
# no error, no warning, and a restore that reports success. Naming the coverage
|
||||||
# explicitly turns "someone forgot" into a failing test (#2293).
|
# explicitly turns "someone forgot" into a failing test.
|
||||||
_BACKED_UP = [
|
_BACKED_UP = [
|
||||||
"users", "projects", "milestones", "notes", "task_logs", "note_drafts",
|
"users", "projects", "milestones", "notes", "task_logs", "note_drafts",
|
||||||
"note_versions", "settings", "rulebooks", "rulebook_topics", "rules",
|
"note_versions", "settings", "rulebooks", "rulebook_topics", "rules",
|
||||||
# v5 (2026-08): the five-year gap this list was written to stop.
|
|
||||||
"systems", "record_systems", "design_systems", "design_tokens",
|
"systems", "record_systems", "design_systems", "design_tokens",
|
||||||
"note_usage_events", "repo_bindings", "note_supersessions",
|
"note_usage_events", "repo_bindings", "note_supersessions",
|
||||||
# v7 (2026-08): the shape ledger (#2787); v8: its history (#2793).
|
# The shape ledger, its history and its consumption edges.
|
||||||
"code_shapes", "code_shape_events", "code_shape_uses",
|
"code_shapes", "code_shape_events", "code_shape_uses",
|
||||||
# v9 (2026-08): the global area catalog (milestone 307). Global, not
|
# The global area catalog. Global, not
|
||||||
# user-scoped, so it rides in EVERY export — including a single-user
|
# user-scoped, so it rides in EVERY export — including a single-user
|
||||||
# one, whose Systems would otherwise restore unmapped.
|
# one, whose Systems would otherwise restore unmapped.
|
||||||
"canonical_systems",
|
"canonical_systems",
|
||||||
# v10 (2026-08): a rule's area tag and its typed edges (milestone 307).
|
# A rule's area tag and its typed edges.
|
||||||
"rule_systems", "rule_relations",
|
"rule_systems", "rule_relations",
|
||||||
# v13 (2026-08): a rule's edit history (milestone 323). note_versions has
|
# A rule's edit history, for the reason note_versions travels.
|
||||||
# always travelled; its sibling has no excuse not to.
|
|
||||||
"rule_versions",
|
"rule_versions",
|
||||||
# v14 (2026-09): rule usage telemetry (milestone 333). Same argument
|
# Rule usage telemetry. Same argument
|
||||||
# note_usage_events makes for itself — pull-through is only ever
|
# note_usage_events makes for itself — pull-through is only ever
|
||||||
# accumulated, so a restore that dropped it would silently reset the
|
# accumulated, so a restore that dropped it would silently reset the
|
||||||
# measurement to zero while everything still looked fine.
|
# measurement to zero while everything still looked fine.
|
||||||
"rule_usage_events",
|
"rule_usage_events",
|
||||||
# v16 (2026-09): why each retrieval floor and budget is where it is
|
# Why each retrieval floor and budget is where it is. The values live in
|
||||||
# (milestone 416). The values live in `settings` and already travelled; the
|
# `settings`; the model moves these dials on the operator's behalf, so a
|
||||||
# argument for them had nowhere to go. Now that the model moves these dials
|
# restore that kept the numbers and dropped the reasons would leave an
|
||||||
# on the operator's behalf, a restore that kept the numbers and dropped the
|
# install tuned by nobody it can name.
|
||||||
# reasons would leave an install tuned by nobody it can name.
|
|
||||||
"retrieval_tuning_events",
|
"retrieval_tuning_events",
|
||||||
# v18 (2026-10): lesson → rule links and their judgments (milestone 440).
|
# Lesson → rule links and their judgments, and "no rule fits" answers.
|
||||||
"lesson_rule_links",
|
"lesson_rule_links",
|
||||||
# v19 (2026-10): "no rule fits" answers (#4631).
|
|
||||||
"lesson_no_rule",
|
"lesson_no_rule",
|
||||||
# v20 (2026-10): System usage telemetry (milestone 444), for the reason
|
# System usage telemetry, for the reason its note and rule twins travel.
|
||||||
# its note and rule twins travel.
|
|
||||||
"system_usage_events",
|
"system_usage_events",
|
||||||
# v21 (2026-10): an install's action → moment corrections (milestone 458).
|
# An install's action → moment corrections, the moments each rule arrives
|
||||||
|
# at, and the proposals and judgments about those mounts.
|
||||||
"moment_mappings",
|
"moment_mappings",
|
||||||
# v22 (2026-10): the moments each rule arrives at (milestone 458).
|
|
||||||
"rule_moments",
|
"rule_moments",
|
||||||
# v23 (2026-10): proposals and judgments about those mounts (step 7).
|
|
||||||
"rule_moment_judgments",
|
"rule_moment_judgments",
|
||||||
# v25 (2026-10): family canon (milestone 463). `platforms` is global like
|
# Family canon. `platforms` is global like
|
||||||
# canonical_systems and rides every export for the same reason: the rows
|
# canonical_systems and rides every export for the same reason: the rows
|
||||||
# below name platforms by slug, and a partial catalog restores partial
|
# below name platforms by slug, and a partial catalog restores partial
|
||||||
# memberships.
|
# memberships.
|
||||||
"platforms", "project_platforms", "family_ideas", "family_idea_platforms",
|
"platforms", "project_platforms", "family_ideas", "family_idea_platforms",
|
||||||
"family_idea_references", "family_adoptions", "family_decisions",
|
"family_idea_references", "family_adoptions", "family_decisions",
|
||||||
# v26 (2026-10): what each user wrote on their Profile page (#5879).
|
# What each user wrote on their Profile page.
|
||||||
"user_profiles",
|
"user_profiles",
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -197,9 +174,8 @@ _BACKED_UP = [
|
|||||||
# reads for correctness and that grows per query; the rest are
|
# reads for correctness and that grows per query; the rest are
|
||||||
# transient/operational.
|
# transient/operational.
|
||||||
#
|
#
|
||||||
# REAL table names, deliberately. This list used to read "embeddings",
|
# REAL table names, deliberately: a name that is not a table looks like
|
||||||
# "invitations", "password_resets" — none of which are tables — so it looked
|
# coverage while naming nothing the schema can confirm.
|
||||||
# like coverage while naming nothing the schema could confirm.
|
|
||||||
_NOT_INCLUDED = [
|
_NOT_INCLUDED = [
|
||||||
"groups", "group_memberships", "project_shares", "note_shares",
|
"groups", "group_memberships", "project_shares", "note_shares",
|
||||||
"api_keys", "note_embeddings", "rule_embeddings", "milestone_embeddings",
|
"api_keys", "note_embeddings", "rule_embeddings", "milestone_embeddings",
|
||||||
@@ -207,7 +183,7 @@ _NOT_INCLUDED = [
|
|||||||
"app_logs", "notifications",
|
"app_logs", "notifications",
|
||||||
"invitation_tokens", "password_reset_tokens",
|
"invitation_tokens", "password_reset_tokens",
|
||||||
"retrieval_logs",
|
"retrieval_logs",
|
||||||
# Verdicts on retrieval_logs rows (#4772), so they go where their subject
|
# Verdicts on retrieval_logs rows, so they go where their subject
|
||||||
# goes: a restore that carried a judgment of a call it did not carry would
|
# goes: a restore that carried a judgment of a call it did not carry would
|
||||||
# leave a verdict on nothing. A sample, re-takeable by the review pass.
|
# leave a verdict on nothing. A sample, re-takeable by the review pass.
|
||||||
"retrieval_judgments",
|
"retrieval_judgments",
|
||||||
@@ -215,9 +191,9 @@ _NOT_INCLUDED = [
|
|||||||
# forge tokens is a token-exfiltration file. Users re-add connections
|
# forge tokens is a token-exfiltration file. Users re-add connections
|
||||||
# after a restore; the per-project pin (projects.forge_connection_id) is
|
# after a restore; the per-project pin (projects.forge_connection_id) is
|
||||||
# deliberately not exported either, so restored projects fall back to
|
# deliberately not exported either, so restored projects fall back to
|
||||||
# keyring-by-host resolution — the documented unpinned behavior (#2778).
|
# keyring-by-host resolution — the documented unpinned behavior.
|
||||||
"forge_connections",
|
"forge_connections",
|
||||||
# Derived, like note_embeddings: the CSS consumer map (milestone 302) is
|
# Derived, like note_embeddings: the CSS consumer map is
|
||||||
# rebuilt from the repo archive by every coverage sync, and carries no
|
# rebuilt from the repo archive by every coverage sync, and carries no
|
||||||
# judgment — the first refresh after a restore recreates it exactly.
|
# judgment — the first refresh after a restore recreates it exactly.
|
||||||
"code_shape_consumers",
|
"code_shape_consumers",
|
||||||
@@ -226,20 +202,14 @@ _NOT_INCLUDED = [
|
|||||||
|
|
||||||
# Columns a backed-up table deliberately does NOT export, per table. Paired
|
# Columns a backed-up table deliberately does NOT export, per table. Paired
|
||||||
# with the column-coverage guard in tests/test_services_backup.py, this is
|
# with the column-coverage guard in tests/test_services_backup.py, this is
|
||||||
# _NOT_INCLUDED's shape one level down — and it exists because the table guard
|
# _NOT_INCLUDED's shape one level down. The table guard cannot see a column
|
||||||
# could not see the failure it was written to stop.
|
# missing from a table it counts as covered — note_type dropped would restore
|
||||||
|
# every snippet as a plain note; milestones.body dropped would restore every
|
||||||
|
# plan as a title. A column added to the model and the migration fails loudly
|
||||||
|
# if wrong; one never added to the serialiser fails silently.
|
||||||
#
|
#
|
||||||
# #2293 was six whole tables missing. #3182 was NINE COLUMNS missing from a
|
# So a new column on a backed-up table fails the build unless it is either
|
||||||
# table that had been "covered" for years: note_type and task_kind, so every
|
# exported or named here with a reason. "I forgot" is not expressible.
|
||||||
# snippet and process restored as a plain note and every issue and spike as
|
|
||||||
# `work`; arose_from_id, so every provenance edge vanished; the recurrence
|
|
||||||
# pair, so recurring tasks stopped recurring; milestones.body, which IS the
|
|
||||||
# plan; repo_bindings.ref, the branch a ledger follows. Every one arrived the
|
|
||||||
# same way — a column added to the model and the migration, both of which fail
|
|
||||||
# loudly, and then never added to the serialiser, which fails silently.
|
|
||||||
#
|
|
||||||
# So: a new column on a backed-up table now fails the build unless it is either
|
|
||||||
# exported or named here with a reason. "I forgot" is no longer expressible.
|
|
||||||
_COLUMN_EXCLUSIONS: dict[str, set[str]] = {
|
_COLUMN_EXCLUSIONS: dict[str, set[str]] = {
|
||||||
# Trash is not exported at all, so neither is the batch id that groups a
|
# Trash is not exported at all, so neither is the batch id that groups a
|
||||||
# deletion for restore(). Uniform across every soft-deletable table.
|
# deletion for restore(). Uniform across every soft-deletable table.
|
||||||
@@ -248,14 +218,14 @@ _COLUMN_EXCLUSIONS: dict[str, set[str]] = {
|
|||||||
"deleted_at", "deleted_batch_id",
|
"deleted_at", "deleted_batch_id",
|
||||||
# Credentials-adjacent, same reasoning as api_keys and
|
# Credentials-adjacent, same reasoning as api_keys and
|
||||||
# forge_connections: a restored project falls back to keyring-by-host
|
# forge_connections: a restored project falls back to keyring-by-host
|
||||||
# resolution, which is the documented unpinned behaviour (#2778).
|
# resolution, which is the documented unpinned behaviour.
|
||||||
"forge_connection_id",
|
"forge_connection_id",
|
||||||
},
|
},
|
||||||
"milestones": {"deleted_at", "deleted_batch_id"},
|
"milestones": {"deleted_at", "deleted_batch_id"},
|
||||||
"notes": {
|
"notes": {
|
||||||
"deleted_at", "deleted_batch_id",
|
"deleted_at", "deleted_batch_id",
|
||||||
# A claim is a session's attention, not the work's state (milestone
|
# A claim is a session's attention, not the work's state. Restoring
|
||||||
# 381). Restoring one would assert that a session on another install,
|
# one would assert that a session on another install,
|
||||||
# possibly long gone, is working the task right now.
|
# possibly long gone, is working the task right now.
|
||||||
"claimed_by", "claimed_at", "claim_touched_at", "claim_session",
|
"claimed_by", "claimed_at", "claim_touched_at", "claim_session",
|
||||||
},
|
},
|
||||||
@@ -339,7 +309,7 @@ _COLUMN_EXCLUSIONS: dict[str, set[str]] = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
# The same accounting for the way BACK IN (#4197). `_COLUMN_EXCLUSIONS` above
|
# The same accounting for the way BACK IN. `_COLUMN_EXCLUSIONS` above
|
||||||
# makes a dropped column unexpressible on the way out; this one does it for the
|
# makes a dropped column unexpressible on the way out; this one does it for the
|
||||||
# import, where the same mistake is worse — an export gap leaves an obviously
|
# import, where the same mistake is worse — an export gap leaves an obviously
|
||||||
# thin backup, an import gap means holding a complete, correct file and
|
# thin backup, an import gap means holding a complete, correct file and
|
||||||
@@ -553,7 +523,7 @@ def _rule_usage_event_rows(rows) -> list[dict]:
|
|||||||
{
|
{
|
||||||
"user_id": r.user_id, "rule_id": r.rule_id, "event": r.event,
|
"user_id": r.user_id, "rule_id": r.rule_id, "event": r.event,
|
||||||
"source": r.source,
|
"source": r.source,
|
||||||
# The reason a rule was departed from (#4212). Exported because
|
# The reason a rule was departed from. Exported because
|
||||||
# it is the only field on this table that cannot be recomputed:
|
# it is the only field on this table that cannot be recomputed:
|
||||||
# counts can be re-derived from a fresh install's own use, a
|
# counts can be re-derived from a fresh install's own use, a
|
||||||
# stated reason cannot, and a `departed` row that comes back
|
# stated reason cannot, and a `departed` row that comes back
|
||||||
@@ -566,7 +536,7 @@ def _rule_usage_event_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
|
|
||||||
def _moment_mapping_rows(rows) -> list[dict]:
|
def _moment_mapping_rows(rows) -> list[dict]:
|
||||||
"""An install's action → moment corrections (milestone 458). Not
|
"""An install's action → moment corrections. Not
|
||||||
`to_dict()`, for _retrieval_tuning_event_rows' reason: the restore needs
|
`to_dict()`, for _retrieval_tuning_event_rows' reason: the restore needs
|
||||||
`user_id` to remap, and the MCP reader omits it."""
|
`user_id` to remap, and the MCP reader omits it."""
|
||||||
return [
|
return [
|
||||||
@@ -581,7 +551,7 @@ def _moment_mapping_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
|
|
||||||
def _user_profile_rows(rows) -> list[dict]:
|
def _user_profile_rows(rows) -> list[dict]:
|
||||||
"""The Profile page (#5879). Not `to_dict()`: that one fills blanks with
|
"""The Profile page. Not `to_dict()`: that one fills blanks with
|
||||||
the page's defaults and omits `user_id`, and a restore needs the stored
|
the page's defaults and omits `user_id`, and a restore needs the stored
|
||||||
value and the owner."""
|
value and the owner."""
|
||||||
return [
|
return [
|
||||||
@@ -599,7 +569,7 @@ def _user_profile_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
|
|
||||||
def _retrieval_tuning_event_rows(rows) -> list[dict]:
|
def _retrieval_tuning_event_rows(rows) -> list[dict]:
|
||||||
"""The record of why a retrieval dial is where it is (#4102).
|
"""The record of why a retrieval dial is where it is.
|
||||||
|
|
||||||
Not `to_dict()`: that method serves the MCP reader, which already knows
|
Not `to_dict()`: that method serves the MCP reader, which already knows
|
||||||
whose install it is asking about, so it omits `user_id`. A restore has to
|
whose install it is asking about, so it omits `user_id`. A restore has to
|
||||||
@@ -611,8 +581,8 @@ def _retrieval_tuning_event_rows(rows) -> list[dict]:
|
|||||||
"user_id": r.user_id, "surface": r.surface, "dial": r.dial,
|
"user_id": r.user_id, "surface": r.surface, "dial": r.dial,
|
||||||
"old_value": r.old_value, "new_value": r.new_value,
|
"old_value": r.old_value, "new_value": r.new_value,
|
||||||
"actor": r.actor, "reason": r.reason,
|
"actor": r.actor, "reason": r.reason,
|
||||||
# Carried, and NOT defaulted to the current model on the way out
|
# Carried, and NOT defaulted to the current model on the way
|
||||||
# (#4104): a row that was unstamped when it was written is still
|
# out: a row that was unstamped when it was written is still
|
||||||
# unstamped after a round trip, and a backup that quietly filled
|
# unstamped after a round trip, and a backup that quietly filled
|
||||||
# the gap would turn "we don't know" into a stated fact.
|
# the gap would turn "we don't know" into a stated fact.
|
||||||
"embedding_model": r.embedding_model,
|
"embedding_model": r.embedding_model,
|
||||||
@@ -637,9 +607,9 @@ def _code_shape_use_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
def _repo_binding_rows(rows) -> list[dict]:
|
def _repo_binding_rows(rows) -> list[dict]:
|
||||||
return [
|
return [
|
||||||
# `ref` is the branch this binding's ledger follows (#2873). Without
|
# `ref` is the branch this binding's ledger follows. Without it a
|
||||||
# it a restored binding silently falls back to the default branch, and
|
# restored binding silently falls back to the default branch, and the
|
||||||
# the shape ledger starts accounting for a different tree (#3182).
|
# shape ledger starts accounting for a different tree.
|
||||||
{
|
{
|
||||||
"user_id": r.user_id, "project_id": r.project_id,
|
"user_id": r.user_id, "project_id": r.project_id,
|
||||||
"repo_key": r.repo_key, "ref": r.ref,
|
"repo_key": r.repo_key, "ref": r.ref,
|
||||||
@@ -649,9 +619,8 @@ def _repo_binding_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
|
|
||||||
# Row builders for the sections both exporters carry. Pure, like the join-table
|
# Row builders for the sections both exporters carry. Pure, like the join-table
|
||||||
# helpers above; the full and per-user exports used to restate every one of
|
# helpers above, and shared so the full and per-user exports cannot drift: a
|
||||||
# these comprehensions side by side, and a column added to one and not the
|
# column added to one and not the other is a backup that silently drops it.
|
||||||
# other is a backup that silently drops it (#2293's shape, one layer down).
|
|
||||||
|
|
||||||
def _user_rows(rows) -> list[dict]:
|
def _user_rows(rows) -> list[dict]:
|
||||||
return [
|
return [
|
||||||
@@ -685,10 +654,10 @@ def _milestone_rows(rows) -> list[dict]:
|
|||||||
{
|
{
|
||||||
"id": m.id, "user_id": m.user_id, "project_id": m.project_id,
|
"id": m.id, "user_id": m.user_id, "project_id": m.project_id,
|
||||||
"title": m.title, "description": m.description, "status": m.status,
|
"title": m.title, "description": m.description, "status": m.status,
|
||||||
# THE PLAN. A milestone IS the plan (0066) and `body` is its
|
# THE PLAN. A milestone IS the plan and `body` is its design and
|
||||||
# design and intent; `description` is only the one-line summary.
|
# intent; `description` is only the one-line summary. Dropping
|
||||||
# Dropping this restored every plan as a title with no reasoning
|
# this would restore every plan as a title with no reasoning
|
||||||
# behind it (#3182).
|
# behind it.
|
||||||
"body": m.body,
|
"body": m.body,
|
||||||
"order_index": m.order_index,
|
"order_index": m.order_index,
|
||||||
"created_at": m.created_at.isoformat(),
|
"created_at": m.created_at.isoformat(),
|
||||||
@@ -709,11 +678,10 @@ def _note_rows(rows) -> list[dict]:
|
|||||||
"due_date": n.due_date.isoformat() if n.due_date else None,
|
"due_date": n.due_date.isoformat() if n.due_date else None,
|
||||||
"created_at": n.created_at.isoformat(),
|
"created_at": n.created_at.isoformat(),
|
||||||
"updated_at": n.updated_at.isoformat(),
|
"updated_at": n.updated_at.isoformat(),
|
||||||
# WHAT KIND OF RECORD THIS IS — both typing axes (#3182). Missing
|
# WHAT KIND OF RECORD THIS IS — both typing axes. Without them a
|
||||||
# until now, which meant a restore reported success and handed back
|
# restore reports success and hands back a corpus with every
|
||||||
# a corpus with every snippet and process flattened into a plain
|
# snippet and process flattened into a plain note and every issue
|
||||||
# note and every issue and spike into `work`. Nothing recomputes
|
# and spike into `work`. Nothing recomputes these.
|
||||||
# these; the vocabulary is simply gone.
|
|
||||||
"note_type": n.note_type,
|
"note_type": n.note_type,
|
||||||
"task_kind": n.task_kind,
|
"task_kind": n.task_kind,
|
||||||
# Provenance — which record caused this one. Re-mapped in the
|
# Provenance — which record caused this one. Re-mapped in the
|
||||||
@@ -735,7 +703,7 @@ def _note_rows(rows) -> list[dict]:
|
|||||||
n.recurrence_next_spawn_at.isoformat()
|
n.recurrence_next_spawn_at.isoformat()
|
||||||
if n.recurrence_next_spawn_at else None
|
if n.recurrence_next_spawn_at else None
|
||||||
),
|
),
|
||||||
# The verification trio (milestone 317, migration 0092). Operator
|
# The verification trio. Operator
|
||||||
# judgment — "somebody checked this fact, and this is when" —
|
# judgment — "somebody checked this fact, and this is when" —
|
||||||
# which nothing can recompute.
|
# which nothing can recompute.
|
||||||
"verify_with": n.verify_with,
|
"verify_with": n.verify_with,
|
||||||
@@ -885,7 +853,7 @@ def _rule_relation_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
def _lesson_rule_link_rows(rows) -> list[dict]:
|
def _lesson_rule_link_rows(rows) -> list[dict]:
|
||||||
"""Which rule each lesson is an instance of, with the judgment's state,
|
"""Which rule each lesson is an instance of, with the judgment's state,
|
||||||
reason and evidence (milestone 440). Ids are SOURCE ids, remapped through
|
reason and evidence. Ids are SOURCE ids, remapped through
|
||||||
the note and rule maps at restore."""
|
the note and rule maps at restore."""
|
||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
@@ -899,7 +867,7 @@ def _lesson_rule_link_rows(rows) -> list[dict]:
|
|||||||
|
|
||||||
|
|
||||||
def _lesson_no_rule_rows(rows) -> list[dict]:
|
def _lesson_no_rule_rows(rows) -> list[dict]:
|
||||||
"""The "no rule fits" answers, by SOURCE lesson id (#4631)."""
|
"""The "no rule fits" answers, by SOURCE lesson id."""
|
||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
"lesson_id": r.lesson_id, "why": r.why,
|
"lesson_id": r.lesson_id, "why": r.why,
|
||||||
@@ -1322,7 +1290,7 @@ async def export_user_backup(user_id: int) -> dict:
|
|||||||
select(RuleMomentJudgment).where(RuleMomentJudgment.rule_id.in_(_rule_ids))
|
select(RuleMomentJudgment).where(RuleMomentJudgment.rule_id.in_(_rule_ids))
|
||||||
)).scalars().all() if _rule_ids else []
|
)).scalars().all() if _rule_ids else []
|
||||||
# Scoped through the RULE, not the version's user_id. That column is
|
# Scoped through the RULE, not the version's user_id. That column is
|
||||||
# the ACTOR (milestone 323), so filtering on it would carry the
|
# the ACTOR, so filtering on it would carry the
|
||||||
# versions this user wrote on someone ELSE's rule and drop the ones
|
# versions this user wrote on someone ELSE's rule and drop the ones
|
||||||
# someone else wrote on theirs — the opposite of a per-user export.
|
# someone else wrote on theirs — the opposite of a per-user export.
|
||||||
rule_versions = (await session.execute(
|
rule_versions = (await session.execute(
|
||||||
@@ -1478,26 +1446,24 @@ async def export_user_backup(user_id: int) -> dict:
|
|||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Per-table builders — the import side's twin of the row helpers above (#4197)
|
# Per-table builders — the import side's twin of the row helpers above
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
#
|
#
|
||||||
# The column guard (#3182) makes a dropped column unexpressible on the way OUT
|
# The column guard makes a dropped column unexpressible on the way OUT because
|
||||||
# because export goes through per-table pure helpers: hand one a stand-in and
|
# export goes through per-table pure helpers: hand one a stand-in and read
|
||||||
# read which KEYS came back. Nothing watched the way back IN, and that is the
|
# which KEYS came back. The way back IN is the worse half to leave unwatched —
|
||||||
# worse half — an export gap leaves an obviously thin backup, an import gap
|
# an export gap leaves an obviously thin backup, an import gap means holding a
|
||||||
# means holding a complete, correct file and restoring an incomplete database
|
# complete, correct file and restoring an incomplete database from it, with a
|
||||||
# from it, with a success message.
|
# success message.
|
||||||
#
|
#
|
||||||
# The guard was one-sided because the code was. `_restore_v2` was one long
|
# These builders are the per-table unit the import guard hands a stand-in to.
|
||||||
# procedural function with the model construction inlined in each loop, so
|
# Each takes an exported row plus the id maps built so far and returns either
|
||||||
# there was no per-table unit to hand a stand-in to. These builders are that
|
# the model instance to add or None for a row that cannot be restored.
|
||||||
# unit. Each takes an exported row plus the id maps built so far and returns
|
|
||||||
# either the model instance to add or None for a row that cannot be restored.
|
|
||||||
#
|
#
|
||||||
# WHAT DELIBERATELY DID NOT MOVE. The loops, the flushes and the id-map
|
# WHAT DELIBERATELY STAYS IN `_restore_v2`: the loops, the flushes and the
|
||||||
# bookkeeping stay in `_restore_v2`: that is the order-dependent part, where a
|
# id-map bookkeeping. That is the order-dependent part, where a mistake is a
|
||||||
# mistake is a restore that half-works, and it gains nothing from being split.
|
# restore that half-works, and it gains nothing from being split. Only the
|
||||||
# Only the construction moved, because the construction is the only thing the
|
# construction lives here, because the construction is the only thing the
|
||||||
# guard has to see.
|
# guard has to see.
|
||||||
#
|
#
|
||||||
# RETURNING None IS "SKIP", A None FIELD IS "DEGRADE", and the difference is
|
# RETURNING None IS "SKIP", A None FIELD IS "DEGRADE", and the difference is
|
||||||
@@ -1589,7 +1555,7 @@ def _build_milestone(row: dict, maps: _Maps) -> Milestone | None:
|
|||||||
def _build_note(row: dict, maps: _Maps) -> Note | None:
|
def _build_note(row: dict, maps: _Maps) -> Note | None:
|
||||||
"""`parent_id` and `arose_from_id` land NULL and are patched afterwards.
|
"""`parent_id` and `arose_from_id` land NULL and are patched afterwards.
|
||||||
|
|
||||||
Both are ids in the SOURCE database (#3182), and the target note may not
|
Both are ids in the SOURCE database, and the target note may not
|
||||||
have been created yet when this row is built.
|
have been created yet when this row is built.
|
||||||
"""
|
"""
|
||||||
uid = maps.users.get(row.get("user_id", 0))
|
uid = maps.users.get(row.get("user_id", 0))
|
||||||
@@ -1733,9 +1699,9 @@ def _build_retrieval_tuning_event(row: dict, maps: _Maps) -> RetrievalTuningEven
|
|||||||
new_value=row.get("new_value"),
|
new_value=row.get("new_value"),
|
||||||
actor=row.get("actor") or "model",
|
actor=row.get("actor") or "model",
|
||||||
reason=row.get("reason", ""),
|
reason=row.get("reason", ""),
|
||||||
# .get with no default, deliberately (v17): an archive written before
|
# .get with no default, deliberately: an archive older than v17 has
|
||||||
# the stamp existed has no key here, and None is the right answer for
|
# no key here, and None is the right answer for it — the same
|
||||||
# it — the same "unstamped" a pre-#4104 row carries in place.
|
# "unstamped" an unstamped row carries in place.
|
||||||
embedding_model=row.get("embedding_model"),
|
embedding_model=row.get("embedding_model"),
|
||||||
shape_version=row.get("shape_version"),
|
shape_version=row.get("shape_version"),
|
||||||
created_at=_dt(row.get("created_at")),
|
created_at=_dt(row.get("created_at")),
|
||||||
@@ -1784,16 +1750,15 @@ def _build_rule(row: dict, maps: _Maps) -> Rule | None:
|
|||||||
why=row.get("why") or None,
|
why=row.get("why") or None,
|
||||||
how_to_apply=row.get("how_to_apply") or None,
|
how_to_apply=row.get("how_to_apply") or None,
|
||||||
when_to_apply=row.get("when_to_apply") or None,
|
when_to_apply=row.get("when_to_apply") or None,
|
||||||
# A file written before milestone 394 carries `tier` and `always_on`;
|
# An older archive may carry `tier` and `always_on`; neither is read.
|
||||||
# neither is read. Dropping a field the schema no longer has is the
|
# Dropping a field the schema does not have is the tolerant direction
|
||||||
# tolerant direction — an archive records what WAS, and refusing it
|
# — an archive records what WAS, and refusing it because it remembers
|
||||||
# because it remembers a deleted column would make every pre-394
|
# a deleted column would make every such backup unrestorable.
|
||||||
# backup unrestorable.
|
|
||||||
#
|
#
|
||||||
# Same shape, same reason: a file written before 0098 has no kind, and
|
# Same shape, same reason: an archive with no `kind` predates
|
||||||
# every rule in it was a rule. Defaulting the other way would restore
|
# preferences, and every rule in it is a rule. Defaulting the other
|
||||||
# an old backup with things that had always bound quietly no longer
|
# way would restore an old backup with things that had always bound
|
||||||
# binding.
|
# quietly not binding.
|
||||||
kind=row.get("kind") or "rule",
|
kind=row.get("kind") or "rule",
|
||||||
verify_with=row.get("verify_with") or None,
|
verify_with=row.get("verify_with") or None,
|
||||||
expires_when=row.get("expires_when") or None,
|
expires_when=row.get("expires_when") or None,
|
||||||
@@ -1802,9 +1767,8 @@ def _build_rule(row: dict, maps: _Maps) -> Rule | None:
|
|||||||
# it would put every constraint at the top of the sweep with nothing
|
# it would put every constraint at the top of the sweep with nothing
|
||||||
# having actually changed.
|
# having actually changed.
|
||||||
verified_at=_dt_or_none(row.get("verified_at")),
|
verified_at=_dt_or_none(row.get("verified_at")),
|
||||||
# Remapped through the note map like every other note edge. Exported
|
# Remapped through the note map like every other note edge; without
|
||||||
# since 0088 but dropped on the way back in until milestone 312 — a
|
# it a restore silently loses every rule's provenance link. SET NULL
|
||||||
# restore silently lost every rule's provenance link. SET NULL
|
|
||||||
# semantics apply here too: a source note that didn't restore leaves
|
# semantics apply here too: a source note that didn't restore leaves
|
||||||
# the rule intact.
|
# the rule intact.
|
||||||
arose_from_id=maps.notes.get(row.get("arose_from_id") or 0),
|
arose_from_id=maps.notes.get(row.get("arose_from_id") or 0),
|
||||||
@@ -2056,7 +2020,7 @@ def _build_system(row: dict, maps: _Maps) -> System | None:
|
|||||||
color=row.get("color"),
|
color=row.get("color"),
|
||||||
status=row.get("status", "active"),
|
status=row.get("status", "active"),
|
||||||
order_index=row.get("order_index", 0),
|
order_index=row.get("order_index", 0),
|
||||||
# Absent in a backup taken before milestone 444: no paths, not an error.
|
# Absent in an archive older than v20: no paths, not an error.
|
||||||
path_patterns=list(row.get("path_patterns") or []),
|
path_patterns=list(row.get("path_patterns") or []),
|
||||||
# An unknown slug restores UNMAPPED rather than failing: the System and
|
# An unknown slug restores UNMAPPED rather than failing: the System and
|
||||||
# its records are the payload, the mapping is an aid.
|
# its records are the payload, the mapping is an aid.
|
||||||
@@ -2142,7 +2106,7 @@ def _build_rule_usage_event(row: dict, maps: _Maps) -> RuleUsageEvent | None:
|
|||||||
"""Resolved through the RULE map, not the note map. A rule id run through
|
"""Resolved through the RULE map, not the note map. A rule id run through
|
||||||
the note map would either drop (best case) or land on whatever note took
|
the note map would either drop (best case) or land on whatever note took
|
||||||
that number, producing telemetry that is wrong rather than missing and
|
that number, producing telemetry that is wrong rather than missing and
|
||||||
that nothing downstream could detect (milestone 333)."""
|
that nothing downstream could detect."""
|
||||||
rid = maps.rules.get(row.get("rule_id", 0))
|
rid = maps.rules.get(row.get("rule_id", 0))
|
||||||
if rid is None:
|
if rid is None:
|
||||||
return None
|
return None
|
||||||
@@ -2217,10 +2181,10 @@ def _build_code_shape(row: dict, maps: _Maps) -> CodeShape | None:
|
|||||||
status=status,
|
status=status,
|
||||||
snippet_id=snippet_id,
|
snippet_id=snippet_id,
|
||||||
reason=row.get("reason"),
|
reason=row.get("reason"),
|
||||||
# Exported since #2874 and dropped on the way back in until #4197 —
|
# Without it every judged row restores with its verdict and without
|
||||||
# every judged row restored with its verdict and without the code for
|
# the code for WHY, which is the column the accounting reads to tell a
|
||||||
# WHY, which is the column the accounting reads to tell a scoped-css
|
# scoped-css exemption from a convention-plumbing one. Kept on a
|
||||||
# exemption from a convention-plumbing one. Kept on a downgrade for
|
# downgrade for
|
||||||
# the same reason `reason` is: the free text and its code are one
|
# the same reason `reason` is: the free text and its code are one
|
||||||
# statement, and the existing downgrade clears the judgment's author
|
# statement, and the existing downgrade clears the judgment's author
|
||||||
# and date, not its argument.
|
# and date, not its argument.
|
||||||
@@ -2230,20 +2194,20 @@ def _build_code_shape(row: dict, maps: _Maps) -> CodeShape | None:
|
|||||||
first_seen_commit=row.get("first_seen_commit", ""),
|
first_seen_commit=row.get("first_seen_commit", ""),
|
||||||
last_seen_commit=row.get("last_seen_commit", ""),
|
last_seen_commit=row.get("last_seen_commit", ""),
|
||||||
vanished_at=_dt(row["vanished_at"]) if row.get("vanished_at") else None,
|
vanished_at=_dt(row["vanished_at"]) if row.get("vanished_at") else None,
|
||||||
# Fingerprints restore; proposals (#2792) deliberately do not — they
|
# Fingerprints restore; proposals deliberately do not — they
|
||||||
# are mechanical, and the next refresh recomputes them against the
|
# are mechanical, and the next refresh recomputes them against the
|
||||||
# restored snippet ids.
|
# restored snippet ids.
|
||||||
signature=row.get("signature", ""),
|
signature=row.get("signature", ""),
|
||||||
body_sha=row.get("body_sha", ""),
|
body_sha=row.get("body_sha", ""),
|
||||||
classified_sha=row.get("classified_sha", ""),
|
classified_sha=row.get("classified_sha", ""),
|
||||||
# Both exported and both dropped until #4197. `recheck_at` is the
|
# `recheck_at` is the standing "this judgment asks to be confirmed
|
||||||
# standing "this judgment asks to be confirmed again" flag, so losing
|
# again" flag, so losing
|
||||||
# it restores a tree that looks settled and is not.
|
# it restores a tree that looks settled and is not.
|
||||||
recheck_at=_dt(row["recheck_at"]) if row.get("recheck_at") else None,
|
recheck_at=_dt(row["recheck_at"]) if row.get("recheck_at") else None,
|
||||||
# RE-MAPPED, not carried: it is a FK to notes.id like `snippet_id`,
|
# RE-MAPPED, not carried: it is a FK to notes.id like `snippet_id`,
|
||||||
# and a raw source id would point at whatever snippet took that
|
# and a raw source id would point at whatever snippet took that
|
||||||
# number in the destination — wrong rather than missing, which is the
|
# number in the destination — wrong rather than missing, the same
|
||||||
# milestone 333 trap one table over.
|
# trap `_build_rule_usage_event` avoids.
|
||||||
diverges_from=maps.notes.get(row.get("diverges_from") or 0),
|
diverges_from=maps.notes.get(row.get("diverges_from") or 0),
|
||||||
created_at=_dt(row.get("created_at")),
|
created_at=_dt(row.get("created_at")),
|
||||||
updated_at=_dt(row.get("updated_at")),
|
updated_at=_dt(row.get("updated_at")),
|
||||||
@@ -2279,8 +2243,8 @@ def _build_code_shape_event(row: dict, maps: _Maps) -> CodeShapeEvent | None:
|
|||||||
|
|
||||||
|
|
||||||
def _build_code_shape_use(row: dict, maps: _Maps) -> CodeShapeUse | None:
|
def _build_code_shape_use(row: dict, maps: _Maps) -> CodeShapeUse | None:
|
||||||
"""Consumption edges (#2870) ride their shape AND their snippet — both
|
"""Consumption edges ride their shape AND their snippet — both ends
|
||||||
ends must have survived, or the edge is no longer a fact."""
|
must have survived, or the edge is no longer a fact."""
|
||||||
shape_id = maps.shapes.get(row.get("shape_id") or 0)
|
shape_id = maps.shapes.get(row.get("shape_id") or 0)
|
||||||
snippet_id = maps.notes.get(row.get("snippet_id") or 0)
|
snippet_id = maps.notes.get(row.get("snippet_id") or 0)
|
||||||
if shape_id is None or snippet_id is None:
|
if shape_id is None or snippet_id is None:
|
||||||
@@ -2377,7 +2341,7 @@ async def _restore_v1(data: dict) -> dict:
|
|||||||
# Patch the two note->note edges now that every note has a new id.
|
# Patch the two note->note edges now that every note has a new id.
|
||||||
# Both are ids in the SOURCE database, so writing either straight into
|
# Both are ids in the SOURCE database, so writing either straight into
|
||||||
# the constructor would point at whatever record happens to hold that
|
# the constructor would point at whatever record happens to hold that
|
||||||
# number here — a restore that succeeds and silently re-parents (#3182).
|
# number here — a restore that succeeds and silently re-parents.
|
||||||
# An edge whose target did not survive the import is left NULL rather
|
# An edge whose target did not survive the import is left NULL rather
|
||||||
# than guessed at.
|
# than guessed at.
|
||||||
for n_data in data.get("notes", []):
|
for n_data in data.get("notes", []):
|
||||||
@@ -2417,7 +2381,7 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
|
|
||||||
Every model is constructed by a `_build_*` helper above, never inline.
|
Every model is constructed by a `_build_*` helper above, never inline.
|
||||||
That is what gives the import side a per-table unit the column guard can
|
That is what gives the import side a per-table unit the column guard can
|
||||||
hand a stand-in to (#4197); what stays here is the ordering, the flushes
|
hand a stand-in to; what stays here is the ordering, the flushes
|
||||||
and the id-map bookkeeping, which is the part a split would only make
|
and the id-map bookkeeping, which is the part a split would only make
|
||||||
harder to read.
|
harder to read.
|
||||||
"""
|
"""
|
||||||
@@ -2488,8 +2452,8 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
stats["notes"] += 1
|
stats["notes"] += 1
|
||||||
|
|
||||||
# 4b. Patch the note->note edges. Deferred for the same reason
|
# 4b. Patch the note->note edges. Deferred for the same reason
|
||||||
# parent_id always has been: these are ids in the SOURCE database
|
# parent_id is: these are ids in the SOURCE database. An edge whose
|
||||||
# (#3182). An edge whose target did not survive stays NULL.
|
# target did not survive stays NULL.
|
||||||
for new_note_id, old_parent_id in notes_with_parents:
|
for new_note_id, old_parent_id in notes_with_parents:
|
||||||
new_parent_id = maps.notes.get(old_parent_id)
|
new_parent_id = maps.notes.get(old_parent_id)
|
||||||
if new_parent_id:
|
if new_parent_id:
|
||||||
@@ -2537,8 +2501,8 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
|
|
||||||
# 8b. Retrieval tuning history (v16) — restored beside the settings it
|
# 8b. Retrieval tuning history (v16) — restored beside the settings it
|
||||||
# explains, and for the same reason: the numbers above are the state,
|
# explains, and for the same reason: the numbers above are the state,
|
||||||
# these rows are the argument for it. From milestone 416 those dials
|
# these rows are the argument for it. Those dials move on the
|
||||||
# move on the operator's behalf, so an install restored with the values
|
# operator's behalf, so an install restored with the values
|
||||||
# and without the reasons is one tuned by nobody it can name.
|
# and without the reasons is one tuned by nobody it can name.
|
||||||
for t_data in data.get("retrieval_tuning_events", []):
|
for t_data in data.get("retrieval_tuning_events", []):
|
||||||
event = _build_retrieval_tuning_event(t_data, maps)
|
event = _build_retrieval_tuning_event(t_data, maps)
|
||||||
@@ -2596,15 +2560,14 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
|
|
||||||
# 12-14. Rulebook subscriptions, rule and topic suppressions (v3-v14)
|
# 12-14. Rulebook subscriptions, rule and topic suppressions (v3-v14)
|
||||||
# `rulebook_subscriptions`, `rule_suppressions` and `topic_suppressions`
|
# `rulebook_subscriptions`, `rule_suppressions` and `topic_suppressions`
|
||||||
# are READ BY NOBODY since milestone 414 dropped their tables. An
|
# are READ BY NOBODY: their tables do not exist. An archive carrying
|
||||||
# archive carrying them still imports, for the reason 14b gives.
|
# them still imports, for the reason 14b gives.
|
||||||
|
|
||||||
# 14b. Always-on rulebook exclusions (v10, milestone 297)
|
# 14b. Always-on rulebook exclusions (v10)
|
||||||
# `rulebook_exclusions` was a v10 section and is READ BY NOBODY since
|
# `rulebook_exclusions` is a v10 section READ BY NOBODY: its table
|
||||||
# milestone 394 removed the table. An archive carrying it still
|
# does not exist. An archive carrying it still imports — the key is
|
||||||
# imports — the key is simply not looked at — because refusing a
|
# simply not looked at — because refusing a backup for remembering
|
||||||
# backup for remembering something we deleted would make every v10-v13
|
# something deleted would make every such archive unrestorable.
|
||||||
# archive unrestorable.
|
|
||||||
|
|
||||||
# --- v5 sections. Every one is data.get()-guarded, so a v2/v3/v4
|
# --- v5 sections. Every one is data.get()-guarded, so a v2/v3/v4
|
||||||
# payload restores without them rather than failing on an absent key.
|
# payload restores without them rather than failing on an absent key.
|
||||||
@@ -2669,7 +2632,7 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
session.add(relation)
|
session.add(relation)
|
||||||
stats["rule_relations"] += 1
|
stats["rule_relations"] += 1
|
||||||
|
|
||||||
# Lesson → rule links (milestone 440): after both notes and rules are
|
# Lesson → rule links: after both notes and rules are
|
||||||
# mapped, which is why they sit beside the rule edges. Archives before
|
# mapped, which is why they sit beside the rule edges. Archives before
|
||||||
# v18 carry no section and restore with none.
|
# v18 carry no section and restore with none.
|
||||||
for lr in data.get("lesson_rule_links", []):
|
for lr in data.get("lesson_rule_links", []):
|
||||||
@@ -2678,7 +2641,7 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
continue
|
continue
|
||||||
session.add(link)
|
session.add(link)
|
||||||
stats["lesson_rule_links"] += 1
|
stats["lesson_rule_links"] += 1
|
||||||
# "No rule fits" answers (#4631); archives before v19 carry none.
|
# "No rule fits" answers; archives before v19 carry none.
|
||||||
for nr in data.get("lesson_no_rule", []):
|
for nr in data.get("lesson_no_rule", []):
|
||||||
answer = _build_lesson_no_rule(nr, maps)
|
answer = _build_lesson_no_rule(nr, maps)
|
||||||
if answer is None:
|
if answer is None:
|
||||||
@@ -2746,9 +2709,9 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
maps.decisions[int(fd["id"])] = decision.id
|
maps.decisions[int(fd["id"])] = decision.id
|
||||||
stats["family_decisions"] += 1
|
stats["family_decisions"] += 1
|
||||||
|
|
||||||
# A rule's edit history (milestone 323). Must come after the rules
|
# A rule's edit history. Must come after the rules themselves — the
|
||||||
# themselves — the rule map is only populated above — and both ids are
|
# rule map is only populated above — and both ids are ids in the
|
||||||
# ids in the SOURCE database, which is #3182's arose_from_id trap.
|
# SOURCE database, the same trap as a note's arose_from_id.
|
||||||
for rv in data.get("rule_versions", []):
|
for rv in data.get("rule_versions", []):
|
||||||
version = _build_rule_version(rv, maps)
|
version = _build_rule_version(rv, maps)
|
||||||
if version is None:
|
if version is None:
|
||||||
@@ -2834,7 +2797,7 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
session.add(binding)
|
session.add(binding)
|
||||||
stats["repo_bindings"] += 1
|
stats["repo_bindings"] += 1
|
||||||
|
|
||||||
# 21. Code shapes (v7, #2787)
|
# 21. Code shapes (v7)
|
||||||
for cs_data in data.get("code_shapes", []):
|
for cs_data in data.get("code_shapes", []):
|
||||||
shape = _build_code_shape(cs_data, maps)
|
shape = _build_code_shape(cs_data, maps)
|
||||||
if shape is None:
|
if shape is None:
|
||||||
@@ -2853,7 +2816,7 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
session.add(event)
|
session.add(event)
|
||||||
stats["code_shape_events"] += 1
|
stats["code_shape_events"] += 1
|
||||||
|
|
||||||
# v9: consumption edges (#2870).
|
# v9: consumption edges.
|
||||||
for use in data.get("code_shape_uses", []):
|
for use in data.get("code_shape_uses", []):
|
||||||
edge = _build_code_shape_use(use, maps)
|
edge = _build_code_shape_use(use, maps)
|
||||||
if edge is None:
|
if edge is None:
|
||||||
@@ -2882,13 +2845,13 @@ async def _restore_v2(data: dict) -> dict:
|
|||||||
inception = p_data.get("inception")
|
inception = p_data.get("inception")
|
||||||
if isinstance(inception, dict):
|
if isinstance(inception, dict):
|
||||||
choices = dict(inception.get("choices") or {})
|
choices = dict(inception.get("choices") or {})
|
||||||
# DROPPED, not remapped (milestone 394). A pre-394 archive
|
# DROPPED, not remapped. An older archive may carry the
|
||||||
# carries the retired exclusion choice; restoring it would put
|
# retired exclusion choice; restoring it would put
|
||||||
# a key back that `validate_inception` now rejects as unknown,
|
# a key back that `validate_inception` now rejects as unknown,
|
||||||
# so the next edit to that project would fail on data this
|
# so the next edit to that project would fail on data this
|
||||||
# importer wrote.
|
# importer wrote.
|
||||||
choices.pop("exclude_always_on_rulebooks", None)
|
choices.pop("exclude_always_on_rulebooks", None)
|
||||||
# Same for subscribe_rulebooks since milestone 414.
|
# Same for the retired subscribe_rulebooks choice.
|
||||||
choices.pop("subscribe_rulebooks", None)
|
choices.pop("subscribe_rulebooks", None)
|
||||||
ds = choices.get("design_system_id")
|
ds = choices.get("design_system_id")
|
||||||
choices["design_system_id"] = maps.design_systems.get(ds) if ds else None
|
choices["design_system_id"] = maps.design_systems.get(ds) if ds else None
|
||||||
|
|||||||
@@ -224,8 +224,8 @@ async def _apply_supersession_penalty(
|
|||||||
stale = await superseded_ids([int(note.id) for _score, note in scored])
|
stale = await superseded_ids([int(note.id) for _score, note in scored])
|
||||||
except Exception:
|
except Exception:
|
||||||
# Fail OPEN, and the direction matters: ranking without the penalty is
|
# Fail OPEN, and the direction matters: ranking without the penalty is
|
||||||
# the behaviour that shipped for months. Returning nothing, or raising,
|
# a slightly worse search. Returning nothing, or raising, would turn a
|
||||||
# would turn a supersession-lookup hiccup into a broken search.
|
# supersession-lookup hiccup into a broken one.
|
||||||
logger.warning("Supersession lookup failed — ranking unpenalised", exc_info=True)
|
logger.warning("Supersession lookup failed — ranking unpenalised", exc_info=True)
|
||||||
return scored[:limit]
|
return scored[:limit]
|
||||||
|
|
||||||
|
|||||||
@@ -985,13 +985,12 @@ PULL_WINDOW = timedelta(hours=6)
|
|||||||
# THE PROBLEM THIS EXISTS FOR. `kind` has exactly two values, so a frozen
|
# THE PROBLEM THIS EXISTS FOR. `kind` has exactly two values, so a frozen
|
||||||
# dataclass, a module constant, a sync predicate, a class and an async service
|
# dataclass, a module constant, a sync predicate, a class and an async service
|
||||||
# function are all siblings of one another. Nothing downstream could possibly
|
# function are all siblings of one another. Nothing downstream could possibly
|
||||||
# discriminate on a distinction the column does not carry — which is how a
|
# discriminate on a distinction the column does not carry, which leaves a
|
||||||
# directory base rate ended up standing alone as the entire divergence
|
# directory base rate standing alone as the entire divergence argument. The
|
||||||
# argument. It was not chosen over a better signal; it was the only signal
|
# consequence: a registry module of pure helpers gets prompted to build from
|
||||||
# there was. Measured consequence: writing a registry module of pure helpers
|
# an `async_session` service canon, and one snippet can be recorded as the
|
||||||
# produced five prompts to build them from the `async_session` service canon,
|
# canon for a `class`, a plain `def`, an `async def` and a dozen test
|
||||||
# and on another project one snippet had been recorded as the canon for a
|
# functions at once.
|
||||||
# `class`, a plain `def`, an `async def` and a dozen test functions at once.
|
|
||||||
#
|
#
|
||||||
# DERIVED FROM THE SIGNATURE ON READ, not stored. `kind` is part of the row
|
# DERIVED FROM THE SIGNATURE ON READ, not stored. `kind` is part of the row
|
||||||
# identity — ("project_id", "repo_key", "path", "symbol", "kind") — so
|
# identity — ("project_id", "repo_key", "path", "symbol", "kind") — so
|
||||||
|
|||||||
Reference in New Issue
Block a user