feat(downloads): capture structured run stats and filtered logs at save time
Backend foundation for the upcoming Downloads modal refresh. Adds three
helpers on GalleryDLService:
_compute_run_stats exit_code + downloaded/skipped/per-item-failure/
warning counts, derived from stdout+stderr
_extract_errors_warnings filters stderr to just [error]/[warning] lines
so the modal can show a concise errors-only view
_truncate_log caps a log string (default 500KB) with head+tail+
elision marker, preventing verbose multi-hour runs
from bloating the download_sessions JSONB
Wires all three into download.metadata_ in the download task. No schema
change — everything nests into the existing metadata JSONB column.
Follow-up PR will consume these fields in the Downloads.vue modal.
This commit is contained in:
@@ -400,10 +400,21 @@ async def _download_source_async(source_id: int, download_id: int = None) -> dic
|
||||
# Update download record with results
|
||||
download.completed_at = utcnow()
|
||||
download.file_count = dl_result.files_downloaded
|
||||
# Store full logs - cleanup task will age out old records
|
||||
# Persist logs in three forms for the Downloads modal:
|
||||
# stdout/stderr: raw streams, capped via _truncate_log so huge
|
||||
# verbose runs don't balloon the JSONB row.
|
||||
# stderr_errors_warnings: pre-filtered [error]/[warning] lines so
|
||||
# the UI can show a concise errors-only panel by default.
|
||||
# run_stats: structured counts for the modal's header row.
|
||||
run_stats = gdl_service._compute_run_stats(
|
||||
dl_result.return_code, dl_result.stdout, dl_result.stderr
|
||||
)
|
||||
stderr_summary = gdl_service._extract_errors_warnings(dl_result.stderr)
|
||||
download.metadata_ = {
|
||||
"stdout": dl_result.stdout if dl_result.stdout else None,
|
||||
"stderr": dl_result.stderr if dl_result.stderr else None,
|
||||
"stdout": gdl_service._truncate_log(dl_result.stdout) or None,
|
||||
"stderr": gdl_service._truncate_log(dl_result.stderr) or None,
|
||||
"stderr_errors_warnings": stderr_summary or None,
|
||||
"run_stats": run_stats,
|
||||
"duration_seconds": dl_result.duration_seconds,
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user