Skip to content

feat(#3996): classify generated-media save failures into safe user-visible reasons - #4025

Draft
aheritier wants to merge 4 commits into
workspace-media-foundationsfrom
workspace-media-materialization
Draft

feat(#3996): classify generated-media save failures into safe user-visible reasons#4025
aheritier wants to merge 4 commits into
workspace-media-foundationsfrom
workspace-media-materialization

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

What

Classifies generated-media save failures into safe user-visible reasons, with runtime/session persistence, provider placeholders, and documentation/tests.

Why

Users need actionable failures when generated media cannot be persisted, without leaking filesystem or provider internals.

Validation

task test; materialization, failure-classification, manifest, migration, and provider tests.

Test instructions

Run:

task test

Generate media while forcing its save operation to fail. Expected: the user sees a safe, actionable classified reason without filesystem or provider internals, and the session remains recoverable.

@aheritier aheritier added area/docs Documentation changes area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/gemini Google Gemini provider support area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/sessions For features/issues/fixes related to session lifecycle (resume, persistence, export) kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 9ffaa5a to 2b15b2e Compare August 24, 2026 15:13
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 2b15b2e to 3dadcf6 Compare August 25, 2026 07:59
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 3dadcf6 to b07ac83 Compare August 25, 2026 08:16
@aheritier
aheritier force-pushed the workspace-media-materialization branch from b07ac83 to 5e896ae Compare August 25, 2026 20:24
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 5e896ae to ef641a7 Compare August 26, 2026 11:09
@aheritier
aheritier force-pushed the workspace-media-materialization branch 2 times, most recently from 3d8769a to ef641a7 Compare September 1, 2026 16:12
@aheritier
aheritier force-pushed the workspace-media-materialization branch from ef641a7 to 7bc0365 Compare September 1, 2026 16:30
Extend the shared chat.MessageDelta streaming pipeline into persistence:
recordAssistantMessage materializes every streamed media delta through
pkg/workspacemedia into the owning session's workspace (the effective
WorkingDir, resolved parent-chain-aware via session.ResolveWorkingDir) and
appends a document part carrying only an owner-qualified, workspace-relative
reference (ArtifactRoot=workspace) plus MIME/name/size — never raw bytes —
so session JSON never carries generated image base64. The owning session ID
is recorded on the part so copies into branched/forked sessions keep naming
the original owner; a provider display name the writer refuses falls back
to a generic generated-N rather than losing the item, and when no workspace
root is available every item fails with a per-item warning — there is
deliberately no other target location, so generated files never land
outside the workspace.

A default message transform (BuiltinStripGeneratedMedia) strips these
parts from outgoing provider history on later turns, replacing each with a
short, stable placeholder (position/count, sanitized name, MIME) so a
media-only reply never becomes an empty turn; it runs before
strip_unsupported_modalities, and compaction counts a Content/MultiContent
text mirror once. Branch/fork owner qualification, transform order, and
per-item placeholder behavior are covered by tests.
Provider-supplied display names and MIME types are untrusted model output;
this hardens every place they are stored or shown:

- Every stored or displayed display name now goes through the shared
  chat.SanitizeDisplayName helper (introduced with the Gemini request
  diagnostics), so control characters, path separators, traversal-like
  sequences, and angle brackets can never forge an XML/tag boundary in
  harness prompts, and every field is bounded by MaxSanitizedFieldBytes
  (128, UTF-8-safe) via chat.TruncateUTF8Bytes.
- sanitizeMimeType trims control characters, enforces the same field
  bound, and requires conservative RFC 6838 type/subtype syntax, falling
  back to application/octet-stream; only the safe MIME is ever persisted.
- Materialization warnings and strip-time placeholders are capped at 512
  bytes after formatting, fall back to canonical display names, and never
  include paths or raw OS errors; harness prompt interpolation runs both
  sanitizers first, closing an XML/role injection path.
- strip_unsupported_modalities skips generated-media parts independently
  of transform registration order, so a reordering can never silently
  drop a generated artifact before its placeholder replaces it.

Tests cover the classification chain end-to-end: bounded/overlong
metadata, empty-name/MIME fallbacks, multi-artifact placeholders per
provider converter, ownerless marker preservation, harness
injection inertness, and genuine single-call batch partial success.
Add a generated-media manifest to the session stores (migration
028_add_generated_media_manifest_table on the SQLite store, an in-memory
map on the memory store) recording owner session + final relative path +
sanitized MIME + creation time, written by materialization only after a
successful workspace write. Resolution must verify references against it,
so tampered session JSON can never select an arbitrary workspace file such
as .env; the store API also rejects absolute/traversal/backslash/NUL path
shapes on both add and lookup. The table has no FK to sessions
(materialization can precede the lazily persisted session row);
DeleteSession prunes it explicitly.

The persisted part keeps carrying ArtifactRoot=workspace plus the exact
final workspace-relative path the writer returned; strip/no-resend
predicates still key on a non-empty ArtifactPath, and a reference whose
root kind is empty/unknown stays unresolvable by design.

No workspace root (or an unwritable one) keeps the existing per-item
sanitized warning contract with no data-dir fallback, preserving the
turn's text and surviving siblings. Writer MIME/extension corrections now
surface a bounded notice naming the final path; collision suffixes are
persisted verbatim. Prompt-directed naming and the out-of-workspace
confirmation flow are deliberately not part of this slice, and the TUI
resolver is not yet wired to the manifest/workspace root.

Also harden session.ResolveWorkingDir: stored roots must be exactly clean
(filepath.Clean(dir) == dir), rejecting absolute values smuggling ..
segments.
…sible reasons

Replace the generic "see debug log for details" materialization and
manifest warnings with fixed, classified sentences: no session
workspace, unwritable/read-only location, deleted root, filename
collision exhaustion, refused save path, and a saved-but-unrecorded
manifest failure that may not render inline. Unclassified causes tell
the user to enable --debug and retry. Every reason is a constant, so
no absolute path, session ID, provider metadata, or raw OS error can
leak into a WarningEvent; the detailed error still reaches the debug
log only. Add workspacemedia.ErrNameExhausted so collision exhaustion
is matchable without echoing the requested path.
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 7bc0365 to 41c56eb Compare September 1, 2026 19:58

@aheritier aheritier left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the three-dot delta of this PR only (9ffb3af94...41c56eba, 4 commits, 28 files). Validated in a throwaway checkout: go build ./... and GOOS=windows go build ./... clean; go test -count=1 -race and -count=3 green on pkg/runtime, pkg/session, pkg/workspacemedia, pkg/compaction, pkg/chat and the three provider packages; golangci-lint run 0 issues.

CI (head 41c56eb): 15 success, 3 skipped, 1 failure. windows-tests fails on TestHTTPPostHonoursContextCancellation ("579.2702ms" is not less than "250ms", pkg/hooks/builtins/http_post_test.go:171) — a wall-clock assertion in a package this PR does not touch, and the only failing package in the run. Not caused by this change, but CI is not green, so this is a COMMENT, not an approval (also still draft).

What this PR gets right

  • The classification is structural, not text matching: errors.Is on ErrWorkingDirUnavailable, ErrNameExhausted, ErrPathEscape, fs.ErrPermission, EROFS, fs.ErrNotExist (pkg/runtime/media_save_failure.go:24-34), and every returned reason is a compile-time constant, so no path, session ID or OS error string can reach the warning. Verified load-bearing: replacing one branch with err.Error() under go test -overlay fails TestMediaSaveFailureReason and TestMaterializeGeneratedMedia_ClassifiedWriteFailureReasons.
  • It resolves the dangling doc references flagged on #4023: DocumentSource.ArtifactPath / ArtifactRoot / ArtifactOwnerSessionID are introduced in pkg/chat/document.go, and recordAssistantMessage no longer discards media-only turns (pkg/runtime/loop.go:877, :1177).
  • Failure surfaces reuse the existing Warning/WarningEvent path — no new event type, so JSON/API clients are unaffected.
  • The migration catalogue change updates the pinned digest in the same commit (pkg/session/migrations_pinned_test.go:42), and DeleteSession manifest pruning is load-bearing (neutering the DELETE fails TestGeneratedMediaManifest_DeleteSessionPrunesRecords).

[blocking] The len(res.Media) == 0 guard in runTurn has no test; regressing it produces a false rate-limit diagnostic on every media-only turn.
pkg/runtime/loop.go:877. Reverting just that clause to the pre-PR condition under go test -overlay leaves the entire package green:

=== M2: revert len(res.Media)==0 at loop.go:877
ok  	github.com/docker/docker-agent/pkg/runtime	2.262s

A short probe on the existing scaffolding catches it immediately:

WARNING EMITTED: "Model test/mock-model returned an empty response (stop reason: stop).
  This usually means the provider rate-limited the request or the output token limit was reached."

The sibling guard at :1177 is covered (reverting it fails TestRunStream_MediaOnlyAssistantHistoryRemainsCoherent and …_UnknownModel), so this is an isolated gap, not a philosophy difference. emptyTurnWarning is only tested as a pure function (runtime_test.go:5024-5059); nothing asserts a media-only turn stays silent. Fix: assert on the event stream inside the existing TestRunStream_MediaOnlyAssistantHistoryRemainsCoherent — no *WarningEvent matching the empty-turn text.

[blocking] The DB-backed blob work this lane is supposed to carry is absent, so the per-item size cap cannot be verified at all.
git grep -iE "MaxGeneratedBlobBytes|generated_media_blobs|GeneratedMediaBlobs|AddGeneratedBlob" 41c56eba -- pkg returns no matches. The catalogue ends at 028_add_generated_media_manifest_table (pkg/session/migrations.go:450) — there is no blob migration, no exported cap constant, and no byte check anywhere before workspacemediaWrite(root, requested, m.Data, m.MimeType) (pkg/runtime/loop.go:1394) writes whatever the provider produced. Consequently there is no overflow/degrade path and no blob-side prune either. Either this lane still needs that content, or the cap needs a different owner — but as it stands "20MiB enforced before the write" is not implemented anywhere in the stack up to this head.

[should-fix] Disk-full and quota failures fall through to non-actionable "retry" advice.
pkg/runtime/media_save_failure.go:30 handles fs.ErrPermission/EROFS and :32 handles fs.ErrNotExist, but nothing handles ENOSPC:

errno=no space left on device  reason="Enable --debug and retry to capture technical details."
errno=disc quota exceeded     reason="Enable --debug and retry to capture technical details."
errno=permission denied       reason="The save location is not writable."

Generated images are exactly the payload that fills a disk, and retrying will not help. Suggest a case errors.Is(err, syscall.ENOSPC) (plus EDQUOT if it stays portable) returning something like "There is not enough free space to save it." The current output is safe, just unhelpful — which is the one thing this PR exists to fix.

[should-fix] A media-only turn whose every item fails to save persists a completely empty assistant message.
Probe calling recordAssistantMessage with one media item on a session with no workspace root:

persisted messages: 1
  [0] role="assistant" content="" multicontent=0 toolcalls=0
warning: "Failed to save generated media item 1/1 (image/png, cat.png). No session workspace is available to save into."

The len(res.Media) == 0 guard at :1177 lets the message through, materializeGeneratedMedia then returns zero parts, and the Content mirror at :1252-1263 is skipped because Content is empty. Not a provider break — anthropic (client.go:435-437), oaistream (messages.go:88) and gemini (client.go:248) all drop an empty assistant message — but the session and TUI gain a blank assistant turn plus a message event. Consider skipping the message (or persisting the failure text) when mediaParts, Content and ToolCalls are all empty.

[should-fix] Plan/session-local references shipped in user-facing docs and durable comments.

  • docs/features/tui/index.md:299: "A future step will add TUI rendering … today this slice covers the domain, persistence, and safety mechanics only." "this slice" / "a future step" are stack-round language that means nothing to a reader of merged docs, and it promises a feature with no issue reference. Suggest phrasing it as a current limitation, citing #3996.
  • pkg/runtime/strip_generated_media.go:20: "(plan step 4, "Context bloat decision")" — the surrounding sentence already explains the rationale, so the label is redundant; drop it or cite #3996.
  • pkg/runtime/materialize_generated_media_test.go:87: "the core Phase-2.3 contract".

[optional] DeleteSession prunes the manifest before it knows the session existed, and not atomically.
pkg/session/store.go:1005-1019: the manifest DELETE runs before the rowsAffected == 0 → ErrNotFound check, so rows are removed for a session that was never there, and the two statements are not in one transaction — a failure between them leaves orphans behind an already-deleted session. Harmless for metadata rows; it becomes a space question once blobs share this key.

[optional] Sub-session manifest rows survive their parent's deletion.
Sub-agent sessions are separate sessions rows inheriting parent.WorkingDir (pkg/runtime/agent_delegation.go:254), and materialization keys the manifest by the sub-session ID, but DeleteSession only deletes WHERE id = ? (store.go:1005) with no cascade. Pre-existing (child session rows already leak the same way) and consistent with accepting orphans, but it is a third orphan source beyond compaction and branch truncation — worth naming wherever that policy gets documented.

Sub-agent classification checks out otherwise: a delegated session with no WorkingDir resolves through the bounded, cycle-safe parent walk (pkg/session/working_dir.go:57-92), and a genuine miss classifies as "No session workspace is available to save into." — no crash, no leaked path (TestMaterializeGeneratedMedia_NoWorkspaceRoot, …_InheritsWorkspaceFromParent).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation changes area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/gemini Google Gemini provider support area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/sessions For features/issues/fixes related to session lifecycle (resume, persistence, export) kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant