Skip to content

agy-cli: silent long generation hits TURN_TIMEOUT, then magpie's prefill continuation is rejected ("live tool_result + assistant prefill tail"), turn dies and session is lost #1319

Open

Description

@wangbingnan136

Summary

In an agentic Codex session (Codex Desktop → magpie gateway → agy-cli → gemini-4-carbon), a turn died with:

agy-cli: agy bridge: a request combining a live tool_result with an assistant prefill tail is not supported

Three problems chain together:

  1. The turn was apparently killed by TURN_TIMEOUT_MS (5 min of agy silence). The model had already streamed a short commentary sentence and then went silent (most likely while generating a large tool call). About 300 s later the request failed. This step is inferred: no separate timeout line was logged.
  2. magpie then tried to continue the cut-off reply by sending the partial text back as an assistant prefill. In an agent loop, the trailing messages are … user(tool_result) → assistant(partial text). validateBody rejects exactly that combination (index.mjs ~L5549). magpie only falls back from prefill when the error matches its own "prefill refused" pattern, roughly does not support (assistant message )?prefill|requests ending with a model turn are not supported. The bridge's wording (…prefill tail is not supported) does not match, so magpie never retries without prefill. The client gets a non-retryable 400, and the turn ends.
  3. The agy session is dropped on timeout (timeoutTurn → dropFromPool + stop()). On the user's next "continue", a fresh agy session replays roughly 156k tokens of history through the 112 KiB budget ([... earlier conversation history omitted to fit ACP 112 KiB prompt budget ...]). The model loses the earlier agreed spec and starts re-reading its own thread through tools. It also tries to parse agy's spilled .system_generated/steps/N/output.txt file, which is not plain JSON because it starts with a Created At / Wall time / Output: header.

Environment

  • agy-cli plugin 0.1.95 (same code on main today)
  • agy 1.5.7, magpie 0.1.3960 (as printed by magpie --version), macOS 27.0.1
  • Client: Codex Desktop (Responses API) via magpie; model agy-cli/gemini-4-carbon, effort high

Timeline (local time, from magpie app.log and the Codex rollout)

time event
17:06:44 tool result returned; Codex sends the next request (tail = tool_result)
17:08:41 reasoning done; assistant text starts streaming ("…now writing manifest + build code…")
17:08:41 → 17:13:43 no further output for ~302 s; agy's transcript shows that step with output_tokens: 7332 but no tool_calls recorded
17:13:43 gateway: codex's call to agy-cli/gemini-4-carbon on agy-cli failed: 200 after 6m58.612s: agy-cli: agy bridge: a request combining a live tool_result with an assistant prefill tail is not supported
17:13:44 client retry: failed: 400 after 667ms: same message → turn ends
17:17:54 user says "continue" → new agy conversation; first prompt contains the 112 KiB "history omitted" notes

Expected

  • A long silent generation (big tool call / file write) should not be killed at 5 min. Or, if it is, the error should be one the client or magpie can retry cleanly, and the conversation should not lose its agy session.
  • A continuation after a cut-off reply should not dead-end when the tail contains a tool_result.

Suggestions

  1. In validateBody, either support "tool_result + prefill tail": render the tool results, then the prefill carrier note, in the same turn. Or reject it with wording or a code that magpie's prefill-refusal detection recognizes (e.g. does not support assistant message prefill …). Then magpie falls back to a non-prefill retry instead of surfacing a 400.
  2. Consider a larger default AGY_BRIDGE_TURN_TIMEOUT_MS for turns that have already streamed text. Or reset the timer on agy-side progress signals during long tool-call generation, if ACP exposes any.
  3. On timeoutTurn, keep the session resumable (session/resume) rather than forcing a 112 KiB history replay on the next turn.

Workaround on my side: launchctl setenv AGY_BRIDGE_TURN_TIMEOUT_MS 900000, then restart magpie.

Happy to provide more log excerpts if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions