Repository navigation
feat(gateway): inject by default in combined deployments for host conversations - #2417
Conversation
hidb4ai
left a comment
There was a problem hiding this comment.
The default-selection logic matches the requested combined/standalone behavior, but the turn cache included from #2416 still breaks session separation and stable per-turn reuse. These prerequisite issues should be fixed before enabling injection by default.
| let prefix = serde_json::to_string(&request.messages[..=last_user]).ok()?; | ||
| let mut hasher = DefaultHasher::new(); | ||
| scope_id.hash(&mut hasher); | ||
| prefix.hash(&mut hasher); |
There was a problem hiding this comment.
[P2] Partition cached turns by the Responses session identity
Different Responses sessions collide whenever their Scope and message prefix match, because the key omits prompt_cache_key. With the compiled gateway and a real SQLite-backed Core, I sent session-a with no matching memory, remembered the matching fact through the public API, then sent the same input with prompt_cache_key: "session-b". Session B still sent no injected memory, although a direct prepare and a changed-query control both returned it. The new session inherits the cached empty result for up to five minutes. This violates the session separation requested in #2406 and suppresses newly available memory in an independent conversation. Include the supported session identity in the cache partition and cover identical opening prompts in different sessions.
There was a problem hiding this comment.
Thanks for the repro. This review is on the earlier head. After frf12's correction on #2406, #2416 no longer keys anything on prompt_cache_key or on the message prefix as identity. Replay state is partitioned by Scope plus the host-supplied X-PowerContext-Conversation-ID (RFC 2199), and requests without that header are never cached, so they prepare on their own. In your scenario, session B with the same input and no header (or a different conversation ID) prepares again and gets the new memory. Covered by conversations_with_identical_messages_prepare_separately and requests_without_a_conversation_id_never_inherit_an_injection. #2417 now also only injects requests with a conversation ID by default.
| let cached = turn.and_then(|key| self.turns.get(key, now_ms())); | ||
| let prepared = match cached { | ||
| Some(prepared) => prepared, | ||
| None => { | ||
| let Some(prepared) = self.prepare_turn(call, evidence).await else { |
There was a problem hiding this comment.
[P2] Commit one prepared representation for concurrent requests of a turn
Two overlapping requests for the same uncached turn can both miss here before either awaited prepare finishes. A public Executor probe with a barrier in the scripted Core reproduced both requests being released with different injected blocks; a third request then replayed the replacement block and binding. The last preparation overwrites the cached result. Thus an overlapping retry can change the context and provider prefix within a turn, despite the prerequisite's reuse guarantee. Coordinate preparation or commit per turn so every released request uses the same selected block and binding, even if preparation is computed more than once.
There was a problem hiding this comment.
Agreed. The current #2416 commits one value per conversation and anchor with put-if-absent: when overlapping requests both prepare, the first stored result wins, and the later request injects that stored block and binding instead of its own. 23395cb adds concurrent_requests_of_a_turn_commit_one_prepared_value: two requests meet at a barrier inside prepare, so both prepare with different results; both released requests and a following one inject the same value. It fails when the commit is changed back to overwrite. #2417 is rebased on it.
1305507 to
6e56a38
Compare
5bbc499 to
7d51119
Compare
hidb4ai
left a comment
There was a problem hiding this comment.
Rechecked 7d51119: both findings from my previous review are fixed. The remaining blocker is the Codex identity gap already raised on #2416.
With this gateway binary and a real Core over UDS backed by SQLite, three Responses tool-loop requests carrying thread-id / session-id but no X-PowerContext-Conversation-ID made three prepare calls with PC_GATEWAY_INJECTION=on; the combined unset default injected nothing on all three requests. These were synthetic stock-shaped requests, not a native Codex run. Thus the reported client still lacks the prerequisite's prepare/replay lifecycle. Please resolve that host identity path and verify same-thread replay plus parent/subagent isolation before merging this default change.
…versations Routing an agent through a combined gateway gave no prepared context unless the operator also set PC_GATEWAY_INJECTION=on. With per-conversation replay (oceanbase#2406), a host that sends X-PowerContext-Conversation-ID gets one prepare per user message and the exact same injection across its tool loop. Make that the default when the gateway reaches a configured Core over UDS: unset PC_GATEWAY_INJECTION injects only requests with a conversation ID, so the default never re-prepares at every tool step. PC_GATEWAY_INJECTION=on injects every request, off disables injection, standalone gateways stay opt-in, and an unrecognized value disables injection with a warning. Document the default and the opt-outs in the gateway README and in both locales of the server deployment guide. Refs oceanbase#2415 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7d51119 to
7cd52ad
Compare
Withdrawn at the maintainer's request; the requirements had not been discussed.
|
@hidb4ai the Codex identity path is now #2419 (refs #2418), kept separate from this PR so this one stays the default policy only; it is rebased onto master. #2419 takes the conversation from Codex's |
Dismissing this stale blocker for the sequencing agreed with the maintainer: this PR only enables the default for requests that already carry X-PowerContext-Conversation-ID; native Codex thread identity is being added in follow-up PR #2419, which must rebase onto this merge before it is merged.
Which issue or RFC does this PR close?
Refs #2415. This PR contains only the deployment default policy; the lifecycle it relies on (#2406) merged in #2416. Following the maintainer's note, #2415 stays open until the supported client identities are settled.
Rationale for this change
A deployment that routes an agent through the combined gateway gets no prepared context unless the operator also sets
PC_GATEWAY_INJECTION=on. #2416 gives a host that sendsX-PowerContext-Conversation-IDone preparation per user message and an exact replay across its tool loop. That lifecycle can be the default. Injecting requests without a conversation ID by default would bring back a preparation at every tool step, which the review asked not to make the default experience.What changes are included in this PR?
injection_modeinpc-gateway-app/src/main.rsandinjectsiningress.rs:PC_GATEWAY_CORE_UDS): inject only requests that carry a conversation ID (Injection::ConversationsOnly).PC_GATEWAY_CORE_BASE): no injection, as before.PC_GATEWAY_INJECTION=on: inject every request, as before. Requests without a conversation ID prepare individually.PC_GATEWAY_INJECTION=off: no injection, in any deployment.PC_GATEWAY_INJECTION_RESPONSES=offstill keeps only the Responses route uninjected.Docs:
gateway/README.md, and a new "Gateway context injection" / "网关上下文注入" subsection indocs/{en,zh}/docs/operate/deploy-server.md. They cover the default, the conversation header,on/off, the Responses opt-out, standalone opt-in, and that an empty or failed preparation sends the request without injection.Are there any user-facing changes?
Yes. In combined deployments, requests that carry
X-PowerContext-Conversation-IDare now injected unlessPC_GATEWAY_INJECTION=offis set. Requests without the header, standalone deployments, and deployments that set the variable explicitly behave as before.How was this change tested?
cargo fmt --all -- --check,cargo clippy --all-targets -- -D warningsandcargo testingateway/: 106 tests, all pass.combined_deployments_inject_conversations_unless_set,standalone_deployments_inject_only_when_turned_onandunknown_values_never_enable_injection: the setting (unset, on, off, unknown) by deployment.the_default_injects_only_requests_with_a_conversation_id: requests with and without the header on both routes.prekhooks andtests/test_full_capability_docs.pypass for the changed docs.anged docs.
AI usage statement
Built with Claude Code (Claude Opus 5.5), which wrote the change, the tests and the docs, and ran the checks above.
🤖 Generated with Claude Code