Skip to content

feat(gateway): inject by default in combined deployments for host conversations - #2417

Merged
frf12 merged 1 commit into
oceanbase:masterfrom
jasper-yl:feat/gateway-injection-default
Nov 2, 2026
Merged

frf12 merged 1 commit into
oceanbase:masterfrom
jasper-yl:feat/gateway-injection-default

Conversation

@jasper-yl

@jasper-yl jasper-yl commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Refs #2415. This PR contains only the deployment default policy; the lifecycle it relies on (#2406) merged in #2416. Following the maintainer's note, #2415 stays open until the supported client identities are settled.

Rationale for this change

A deployment that routes an agent through the combined gateway gets no prepared context unless the operator also sets PC_GATEWAY_INJECTION=on. #2416 gives a host that sends X-PowerContext-Conversation-ID one preparation per user message and an exact replay across its tool loop. That lifecycle can be the default. Injecting requests without a conversation ID by default would bring back a preparation at every tool step, which the review asked not to make the default experience.

What changes are included in this PR?

  • injection_mode in pc-gateway-app/src/main.rs and injects in ingress.rs:

    • Unset, in a combined deployment (PC_GATEWAY_CORE_UDS): inject only requests that carry a conversation ID (Injection::ConversationsOnly).
    • Unset, in a standalone deployment (PC_GATEWAY_CORE_BASE): no injection, as before.
    • PC_GATEWAY_INJECTION=on: inject every request, as before. Requests without a conversation ID prepare individually.
    • PC_GATEWAY_INJECTION=off: no injection, in any deployment.
    • Any other value: no injection, with a warning, so a typo never turns it on.

    PC_GATEWAY_INJECTION_RESPONSES=off still keeps only the Responses route uninjected.

  • Docs: gateway/README.md, and a new "Gateway context injection" / "网关上下文注入" subsection in docs/{en,zh}/docs/operate/deploy-server.md. They cover the default, the conversation header, on / off, the Responses opt-out, standalone opt-in, and that an empty or failed preparation sends the request without injection.

Are there any user-facing changes?

Yes. In combined deployments, requests that carry X-PowerContext-Conversation-ID are now injected unless PC_GATEWAY_INJECTION=off is set. Requests without the header, standalone deployments, and deployments that set the variable explicitly behave as before.

How was this change tested?

  • cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings and cargo test in gateway/: 106 tests, all pass.
  • New tests:
    • combined_deployments_inject_conversations_unless_set, standalone_deployments_inject_only_when_turned_on and unknown_values_never_enable_injection: the setting (unset, on, off, unknown) by deployment.
    • the_default_injects_only_requests_with_a_conversation_id: requests with and without the header on both routes.
  • Chat and Responses tool loops, isolation, and empty or failed preparation without blocking the request are covered by the executor tests merged in fix(gateway): prepare once per user message within a conversation and replay it #2416.
  • prek hooks and tests/test_full_capability_docs.py pass for the changed docs.

anged docs.

AI usage statement

Built with Claude Code (Claude Opus 5.5), which wrote the change, the tests and the docs, and ran the checks above.

🤖 Generated with Claude Code

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default-selection logic matches the requested combined/standalone behavior, but the turn cache included from #2416 still breaks session separation and stable per-turn reuse. These prerequisite issues should be fixed before enabling injection by default.

Comment on lines +57 to +60
let prefix = serde_json::to_string(&request.messages[..=last_user]).ok()?;
let mut hasher = DefaultHasher::new();
scope_id.hash(&mut hasher);
prefix.hash(&mut hasher);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Partition cached turns by the Responses session identity

Different Responses sessions collide whenever their Scope and message prefix match, because the key omits prompt_cache_key. With the compiled gateway and a real SQLite-backed Core, I sent session-a with no matching memory, remembered the matching fact through the public API, then sent the same input with prompt_cache_key: "session-b". Session B still sent no injected memory, although a direct prepare and a changed-query control both returned it. The new session inherits the cached empty result for up to five minutes. This violates the session separation requested in #2406 and suppresses newly available memory in an independent conversation. Include the supported session identity in the cache partition and cover identical opening prompts in different sessions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the repro. This review is on the earlier head. After frf12's correction on #2406, #2416 no longer keys anything on prompt_cache_key or on the message prefix as identity. Replay state is partitioned by Scope plus the host-supplied X-PowerContext-Conversation-ID (RFC 2199), and requests without that header are never cached, so they prepare on their own. In your scenario, session B with the same input and no header (or a different conversation ID) prepares again and gets the new memory. Covered by conversations_with_identical_messages_prepare_separately and requests_without_a_conversation_id_never_inherit_an_injection. #2417 now also only injects requests with a conversation ID by default.

Comment on lines +389 to +393
let cached = turn.and_then(|key| self.turns.get(key, now_ms()));
let prepared = match cached {
Some(prepared) => prepared,
None => {
let Some(prepared) = self.prepare_turn(call, evidence).await else {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Commit one prepared representation for concurrent requests of a turn

Two overlapping requests for the same uncached turn can both miss here before either awaited prepare finishes. A public Executor probe with a barrier in the scripted Core reproduced both requests being released with different injected blocks; a third request then replayed the replacement block and binding. The last preparation overwrites the cached result. Thus an overlapping retry can change the context and provider prefix within a turn, despite the prerequisite's reuse guarantee. Coordinate preparation or commit per turn so every released request uses the same selected block and binding, even if preparation is computed more than once.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The current #2416 commits one value per conversation and anchor with put-if-absent: when overlapping requests both prepare, the first stored result wins, and the later request injects that stored block and binding instead of its own. 23395cb adds concurrent_requests_of_a_turn_commit_one_prepared_value: two requests meet at a barrier inside prepare, so both prepare with different results; both released requests and a following one inject the same value. It fails when the commit is changed back to overwrite. #2417 is rebased on it.

@jasper-yl
jasper-yl force-pushed the feat/gateway-injection-default branch from 1305507 to 6e56a38 Compare November 2, 2026 16:24
@jasper-yl jasper-yl changed the title feat(gateway): enable injection by default in combined deployments feat(gateway): inject by default in combined deployments for host conversations Nov 2, 2026
@jasper-yl
jasper-yl force-pushed the feat/gateway-injection-default branch 2 times, most recently from 5bbc499 to 7d51119 Compare November 2, 2026 16:33
hidb4ai
hidb4ai previously requested changes Nov 2, 2026

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked 7d51119: both findings from my previous review are fixed. The remaining blocker is the Codex identity gap already raised on #2416.

With this gateway binary and a real Core over UDS backed by SQLite, three Responses tool-loop requests carrying thread-id / session-id but no X-PowerContext-Conversation-ID made three prepare calls with PC_GATEWAY_INJECTION=on; the combined unset default injected nothing on all three requests. These were synthetic stock-shaped requests, not a native Codex run. Thus the reported client still lacks the prerequisite's prepare/replay lifecycle. Please resolve that host identity path and verify same-thread replay plus parent/subagent isolation before merging this default change.

frf12
frf12 previously requested changes Nov 2, 2026

@frf12 frf12 left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Withdrawn pending discussion with the maintainer. Please disregard the requirements in this review.

…versations

Routing an agent through a combined gateway gave no prepared context unless
the operator also set PC_GATEWAY_INJECTION=on. With per-conversation replay
(oceanbase#2406), a host that sends X-PowerContext-Conversation-ID gets one prepare
per user message and the exact same injection across its tool loop.

Make that the default when the gateway reaches a configured Core over UDS:
unset PC_GATEWAY_INJECTION injects only requests with a conversation ID, so
the default never re-prepares at every tool step. PC_GATEWAY_INJECTION=on
injects every request, off disables injection, standalone gateways stay
opt-in, and an unrecognized value disables injection with a warning.

Document the default and the opt-outs in the gateway README and in both
locales of the server deployment guide.

Refs oceanbase#2415

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jasper-yl
jasper-yl force-pushed the feat/gateway-injection-default branch from 7d51119 to 7cd52ad Compare November 2, 2026 18:32
@frf12
frf12 dismissed their stale review November 2, 2026 18:42

Withdrawn at the maintainer's request; the requirements had not been discussed.

@jasper-yl

Copy link
Copy Markdown
Contributor Author

@hidb4ai the Codex identity path is now #2419 (refs #2418), kept separate from this PR so this one stays the default policy only; it is rebased onto master.

#2419 takes the conversation from Codex's thread-id on the Responses route when X-PowerContext-Conversation-ID is absent. It was verified with a native Codex 0.145 CLI run (not synthetic requests) against an enforcing master Server. One session in which the parent spawned a subagent made 10 requests and 2 prepare_context calls, one per thread. The parent's tool continuations replayed, and the subagent prepared on its own although it shared the parent's session-id and prompt_cache_key. With thread-id removed, the same session made 10 calls. With #2419 merged, the combined unset default here also covers an unmodified Codex.

@frf12
frf12 dismissed hidb4ai’s stale review November 2, 2026 19:51

Dismissing this stale blocker for the sequencing agreed with the maintainer: this PR only enables the default for requests that already carry X-PowerContext-Conversation-ID; native Codex thread identity is being added in follow-up PR #2419, which must rebase onto this merge before it is merged.

@frf12
frf12 merged commit c56736a into oceanbase:master Nov 2, 2026
38 checks passed
@hidb4ai hidb4ai added the 1.5.0 v1.5.0 release tracking label Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.5.0 v1.5.0 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants