🅰️ feat: Native Anthropic Provider for Custom Endpoints#13748
Conversation
Let a custom endpoint declare `provider: anthropic` to use the native Anthropic `/v1/messages` client (the agents SDK's ChatAnthropic) against its own `baseURL`/`apiKey`/`headers`, instead of being forced through the OpenAI-compatible client. Enables Anthropic itself and Anthropic-compatible gateways (AI gateways, OpenCode Zen, etc.) as custom endpoints — including for agents and role-scoped model access. Closes #10655 (Option 1: explicit provider). - Schema: add optional `provider` (currently `anthropic`) to the custom `endpointSchema` in data-provider. - Routing: `getProviderConfig` maps a custom endpoint with `provider: anthropic` to `Providers.ANTHROPIC` (was always `Providers.OPENAI`). - Config: `initializeCustom` builds the native Anthropic config via the Anthropic `getLLMConfig` (custom baseURL/apiKey/headers) and returns `provider: anthropic`; `useLegacyContent` is left unset to match the built-in Anthropic endpoint. The OpenAI-compatible path is unchanged for endpoints without `provider`. - Summarization: `resolveSummarizationProvider` builds an Anthropic config for a cross-endpoint native-Anthropic summarization target (self-summarize already reuses the agent's client options). Title generation already resolves via `agent.endpoint`, and provider-specific handling (tool conflicts, content/PDF validation, token counting, streamUsage) already branches on `Providers.ANTHROPIC`, so it applies automatically. Note: model auto-fetch (`models.fetch`) uses the OpenAI `/models` convention and is not used for this provider — list models explicitly under `models.default`.
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 61e5608867
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| * (Anthropic itself or Anthropic-compatible gateways). Omit for | ||
| * OpenAI-compatible endpoints. | ||
| */ | ||
| provider: z.literal(EModelEndpoint.anthropic).optional(), |
There was a problem hiding this comment.
When a custom endpoint sets provider: anthropic, the backend uses the native client, but the endpoint metadata sent to the UI still comes from loadCustomEndpointsConfig as type: custom with no provider/default-param override. The Agents model panel then falls back to the custom/OpenAI parameter set, so users see and persist fields like max_tokens instead of Anthropic fields like maxOutputTokens/thinking; the native Anthropic initializer only reads maxOutputTokens, so max-output settings from the UI are silently ignored for these endpoints unless admins also duplicate customParams.defaultParamsEndpoint: anthropic. Please expose this provider (or synthesize the Anthropic default params) in the endpoint config used by the client.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in c6130d2. loadCustomEndpointsConfig now synthesizes the client customParams.defaultParamsEndpoint from provider (e.g. anthropic) when it isn't explicitly set, so the Agents model panel and preset/convo parsing use the Anthropic parameter set (maxOutputTokens/thinking) instead of OpenAI's max_tokens. An explicit non-default defaultParamsEndpoint still wins. Test added in custom/config.spec.ts.
| overrideProvider = | ||
| customEndpointConfig.provider === EModelEndpoint.anthropic | ||
| ? Providers.ANTHROPIC | ||
| : Providers.OPENAI; |
There was a problem hiding this comment.
When the custom endpoint name is one of the known custom-provider names (for example OpenRouter/openrouter) and it also sets provider: anthropic, the earlier providerConfigMap[provider.toLowerCase()] branch loads initializeCustom and this new override assignment is skipped because customEndpointConfig is only fetched afterward. initializeCustom eventually returns options.provider = anthropic, but values derived before that from overrideProvider (notably the getModelMaxTokens(... providerEndpointMap[overrideProvider] ...) call in initializeAgent) still use the OpenAI/default token map, so Claude models under these endpoint names get the wrong context budget unless admins add explicit token/max-context overrides.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in c6130d2. Moved the provider: anthropic override to the end of getProviderConfig, after all customEndpointConfig resolution, so it applies even when the endpoint name collides with a known custom-provider (e.g. openrouter) and takes the providerConfigMap branch. overrideProvider is now ANTHROPIC before getModelMaxTokens(providerEndpointMap[overrideProvider]) runs, so the context/token budget uses the Anthropic map. Test added in providers.spec.ts.
| headers: endpointConfig.headers, | ||
| addParams: endpointConfig.addParams, | ||
| dropParams: endpointConfig.dropParams, | ||
| }); |
There was a problem hiding this comment.
For provider: anthropic custom endpoints, this native path bypasses getOpenAIConfig, but it only forwards headers/add/drop params and drops customParams.paramDefinitions defaults. Admin-defined defaults that worked on the OpenAI-compatible custom path via extractDefaultParams—for example enabling promptCache, web_search, or default thinking settings—are no longer applied unless every agent explicitly persists those fields, and title/summarization calls that pass only a model will miss them as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in c6130d2. The native path now passes defaultParams: extractDefaultParams(customParams?.paramDefinitions) to the Anthropic getLLMConfig (same helper getOpenAIConfig uses), so admin defaults like web_search/promptCache/thinking apply without every agent persisting them. Applied in both initializeCustom (covers chat + title) and the cross-endpoint Anthropic branch of resolveSummarizationProvider. Test added in custom/initialize.spec.ts.
GitNexus: ❌ deploy failedThe deploy failed — the previous index (if any) continues to be served. |
Address Codex P2 findings — the native Anthropic path must match the OpenAI-compatible path's parameter handling: - UI param set: `loadCustomEndpointsConfig` now surfaces `provider` as the client `customParams.defaultParamsEndpoint`, so the Agents model panel shows Anthropic fields (`maxOutputTokens`/`thinking`) instead of OpenAI `max_tokens` (which the native initializer ignored). An explicit non-default `defaultParamsEndpoint` still wins. - Provider override: `getProviderConfig` re-applies `provider: anthropic` after all `customEndpointConfig` resolution, so it also wins when the endpoint name collides with a known custom provider (e.g. `openrouter`) — fixing the token/context budget derived from `overrideProvider`. - Default params: the native path (and cross-endpoint Anthropic summarization) now apply `customParams.paramDefinitions` defaults via `extractDefaultParams`, matching what `getOpenAIConfig` does for the OpenAI-compatible path. Adds tests for each.
|
@codex review |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…13748) *🅰️ feat: Native Anthropic provider for Custom Endpoints Let a custom endpoint declare `provider: anthropic` to use the native Anthropic `/v1/messages` client (the agents SDK's ChatAnthropic) against its own `baseURL`/`apiKey`/`headers`, instead of being forced through the OpenAI-compatible client. Enables Anthropic itself and Anthropic-compatible gateways (AI gateways, OpenCode Zen, etc.) as custom endpoints — including for agents and role-scoped model access. Closes danny-avila#10655 (Option 1: explicit provider). - Schema: add optional `provider` (currently `anthropic`) to the custom `endpointSchema` in data-provider. - Routing: `getProviderConfig` maps a custom endpoint with `provider: anthropic` to `Providers.ANTHROPIC` (was always `Providers.OPENAI`). - Config: `initializeCustom` builds the native Anthropic config via the Anthropic `getLLMConfig` (custom baseURL/apiKey/headers) and returns `provider: anthropic`; `useLegacyContent` is left unset to match the built-in Anthropic endpoint. The OpenAI-compatible path is unchanged for endpoints without `provider`. - Summarization: `resolveSummarizationProvider` builds an Anthropic config for a cross-endpoint native-Anthropic summarization target (self-summarize already reuses the agent's client options). Title generation already resolves via `agent.endpoint`, and provider-specific handling (tool conflicts, content/PDF validation, token counting, streamUsage) already branches on `Providers.ANTHROPIC`, so it applies automatically. Note: model auto-fetch (`models.fetch`) uses the OpenAI `/models` convention and is not used for this provider — list models explicitly under `models.default`. *🅰️ fix: Anthropic custom-endpoint param parity (Codex review) Address Codex P2 findings — the native Anthropic path must match the OpenAI-compatible path's parameter handling: - UI param set: `loadCustomEndpointsConfig` now surfaces `provider` as the client `customParams.defaultParamsEndpoint`, so the Agents model panel shows Anthropic fields (`maxOutputTokens`/`thinking`) instead of OpenAI `max_tokens` (which the native initializer ignored). An explicit non-default `defaultParamsEndpoint` still wins. - Provider override: `getProviderConfig` re-applies `provider: anthropic` after all `customEndpointConfig` resolution, so it also wins when the endpoint name collides with a known custom provider (e.g. `openrouter`) — fixing the token/context budget derived from `overrideProvider`. - Default params: the native path (and cross-endpoint Anthropic summarization) now apply `customParams.paramDefinitions` defaults via `extractDefaultParams`, matching what `getOpenAIConfig` does for the OpenAI-compatible path. Adds tests for each.
1 participant
Summary
I added a
provideroption to custom endpoints so a custom endpoint can declareprovider: anthropicand use the native Anthropic/v1/messagesclient (the agents SDK'sChatAnthropic) against its ownbaseURL/apiKey/headers, instead of being forced through the OpenAI-compatible client. Until now, custom endpoints were always routed toProviders.OPENAI, so pointing one at the Anthropic API (or an Anthropic-compatible gateway) failed with404 MODEL_NOT_FOUNDbecause the OpenAI client posts to/chat/completions.This resolves #10655 (Option 1 — explicit provider) and unlocks Anthropic-compatible inference APIs that are gaining traction (AI gateways, OpenCode Zen, etc.) as first-class custom endpoints — usable in agents and for role-scoped model access.
This is distinct from the existing
customParams.defaultParamsEndpoint: anthropic, which only borrows Anthropic param defaults while still sending OpenAI-shaped requests.provider: anthropicspeaks the actual Anthropic protocol.providerfield (currentlyanthropic) to the customendpointSchemaindata-provider.provider: anthropictoProviders.ANTHROPICingetProviderConfig(was alwaysProviders.OPENAI).initializeCustomvia the AnthropicgetLLMConfig(custombaseURL/apiKey/headers), returningprovider: anthropic; leftuseLegacyContentunset to match the built-in Anthropic endpoint. The OpenAI-compatible path is unchanged for endpoints withoutprovider.resolveSummarizationProvider(self-summarize already reuses the agent's client options).librechat.example.yaml.Title generation already resolves config via
agent.endpoint, and provider-specific behavior (tool conflicts, content/PDF validation, token counting,streamUsage) already branches onProviders.ANTHROPIC, so it applies automatically once the provider resolves.Change Type
Testing
Default behavior is unchanged for custom endpoints without
provider. To verify the feature, add a custom endpoint withprovider: anthropicpointing at the Anthropic API (or an Anthropic-compatible gateway), list it inagents.allowedProviders, and confirm chat + title generation succeed via the native/v1/messagesAPI (no404 MODEL_NOT_FOUND).Test Configuration:
Automated coverage added/updated and passing:
packages/api/src/endpoints/custom/initialize.spec.ts— native Anthropic config points at the custom baseURL/apiKey, attaches configured headers, returnsprovider: anthropic, and does not setuseLegacyContent; the OpenAI-compatible path is unchanged whenprovideris unset.packages/api/src/endpoints/config/providers.spec.ts—getProviderConfigroutesprovider: anthropictoProviders.ANTHROPICand defaults toProviders.OPENAIotherwise.tsc --noEmit(data-provider, data-schemas,@librechat/api,@librechat/client), ESLint, and the import-sort check pass for all changed files; thepackages/apiendpoints/agentssuites (1514 tests) and theapiagents-client suite are green.Known limitation
Model auto-fetch (
models.fetch: true) uses the OpenAI/models+ Bearer convention and is not used for this provider — list models explicitly undermodels.default(as in the example).Checklist