Skip to content

Fixing JSON schema export of Settings#1309

Merged
jamesbraza merged 1 commit intomainfrom
fixing-json-schema
Mar 3, 2026
Merged

Fixing JSON schema export of Settings#1309
jamesbraza merged 1 commit intomainfrom
fixing-json-schema

Conversation

@jamesbraza
Copy link
Copy Markdown
Collaborator

Seen in dev logs when generating an OpenAPI spec:

Traceback (most recent call last):
  File "/srv/.venv/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py", line 416, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/srv/.venv/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/fastapi/applications.py", line 1160, in __call__
    await super().__call__(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/applications.py", line 107, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/base.py", line 191, in __call__
    with recv_stream, send_stream, collapse_excgroups():
  File "/usr/local/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
    raise exc
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/base.py", line 193, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/srv/app/main.py", line 234, in add_request_id
    response = await call_next(request)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/base.py", line 168, in call_next
    raise app_exc from app_exc.__cause__ or app_exc.__context__
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/base.py", line 144, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/cors.py", line 87, in __call__
    await self.app(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/srv/.venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/srv/.venv/lib/python3.13/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/routing.py", line 78, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/srv/.venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/srv/.venv/lib/python3.13/site-packages/starlette/routing.py", line 75, in app
    response = await f(request)
  File "/srv/.venv/lib/python3.13/site-packages/fastapi/applications.py", line 1107, in openapi
    schema = self.openapi()
  File "/srv/.venv/lib/python3.13/site-packages/fastapi/applications.py", line 1084, in openapi
    self.openapi_schema = get_openapi(
  File "/srv/.venv/lib/python3.13/site-packages/fastapi/openapi/utils.py", line 534, in get_openapi
    field_mapping, definitions = get_definitions(
  File "/srv/.venv/lib/python3.13/site-packages/fastapi/_compat/v2.py", line 323, in get_definitions
    field_mapping, definitions = schema_generator.generate_definitions(inputs=inputs)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 379, in generate_definitions
    self.generate_inner(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 556, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/main.py", line 852, in __get_pydantic_json_schema__
    return handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1134, in function_after_schema
    return self.generate_inner(schema['schema'])
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1134, in function_after_schema
    return self.generate_inner(schema['schema'])
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1604, in model_schema
    json_schema = self.generate_inner(schema['schema'])
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1717, in model_fields_schema
    json_schema = self._named_required_fields_schema(named_required_fields)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1508, in _named_required_fields_schema
    field_json_schema = self.generate_inner(field).copy()
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 528, in js_updates_handler_func
    json_schema = {**current_handler(schema_or_field), **js_updates}
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1576, in model_field_schema
    return self.generate_inner(schema['schema'])
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1175, in default_schema
    json_schema = self.generate_inner(schema['schema'])
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 1245, in nullable_schema
    inner_json_schema = self.generate_inner(schema['schema'])
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 578, in generate_inner
    json_schema = current_handler(schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/_internal/_schema_generation_shared.py", line 37, in __call__
    return self.handler(core_schema)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 511, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 921, in is_instance_schema
    return self.handle_invalid_for_json_schema(schema, f'core_schema.IsInstanceSchema ({schema["cls"]})')
  File "/srv/.venv/lib/python3.13/site-packages/pydantic/json_schema.py", line 2442, in handle_invalid_for_json_schema
    raise PydanticInvalidForJsonSchema(f'Cannot generate a JsonSchema for {error_info}')
pydantic.errors.PydanticInvalidForJsonSchema: Cannot generate a JsonSchema for core_schema.IsInstanceSchema (<class 'paperqa.settings.AsyncContextSerializer'>)

This PR restores the ability to generate a JSON schema for paperqa.Settings

@jamesbraza jamesbraza self-assigned this Mar 3, 2026
Copilot AI review requested due to automatic review settings March 3, 2026 00:20
@jamesbraza jamesbraza added the bug Something isn't working label Mar 3, 2026
@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Mar 3, 2026
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 3, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR restores JSON schema generation for paperqa.Settings (used by downstream OpenAPI generation) by preventing Pydantic from attempting to generate schemas for callable/arbitrary-type fields.

Changes:

  • Wrap several callable/arbitrary-type settings fields in SkipJsonSchema[...] to avoid PydanticInvalidForJsonSchema.
  • Add a regression test that exercises Settings.model_json_schema() during default settings instantiation.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/paperqa/settings.py Marks several fields as SkipJsonSchema[...] to prevent JSON schema generation failures for callables/protocols.
tests/test_configs.py Adds a regression test that calls Settings.model_json_schema() to ensure schema export doesn’t raise.
Comments suppressed due to low confidence (1)

src/paperqa/settings.py:293

  • SkipJsonSchema will omit parse_pdf from model_json_schema(), but ParsingSettings’ custom @model_serializer can still include parse_pdf in JSON (mode="json") as a stable string/FQN. That creates a mismatch between the generated schema/OpenAPI and the actual serialized output. Consider representing parse_pdf as a JSON-schema-friendly type (e.g., string/import-path with custom json schema) instead of skipping it entirely, or ensure it’s never serialized if it shouldn’t be part of the public JSON shape.
    parse_pdf: SkipJsonSchema[PDFParserFn] = Field(
        default_factory=get_default_pdf_parser,
        description="Function to parse PDF, or a fully qualified name to import.",
        examples=["paperqa_docling.parse_pdf_to_pages"],
        exclude=True,  # NOTE: a custom serializer is used below, so it's not excluded
    )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jamesbraza jamesbraza merged commit d9e817a into main Mar 3, 2026
9 of 11 checks passed
@jamesbraza jamesbraza deleted the fixing-json-schema branch March 3, 2026 00:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants