Skip to content

GetResponseAsync_StructuredOutput_NonNative fails with JsonException : '`' is an invalid start of a value. #7144

@ericstj

Description

@ericstj

public virtual async Task GetResponseAsync_StructuredOutput_NonNative()
{
SkipIfNotEnabled();
var capturedOptions = new List<ChatOptions?>();
var captureOutputChatClient = ChatClient.AsBuilder()
.Use((messages, options, nextAsync, cancellationToken) =>
{
capturedOptions.Add(options);
return nextAsync(messages, options, cancellationToken);
})
.Build();
var response = await captureOutputChatClient.GetResponseAsync<Person>("""
Supply an object to represent Jimbo Smith from Cardiff.
""", useJsonSchemaResponseFormat: false);
Assert.Equal("Jimbo Smith", response.Result.FullName);
Assert.Contains("Cardiff", response.Result.HomeTown);
// Verify it used *non-native* structured output, i.e., response format Json with no schema
var responseFormat = Assert.IsType<ChatResponseFormatJson>(Assert.Single(capturedOptions)!.ResponseFormat);
Assert.Null(responseFormat.Schema);
Assert.Null(responseFormat.SchemaName);
Assert.Null(responseFormat.SchemaDescription);
}

Microsoft.Extensions.AI.OpenAIAssistantChatClientIntegrationTests.GetResponseAsync_StructuredOutput_NonNative will occasionally fail with the following exception:

Microsoft.Extensions.AI.OpenAIAssistantChatClientIntegrationTests.GetResponseAsync_StructuredOutput_NonNative [FAIL]
  System.Text.Json.JsonException : '`' is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
  ---- System.Text.Json.JsonReaderException : '`' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
  Stack Trace:
       at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
       at System.Text.Json.JsonSerializer.GetReaderScopedToNextValue(Utf8JsonReader& reader, ReadStack& state)
       at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
       at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
    C:\src\dotnet\extensions\src\Libraries\Microsoft.Extensions.AI\ChatCompletion\ChatResponse{T}.cs(107,0): at Microsoft.Extensions.AI.ChatResponse`1.DeserializeFirstTopLevelObject(String json, JsonTypeInfo`1 typeInfo)
    C:\src\dotnet\extensions\src\Libraries\Microsoft.Extensions.AI\ChatCompletion\ChatResponse{T}.cs(124,0): at Microsoft.Extensions.AI.ChatResponse`1.GetResultCore(Nullable`1& failureReason)
    C:\src\dotnet\extensions\src\Libraries\Microsoft.Extensions.AI\ChatCompletion\ChatResponse{T}.cs(60,0): at Microsoft.Extensions.AI.ChatResponse`1.get_Result()
    C:\src\dotnet\extensions\test\Libraries\Microsoft.Extensions.AI.Integration.Tests\ChatClientIntegrationTests.cs(1109,0): at Microsoft.Extensions.AI.ChatClientIntegrationTests.GetResponseAsync_StructuredOutput_NonNative()
    --- End of stack trace from previous location ---
    ----- Inner Stack Trace -----
       at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
       at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
       at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
       at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
       at System.Text.Json.Utf8JsonReader.Read()
       at System.Text.Json.JsonSerializer.GetReaderScopedToNextValue(Utf8JsonReader& reader, ReadStack& state)

I can make it fail about 2 out of 5 times. When it fails I debugged to see what the model returned. It looks like it returned a markdown block instead of raw JSON. The response was the following.

```json
{
  "fullName": "Jimbo Smith",
  "ageInYears": 34,
  "homeTown": "Cardiff",
  "job": "Programmer"
}
```

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-aiMicrosoft.Extensions.AI librariesuntriaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions