Object store item size fix #977

darkwatchuk · 2025-10-03T16:58:20Z

ObjectStore ObjectMetaData.Size is currently an int and causes a JSON deserialization crash when pointing to an object store with large files.

Changing to a ulong fixes this issue.

e.g. someone has uploaded large files to this object store on demo.nats.io

   [Fact]
   public async Task LargeDemoTest()
   {
       var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
       var cancellationToken = cts.Token;

       await using var nats = new NatsConnection(new NatsOpts { Url = "nats://demo.nats.io" });
       var js = new NatsJSContext(nats);
       var ob = new NatsObjContext(js);

       var objStore = await ob.GetObjectStoreAsync("omarchy-3_0_1", cancellationToken);

       await foreach (var thing in objStore.ListAsync(cancellationToken: cancellationToken))
       {
           Console.WriteLine($"{thing.Name} : {thing.Size}");
       }
   }

[no ci]

Copilot

Pull Request Overview

This PR fixes a JSON deserialization crash that occurs when ObjectStore points to objects with large files by changing the Size property from int to ulong in ObjectMetadata.

Changed ObjectMetadata.Size property from int to ulong to handle large file sizes
Updated all related code to use ulong for size calculations and comparisons
Modified test assertions to use decimal literals for consistency with the new data type

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/NATS.Client.ObjectStore/Models/ObjectMetadata.cs	Changed Size property from int to ulong
src/NATS.Client.ObjectStore/NatsObjStore.cs	Updated size variables and calculations to use ulong
tests/NATS.Client.ObjectStore.Tests/ObjectStoreTest.cs	Updated test assertions to use decimal literals and ulong casts
tests/NATS.Client.ObjectStore.Tests/WatcherTest.cs	Changed count variable from int to ulong

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

tests/NATS.Client.ObjectStore.Tests/WatcherTest.cs

tests/NATS.Client.ObjectStore.Tests/ObjectStoreTest.cs

mtmk · 2025-10-04T22:40:20Z

Aligning with ADR-20:

https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-20.md#objectinfo

type ObjectInfo struct {
    ObjectMeta
    Bucket  string    `json:"bucket"`
    NUID    string    `json:"nuid"`

    // the total object size in bytes
    Size    uint64    `json:"size"`    // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

    ModTime time.Time `json:"mtime"`

    // the total number of chunks
    Chunks  uint32    `json:"chunks"` // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

    // as in http, <digest-algorithm>=<digest-value>
    Digest  string    `json:"digest,omitempty"`
    Deleted bool      `json:"deleted,omitempty"`
}

mtmk · 2025-10-04T22:50:11Z

tests/NATS.Client.ObjectStore.Tests/ObjectStoreTest.cs

+        var info = await store.GetInfoAsync("k1", cancellationToken: cancellationToken);
+
+        // Verify Size is ulong
+        ulong size = info.Size; // Should compile without error


since using large files won't be possible I just check the type here.

mtmk · 2025-10-04T22:50:50Z

src/NATS.Client.ObjectStore/Models/ObjectMetadata.cs

    [JsonPropertyName("chunks")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
-    public int Chunks { get; set; }
+    public uint Chunks { get; set; }


turns out this one is not align with the spec either (ADR-20)

I have long (int 64) in both java and .net v1. ADR has uint32 so I think this is fine.

mtmk

LGTM thanks @darkwatchuk

(I shall ask for one more review since I added a change too)

Disable style warnings in ObjectStoreTest to ensure compatibility with type validation tests

darkwatchuk · 2025-10-06T06:24:06Z

LGTM thanks @darkwatchuk

(I shall ask for one more review since I added a change too)

Thank you!

mtmk · 2025-10-15T10:59:43Z

sorry for the delay @darkwatchuk we should get this merged soon then I'm hoping to get it released some time this week.

scottf

LGTM

scottf · 2025-10-15T13:59:39Z

src/NATS.Client.ObjectStore/Models/ObjectMetadata.cs

    [JsonPropertyName("chunks")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
-    public int Chunks { get; set; }
+    public uint Chunks { get; set; }


I have long (int 64) in both java and .net v1. ADR has uint32 so I think this is fine.

* fix: handle 408 Requests Pending responses for fetch (#973) (#979) * Object store item size fix (#977)

* fix: handle 408 Requests Pending responses for fetch requests (#973) * (merge from main) Object store item size fix (#977)

* #863 Fix naming of consumer group attribute (#870) Signed-off-by: James Thompson <[email protected]> * #848 Tweak dependencies (#853) * #848 Tweak dependencies Signed-off-by: James Thompson <[email protected]> * Update NATS.Client.Core.csproj Signed-off-by: James Thompson <[email protected]> * Update NATS.Client.Core.csproj Signed-off-by: James Thompson <[email protected]> * Make STJ explicit dependency for JetStream Signed-off-by: James Thompson <[email protected]> * Update NATS.Client.JetStream.csproj Signed-off-by: James Thompson <[email protected]> * Force sdk to 8.0.0 Signed-off-by: James Thompson <[email protected]> * Add newer stj for net 6 Signed-off-by: James Thompson <[email protected]> --------- Signed-off-by: James Thompson <[email protected]> * Add support for `Filter` and `Enrich` for OpenTelemetry activities (#859) * Add support for `Filter` and `Enrich` for OpenTelemetry activities * Make `internal` methods in `internal Telemetry` `public` * Fix package versions and whatnot * Remove `TracerProviderBuilderExtensions` * Include `Deserialize` in the receive activity * Revert back accidental change * Add `ParentContext` to `NatsInstrumentationContext` * Make `GetActivityContext` public to provide the ability to get context activity context * Make preprocessor directive more accurate * Revert .csproj formatting * Move public artifacts out of the `Internal` namespace/folder * Fix build script --------- Co-authored-by: Ziya Suzen <[email protected]> * Obsolete ReplyAsync method for NatsJsMsg (#839) * Obsolete ReplyAsync method * Obsolete ReplyAsync method on interface * #851 Move Serialization Interface into Abstractions (#858) * Tweaks Signed-off-by: James Thompson <[email protected]> * Add missing dependencies Signed-off-by: James Thompson <[email protected]> * switch to system.memory Signed-off-by: James Thompson <[email protected]> * Added missing using Signed-off-by: James Thompson <[email protected]> * Reduce csproj file contents Signed-off-by: James Thompson <[email protected]> --------- Signed-off-by: James Thompson <[email protected]> * Release 2.7.0-preview.1 (#905) * #851 Move Serialization Interface into Abstractions (#858) * Obsolete ReplyAsync method for NatsJsMsg (#839) * Add support for `Filter` and `Enrich` for OpenTelemetry activities (#859) * #848 Tweak dependencies (#853) * #863 Fix naming of consumer group attribute (#870) * fix: `NatsInstrumentationOptions.Default` gets reset each time (#907) * initialize `NatsInstrumentationOptions.Default` once * `NatsInstrumentationContext.ActivityContext` doesn't need to be nullable * no need for `GetActivityContext` on NatsJSMsg * Release 2.7.0-preview.2 (#908) * fix: `NatsInstrumentationOptions.Default` gets reset each time (#907) * Also, fixes from main * Fix build warnings (#912) * fix: `NatsJSConsumer` never disposing receive activities (#911) * Fix `NatsJSConsumer` never disposing receive activities * Use `ActivityEndingMsgReader` to `NatsJSFetch` and `NatsJSOrderedConsume` * Consolidate activity ending message readers into one class * Run `dotnet format` * Release 2.7.0-preview.3 (#922) * fix: `NatsJSConsumer` never disposing receive activities (#911) * Merge from main * chore: rework NatsOpt.Default initialization (#921) * chore: rework NatsOpt.Default initialization * chore: removed unused using statement * Fix kv ttl interface (#909) KV TTL should only be allowed on Create and Purge * Release 2.7.0-preview.4 (#931) Fixes merged from main * Release 2.7.0-preview.5 (#944) Merge from main aligning with 2.6.8 release. * Fix JetStream publish retry defaults (#939) Decision to retry jetstream publish requests should be letft to the application since it depends on the delivery and durability requirements of their solution. * Fix publish 503 test (#958) With 2.7.x we have made the default to not re-publish on failure. * Release 2.7.0-preview.6 (#956) * Fix JetStream publish retry defaults (#939) * Merge from main * Release 2.7.0-preview.7 (#970) Merged from main * fix: handle 408 Requests Pending responses for fetch requests (#973) * Handle 408 Requests Pending responses for fetch requests * Remove redundant `Console.WriteLine` that was put there for testing * Release 2.7.0-preview.8 (#986) * fix: handle 408 Requests Pending responses for fetch requests (#973) * (merge from main) Object store item size fix (#977) * Fix Ad-Hoc JSON Serializer to use Default Options (#984) * Fix connection state for consume (#959) * Fix connection state for consume * Enhance connection state handling with NatsConnectionFailedException * Improve error handling for connection failures and add support for configurable 503 error thresholds in JetStream consumers * Add tests for connection failure handling and configurable 503 error thresholds in JetStream consumers * Fix format * Fix test * Update INatsJsConsumer to return INatsJsMsg (#1004) --------- Signed-off-by: James Thompson <[email protected]> Co-authored-by: James Thompson <[email protected]> Co-authored-by: Arad Alvand <[email protected]> Co-authored-by: Yeong Jong Lim <[email protected]> Co-authored-by: regnrat <[email protected]>

darkwatchuk and others added 10 commits March 8, 2025 05:35

Make the parameter-less StreamConfig constructor public

8a54203

Merge branch 'nats-io:main' into main

7a2ddf6

Merge branch 'nats-io:main' into main

deb2c2e

Merge branch 'nats-io:main' into main

07df362

Make NatsMsg<T>.Build Public

2050c17

Add docs

134042d

Add docs

f0fdb45

[no ci]

Merge branch 'nats-io:main' into main

d45d936

Merge branch 'nats-io:main' into main

0b49cce

Change ObjectStore ObjectMetadata.Size to ulong from int

d5c2b4b

mtmk requested a review from Copilot October 4, 2025 21:42

Copilot AI reviewed Oct 4, 2025

View reviewed changes

tests/NATS.Client.ObjectStore.Tests/WatcherTest.cs Outdated Show resolved Hide resolved

tests/NATS.Client.ObjectStore.Tests/ObjectStoreTest.cs Outdated Show resolved Hide resolved

tests/NATS.Client.ObjectStore.Tests/ObjectStoreTest.cs Outdated Show resolved Hide resolved

mtmk self-requested a review October 4, 2025 22:14

Copilot feedback

689c7f5

mtmk self-assigned this Oct 4, 2025

mtmk added bug Something isn't working breaking labels Oct 4, 2025

mtmk added 2 commits October 4, 2025 23:42

Use uint for Chunks field and update related comments/test cases.

d871e11

Add test for validating metadata field types against NATS spec

d125a80

mtmk reviewed Oct 4, 2025

View reviewed changes

mtmk approved these changes Oct 4, 2025

View reviewed changes

mtmk requested a review from scottf October 4, 2025 22:52

Keep format happy

644958b

Disable style warnings in ObjectStoreTest to ensure compatibility with type validation tests

scottf approved these changes Oct 15, 2025

View reviewed changes

mtmk merged commit 4707497 into nats-io:main Oct 16, 2025
20 of 21 checks passed

mtmk added a commit that referenced this pull request Oct 17, 2025

Release 2.6.11

854a125

* fix: handle 408 Requests Pending responses for fetch (#973) (#979) * Object store item size fix (#977)

mtmk mentioned this pull request Oct 17, 2025

Release 2.6.11 #985

Merged

mtmk added a commit that referenced this pull request Oct 17, 2025

Release 2.6.11 (#985)

ee5acdd

* fix: handle 408 Requests Pending responses for fetch (#973) (#979) * Object store item size fix (#977)

mtmk added a commit that referenced this pull request Oct 20, 2025

Release 2.7.0-preview.8

8de44c1

* fix: handle 408 Requests Pending responses for fetch requests (#973) * (merge from main) Object store item size fix (#977)

mtmk mentioned this pull request Oct 20, 2025

Release 2.7.0-preview.8 #986

Merged

mtmk added a commit that referenced this pull request Oct 20, 2025

Release 2.7.0-preview.8 (#986)

56e56dc

* fix: handle 408 Requests Pending responses for fetch requests (#973) * (merge from main) Object store item size fix (#977)

This was referenced Nov 3, 2025

chore(deps): Bump NATS.Client.Core from 2.5.3 to 2.6.11 dylan-mccarthy/Scalable-Process-Agent-System#178

Closed

chore(deps): Bump NATS.Client.Core and NATS.Client.JetStream dylan-mccarthy/Scalable-Process-Agent-System#179

Merged

dependabot bot mentioned this pull request Dec 1, 2025

chore: Bump the nuget-all group with 6 updates ANcpLua/ancplua-mcp#66

Merged

dependabot bot mentioned this pull request Dec 22, 2025

chore(deps): Bump NATS.Net from 2.6.11 to 2.7.0 four-robots/dotgnatly#85

Closed

dependabot bot mentioned this pull request Jan 12, 2026

chore(deps): Bump NATS.Net from 2.6.11 to 2.7.1 four-robots/dotgnatly#86

Open

Object store item size fix #977

Object store item size fix #977

Uh oh!

Conversation

darkwatchuk commented Oct 3, 2025 • edited by mtmk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtmk commented Oct 4, 2025

Uh oh!

mtmk Oct 4, 2025

Choose a reason for hiding this comment

Uh oh!

mtmk Oct 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

scottf Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

mtmk left a comment

Choose a reason for hiding this comment

Uh oh!

darkwatchuk commented Oct 6, 2025

Uh oh!

mtmk commented Oct 15, 2025

Uh oh!

scottf left a comment

Choose a reason for hiding this comment

Uh oh!

scottf Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

darkwatchuk commented Oct 3, 2025 •

edited by mtmk

Loading

mtmk Oct 4, 2025 •

edited

Loading