Skip to content

Conversation

@mtmk
Copy link
Member

@mtmk mtmk commented Sep 22, 2025

@mtmk mtmk linked an issue Sep 22, 2025 that may be closed by this pull request
@mtmk mtmk force-pushed the 855-fix-connection-state-for-consume branch from f767c71 to 0bec181 Compare October 9, 2025 07:50
@mtmk mtmk marked this pull request as ready for review October 28, 2025 12:16
@mtmk mtmk requested a review from Copilot October 28, 2025 12:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes connection state handling for the consume method when connections fail permanently, resolving #855. It adds a new Failed connection state and throws NatsConnectionFailedException when max reconnect retries are exceeded, ensuring consumers stop gracefully rather than hanging indefinitely. Additionally, it implements detection for deleted ephemeral consumers via consecutive 503 error monitoring.

Key Changes:

  • Added NatsConnectionState.Failed state and NatsConnectionFailedException for permanent connection failures
  • Implemented ephemeral consumer deletion detection using configurable consecutive 503 error threshold
  • Updated consume methods to check connection state and propagate failures appropriately

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/NATS.Client.Core/NatsConnection.cs Added Failed state and connection failure checks with new exception type
src/NATS.Client.Core/NatsException.cs Defined new NatsConnectionFailedException class
src/NATS.Client.JetStream/NatsJSConsumer.cs Added exception handling for connection failures and consumer errors
src/NATS.Client.JetStream/NatsJSOrderedConsumer.cs Added catch blocks for connection and consumer-related exceptions
src/NATS.Client.JetStream/Internal/NatsJSConsume.cs Implemented connection state checks and 503 error counter with threshold
src/NATS.Client.JetStream/NatsJSOpts.cs Added MaxConsecutive503Errors configuration option
tests/NATS.Client.Core.Tests/ConnectionRetryTest.cs Updated test to expect NatsConnectionFailedException
tests/NATS.Client.JetStream.Tests/ConsumerConsumeTest.cs Added tests for connection failure and 503 threshold scenarios
tests/NATS.Client.JetStream.Tests/OrderedConsumerTest.cs Added test for ordered consumer connection failure
tools/site_src/documentation/jetstream/consume.md Documented connection failure handling and ephemeral consumer detection

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

@scottf scottf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mtmk mtmk merged commit ac5fe82 into release/2.7 Oct 29, 2025
23 of 25 checks passed
@mtmk mtmk deleted the 855-fix-connection-state-for-consume branch October 29, 2025 11:18
@mtmk mtmk mentioned this pull request Dec 19, 2025
mtmk added a commit that referenced this pull request Dec 19, 2025
* #863 Fix naming of consumer group attribute (#870)

Signed-off-by: James Thompson <[email protected]>

* #848 Tweak dependencies (#853)

* #848 Tweak dependencies

Signed-off-by: James Thompson <[email protected]>

* Update NATS.Client.Core.csproj

Signed-off-by: James Thompson <[email protected]>

* Update NATS.Client.Core.csproj

Signed-off-by: James Thompson <[email protected]>

* Make STJ explicit dependency for JetStream

Signed-off-by: James Thompson <[email protected]>

* Update NATS.Client.JetStream.csproj

Signed-off-by: James Thompson <[email protected]>

* Force sdk to 8.0.0

Signed-off-by: James Thompson <[email protected]>

* Add newer stj for net 6

Signed-off-by: James Thompson <[email protected]>

---------

Signed-off-by: James Thompson <[email protected]>

* Add support for `Filter` and `Enrich` for OpenTelemetry activities (#859)

* Add support for `Filter` and `Enrich` for OpenTelemetry activities

* Make `internal` methods in `internal Telemetry` `public`

* Fix package versions and whatnot

* Remove `TracerProviderBuilderExtensions`

* Include `Deserialize` in the receive activity

* Revert back accidental change

* Add `ParentContext` to `NatsInstrumentationContext`

* Make `GetActivityContext` public to provide the ability to get context activity context

* Make preprocessor directive more accurate

* Revert .csproj formatting

* Move public artifacts out of the `Internal` namespace/folder

* Fix build script

---------

Co-authored-by: Ziya Suzen <[email protected]>

* Obsolete ReplyAsync method for NatsJsMsg (#839)

* Obsolete ReplyAsync method

* Obsolete ReplyAsync method on interface

* #851 Move Serialization Interface into Abstractions (#858)

* Tweaks

Signed-off-by: James Thompson <[email protected]>

* Add missing dependencies

Signed-off-by: James Thompson <[email protected]>

* switch to system.memory

Signed-off-by: James Thompson <[email protected]>

* Added missing using

Signed-off-by: James Thompson <[email protected]>

* Reduce csproj file contents

Signed-off-by: James Thompson <[email protected]>

---------

Signed-off-by: James Thompson <[email protected]>

* Release 2.7.0-preview.1 (#905)

* #851 Move Serialization Interface into Abstractions (#858)
* Obsolete ReplyAsync method for NatsJsMsg (#839)
* Add support for `Filter` and `Enrich` for OpenTelemetry activities (#859)
* #848 Tweak dependencies (#853)
* #863 Fix naming of consumer group attribute (#870)

* fix: `NatsInstrumentationOptions.Default` gets reset each time (#907)

* initialize `NatsInstrumentationOptions.Default` once

* `NatsInstrumentationContext.ActivityContext` doesn't need to be nullable

* no need for `GetActivityContext` on NatsJSMsg

* Release 2.7.0-preview.2 (#908)

* fix: `NatsInstrumentationOptions.Default` gets reset each time (#907)
* Also, fixes from main

* Fix build warnings (#912)

* fix: `NatsJSConsumer` never disposing receive activities (#911)

* Fix `NatsJSConsumer` never disposing receive activities

* Use `ActivityEndingMsgReader` to `NatsJSFetch` and `NatsJSOrderedConsume`

* Consolidate activity ending message readers into one class

* Run `dotnet format`

* Release 2.7.0-preview.3 (#922)

* fix: `NatsJSConsumer` never disposing receive activities (#911)
* Merge from main

* chore: rework NatsOpt.Default initialization (#921)

* chore: rework NatsOpt.Default initialization

* chore: removed unused using statement

* Fix kv ttl interface (#909)

KV TTL should only be allowed on Create and Purge

* Release 2.7.0-preview.4 (#931)

Fixes merged from main

* Release 2.7.0-preview.5 (#944)

Merge from main aligning with 2.6.8 release.

* Fix JetStream publish retry defaults (#939)

Decision to retry jetstream publish requests should be letft to the
application since it depends on the delivery and durability
requirements of their solution.

* Fix publish 503 test (#958)

With 2.7.x we have made the default to not re-publish on failure.

* Release 2.7.0-preview.6 (#956)

* Fix JetStream publish retry defaults (#939)
* Merge from main

* Release 2.7.0-preview.7 (#970)

Merged from main

* fix: handle 408 Requests Pending responses for fetch requests (#973)

* Handle 408 Requests Pending responses for fetch requests

* Remove redundant `Console.WriteLine` that was put there for testing

* Release 2.7.0-preview.8 (#986)

* fix: handle 408 Requests Pending responses for fetch requests (#973)
* (merge from main) Object store item size fix (#977)

* Fix Ad-Hoc JSON Serializer to use Default Options (#984)

* Fix connection state for consume (#959)

* Fix connection state for consume

* Enhance connection state handling with NatsConnectionFailedException

* Improve error handling for connection failures and add support for configurable 503 error thresholds in JetStream consumers

* Add tests for connection failure handling and configurable 503 error thresholds in JetStream consumers

* Fix format

* Fix test

* Update INatsJsConsumer to return INatsJsMsg (#1004)

---------

Signed-off-by: James Thompson <[email protected]>
Co-authored-by: James Thompson <[email protected]>
Co-authored-by: Arad Alvand <[email protected]>
Co-authored-by: Yeong Jong Lim <[email protected]>
Co-authored-by: regnrat <[email protected]>
mtmk added a commit that referenced this pull request Dec 19, 2025
* Replace `OperationCanceledException` with `NatsTimeoutException` (#1022)
* Replace deprecated IndexRange package with Microsoft.Bcl.Memory to solve possible dependency conflicts (for netFramework/netStandard2.0) (#1009)
* PeerInfo.Lag is wrong type (#1021)
* (Merge Branch) Release 2.7 (#874)
  * Update INatsJsConsumer to return INatsJsMsg (#1004)
  * Fix connection state for consume (#959)
  * Fix Ad-Hoc JSON Serializer to use Default Options (#984)
  * fix: handle 408 Requests Pending responses for fetch requests (#973)
  * Fix publish 503 test (#958)
  * Fix JetStream publish retry defaults (#939)
  * Fix kv ttl interface (#909)
  * chore: rework NatsOpt.Default initialization (#921)
  * fix: `NatsJSConsumer` never disposing receive activities (#911)
  * Fix build warnings (#912)
  * fix: `NatsInstrumentationOptions.Default` gets reset each time (#907)
  * #851 Move Serialization Interface into Abstractions (#858)
  * Obsolete ReplyAsync method for NatsJsMsg (#839)
  * Add support for `Filter` and `Enrich` for OpenTelemetry activities (#859)
  * #848 Tweak dependencies (#853)
  * #863 Fix naming of consumer group attribute (#870)
@mtmk mtmk mentioned this pull request Dec 19, 2025
mtmk added a commit that referenced this pull request Dec 19, 2025
* Replace `OperationCanceledException` with `NatsTimeoutException` (#1022)
* Replace deprecated IndexRange package with Microsoft.Bcl.Memory to solve possible dependency conflicts (for netFramework/netStandard2.0) (#1009)
* PeerInfo.Lag is wrong type (#1021)
* (Merge Branch) Release 2.7 (#874)
  * Update INatsJsConsumer to return INatsJsMsg (#1004)
  * Fix connection state for consume (#959)
  * Fix Ad-Hoc JSON Serializer to use Default Options (#984)
  * fix: handle 408 Requests Pending responses for fetch requests (#973)
  * Fix publish 503 test (#958)
  * Fix JetStream publish retry defaults (#939)
  * Fix kv ttl interface (#909)
  * chore: rework NatsOpt.Default initialization (#921)
  * fix: `NatsJSConsumer` never disposing receive activities (#911)
  * Fix build warnings (#912)
  * fix: `NatsInstrumentationOptions.Default` gets reset each time (#907)
  * #851 Move Serialization Interface into Abstractions (#858)
  * Obsolete ReplyAsync method for NatsJsMsg (#839)
  * Add support for `Filter` and `Enrich` for OpenTelemetry activities (#859)
  * #848 Tweak dependencies (#853)
  * #863 Fix naming of consumer group attribute (#870)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stuck after max retries exceeded in ConsumeAsync

3 participants