Skip to content

fix(cluster): stay ended when nodes removed by disconnect() end late - #2237

Merged
PavelPashov merged 1 commit into
redis:mainfrom
maxymlyskov:fix/stay-ended-after-disconnect
Oct 20, 2026
Merged

PavelPashov merged 1 commit into
redis:mainfrom
maxymlyskov:fix/stay-ended-after-disconnect

Conversation

@maxymlyskov

@maxymlyskov maxymlyskov commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

After cluster.disconnect(), cluster.status ends at close and every later command waits in the offline queue and never settles. The README says end "emits after close when no more reconnections will be made", but on main close fires again after end.

Cause: disconnect() calls connectionPool.reset([]), which removes every node in the same loop that disconnects them. Each node's end lands later, finds the pool empty and emits drain, and every drain sets close. The first goes through handleCloseEvent to end, the rest set close again. sendCommand only rejects in end, so in close the command gets queued. quit() lets each node remove itself on end, so it drains once and stays end.

Fix: the drain handler returns early when this.status === "end", the same guard connect() and sendCommand() use. 3 insertions in lib/cluster/index.ts. Other statuses still get close, so #2217's handleAbortedConnect path is unchanged.

connect() after disconnect() still works (checked after the nodes end and in the same tick). One difference: it now starts from end, so it resets retryAttempts like after quit().

Left as is: several drains before the close listener runs still emit duplicate close events before end, though the status ends at end. A pool that drains once per emptying would fix it too, but that's bigger (#1864 tried it in the pool, closed unmerged), so I kept the guard in the cluster.

Real cluster (3 masters + 3 replicas), same in 3 of 3 runs:

[main run 1] disconnect: events connecting > connect > ready > disconnecting > close > end > close > close > close > close > close | final status "close" | final get PENDING after 2000ms | 2384ms
[fix run 1] disconnect: events connecting > connect > ready > disconnecting > close > end | final status "end" | final get rejected: Connection is closed. | 396ms
[main run 1] quit: events connecting > connect > ready > disconnecting > close > end | final status "end" | final get rejected: Connection is closed. | 392ms

Tests:

  • test/functional/cluster/disconnection.ts, "should stay ended when the nodes end after disconnect": after disconnect() it waits for every node's end and a setImmediate, then expects end and a rejected get. Waiting on the cluster's end alone passes on main, the nodes end after it. On main:
  2 passing (643ms)
  1 failing

  1) disconnection
       should stay ended when the nodes end after disconnect:

      AssertionError: expected 'close' to equal 'end'
  • npm run test:js (Redis 8.10.2): 2883 passing, 25 pending, 0 failing. main: 2881 passing, 1 failing (autoPipelining for single node "should handle large pipelines without RangeError", 8000 ms timeout; passed on the branch run).
  • npm run test:cluster: 66 passing on both.
  • build, npx tsd, prettier --check clean. lint: 0 errors, 23 warnings, same as main.

Note

Low Risk
Small guard on an existing lifecycle path; behavior aligns disconnect with quit and only affects post-disconnect status and command rejection.

Overview
Fixes a bug where cluster.disconnect() could leave the client in close instead of end when node connections finish shutting down after the pool is already empty.

The connection pool drain handler now skips setStatus("close") if status is already end, matching guards used in connect() and sendCommand(). That stops late drains (from each node’s end after connectionPool.reset([])) from regressing the cluster to close, where commands sit in the offline queue instead of rejecting with "Connection is closed."

Adds a functional test that **disconnect()**s a multi-node mock cluster, waits for every node end, and asserts status === "end" and get() rejects.

Reviewed by Cursor Bugbot for commit e41b22c. Bugbot is set up for automated code reviews on this repo. Configure here.

disconnect() empties the connection pool before the node connections
have closed. Each node's later "end" then finds the pool empty and emits
"drain", and every drain set the cluster status to "close". The first
one closes and ends the cluster, the rest move the ended cluster back to
"close", where sendCommand() queues commands instead of rejecting them,
so they never settle.

Ignore a drain once the cluster has ended. A drain in any other status
still closes the cluster, so quit(), reconnects and the aborted-connect
path from redis#2217 are unchanged.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-19T14:40:19.808619Z e41b22c PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@PavelPashov PavelPashov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maxymlyskov LGTM thank you for the contribution

@PavelPashov
PavelPashov merged commit 61896e5 into redis:main Oct 20, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants