Repository navigation
fix(cluster): hash autopipelined Buffer keys by their bytes - #2208
Merged
PavelPashov merged 1 commit intoSep 30, 2026
Merged
Conversation
Auto-pipelining built the cluster slot key with a template string, which decodes Buffer keys as UTF-8. Binary keys that are not valid UTF-8 then hash to the wrong slot, and different keys can collapse into the same auto pipeline even though their real slots live on different nodes. The batch then failed with "All keys in the pipeline should belong to the same slots allocation group". Hash Buffer keys by their raw bytes, as Command#getSlot already does.
PavelPashov
approved these changes
Sep 30, 2026
PavelPashov
left a comment
Contributor
There was a problem hiding this comment.
@breken-ai looks good, thank you for the contribution
github-actions Bot
pushed a commit
that referenced
this pull request
Oct 12, 2026
# [6.1.0-beta.1](v6.0.0...v6.1.0-beta.1) (2026-10-12) ### Bug Fixes * accept field arrays in hmget/hmgetBuffer types ([#2167](#2167)) ([c0cd66c](c0cd66c)) * align auto-pipeline callbacks for dynamic commands ([#2180](#2180)) ([7479cab](7479cab)) * clean up Sentinel failover subscriptions after quit ([#2198](#2198)) ([727edfa](727edfa)) * cluster stuck in close ([#2217](#2217)) ([a5fad1d](a5fad1d)) * **cluster:** bucket autopipelined MSET objects by their first key ([#2207](#2207)) ([b4e3237](b4e3237)) * **cluster:** hash autopipelined Buffer keys by their bytes ([#2208](#2208)) ([a8378bd](a8378bd)), closes [Command#getSlot](https://github.com/Command/issues/getSlot) * **cluster:** match Buffer shard channels when unsubscribing ([#2203](#2203)) ([d997d01](d997d01)) * **cluster:** support zero-argument sunsubscribe with sharded subscribers ([#2137](#2137)) ([13ff050](13ff050)), closes [#2133](#2133) * **cluster:** vendor cluster-key-slot to fix empty hash tag slot bug ([#2172](#2172)) ([b479b76](b479b76)) * decode integer replies up to MAX_SAFE_INTEGER exactly ([#2164](#2164)) ([28940b5](28940b5)) * decode RESP3 attribute replies instead of failing the connection ([#2213](#2213)) ([6a8cb3e](6a8cb3e)), closes [#decodeNestedType](https://github.com/redis/ioredis/issues/decodeNestedType) [#decodeMapAsObject](https://github.com/redis/ioredis/issues/decodeMapAsObject) [#decodeMapKey](https://github.com/redis/ioredis/issues/decodeMapKey) * decode RESP3 doubles with Number ([#2150](#2150)) ([6fb5a10](6fb5a10)) * handle rejections from auto-resubscribe and select on reconnect ([#2187](#2187)) ([de2fbdf](de2fbdf)) * inherit connectTimeout for cluster subscribers ([#2220](#2220)) ([db57c2d](db57c2d)) * keep String.prototype in fast mode after defining VerbatimString ([#2206](#2206)) ([90e1261](90e1261)) * keep subscriber mode while other channel kinds remain subscribed ([#2196](#2196)) ([eb97beb](eb97beb)) * keep the exec error out of its own previousErrors ([#2218](#2218)) ([2c602cf](2c602cf)) * preserve empty keys and match patterns in scan streams ([#2199](#2199)) ([b169c3a](b169c3a)) * reject unfulfilled commands dropped on reconnect ([#2194](#2194)) ([d95d05a](d95d05a)), closes [#2193](#2193) * reset retry attempts when connect() is called after giving up ([#2204](#2204)) ([b597303](b597303)), closes [#1463](#1463) * **types:** accept documented TLS profile options ([#2195](#2195)) ([56a7fb8](56a7fb8)), closes [#1626](#1626) * **types:** add event overloads to Cluster interface ([#2201](#2201)) ([3996eaf](3996eaf)), closes [#2191](#2191) * **types:** add smessage and smessageBuffer event overloads ([#2191](#2191)) ([ee48f4c](ee48f4c)), closes [#2190](#2190) * **types:** allow number for zrange stop argument ([#2163](#2163)) ([932f4c2](932f4c2)) ### Features * bless commands ([#2209](#2209)) ([9181ee0](9181ee0)) * **cluster:** allow configuring node role for Pub/Sub subscriber ([#2176](#2176)) ([6b9130c](6b9130c)) * smart client handoffs ([#2165](#2165)) ([d8ae3cb](d8ae3cb))
github-actions Bot
pushed a commit
that referenced
this pull request
Oct 12, 2026
# [6.1.0-beta.1](v6.0.0...v6.1.0-beta.1) (2026-10-12) ### Bug Fixes * accept field arrays in hmget/hmgetBuffer types ([#2167](#2167)) ([c0cd66c](c0cd66c)) * align auto-pipeline callbacks for dynamic commands ([#2180](#2180)) ([7479cab](7479cab)) * clean up Sentinel failover subscriptions after quit ([#2198](#2198)) ([727edfa](727edfa)) * cluster stuck in close ([#2217](#2217)) ([a5fad1d](a5fad1d)) * **cluster:** bucket autopipelined MSET objects by their first key ([#2207](#2207)) ([b4e3237](b4e3237)) * **cluster:** hash autopipelined Buffer keys by their bytes ([#2208](#2208)) ([a8378bd](a8378bd)), closes [Command#getSlot](https://github.com/Command/issues/getSlot) * **cluster:** match Buffer shard channels when unsubscribing ([#2203](#2203)) ([d997d01](d997d01)) * **cluster:** support zero-argument sunsubscribe with sharded subscribers ([#2137](#2137)) ([13ff050](13ff050)), closes [#2133](#2133) * **cluster:** vendor cluster-key-slot to fix empty hash tag slot bug ([#2172](#2172)) ([b479b76](b479b76)) * decode integer replies up to MAX_SAFE_INTEGER exactly ([#2164](#2164)) ([28940b5](28940b5)) * decode RESP3 attribute replies instead of failing the connection ([#2213](#2213)) ([6a8cb3e](6a8cb3e)), closes [#decodeNestedType](https://github.com/redis/ioredis/issues/decodeNestedType) [#decodeMapAsObject](https://github.com/redis/ioredis/issues/decodeMapAsObject) [#decodeMapKey](https://github.com/redis/ioredis/issues/decodeMapKey) * decode RESP3 doubles with Number ([#2150](#2150)) ([6fb5a10](6fb5a10)) * handle rejections from auto-resubscribe and select on reconnect ([#2187](#2187)) ([de2fbdf](de2fbdf)) * inherit connectTimeout for cluster subscribers ([#2220](#2220)) ([db57c2d](db57c2d)) * keep String.prototype in fast mode after defining VerbatimString ([#2206](#2206)) ([90e1261](90e1261)) * keep subscriber mode while other channel kinds remain subscribed ([#2196](#2196)) ([eb97beb](eb97beb)) * keep the exec error out of its own previousErrors ([#2218](#2218)) ([2c602cf](2c602cf)) * preserve empty keys and match patterns in scan streams ([#2199](#2199)) ([b169c3a](b169c3a)) * reject unfulfilled commands dropped on reconnect ([#2194](#2194)) ([d95d05a](d95d05a)), closes [#2193](#2193) * reset retry attempts when connect() is called after giving up ([#2204](#2204)) ([b597303](b597303)), closes [#1463](#1463) * **types:** accept documented TLS profile options ([#2195](#2195)) ([56a7fb8](56a7fb8)), closes [#1626](#1626) * **types:** add event overloads to Cluster interface ([#2201](#2201)) ([3996eaf](3996eaf)), closes [#2191](#2191) * **types:** add smessage and smessageBuffer event overloads ([#2191](#2191)) ([ee48f4c](ee48f4c)), closes [#2190](#2190) * **types:** allow number for zrange stop argument ([#2163](#2163)) ([932f4c2](932f4c2)) ### Features * bless commands ([#2209](#2209)) ([9181ee0](9181ee0)) * **cluster:** allow configuring node role for Pub/Sub subscriber ([#2176](#2176)) ([6b9130c](6b9130c)) * smart client handoffs ([#2165](#2165)) ([d8ae3cb](d8ae3cb))
github-actions Bot
pushed a commit
that referenced
this pull request
Oct 12, 2026
# [6.1.0-beta.1](v6.0.0...v6.1.0-beta.1) (2026-10-12) ### Bug Fixes * accept field arrays in hmget/hmgetBuffer types ([#2167](#2167)) ([c0cd66c](c0cd66c)) * align auto-pipeline callbacks for dynamic commands ([#2180](#2180)) ([7479cab](7479cab)) * clean up Sentinel failover subscriptions after quit ([#2198](#2198)) ([727edfa](727edfa)) * cluster stuck in close ([#2217](#2217)) ([a5fad1d](a5fad1d)) * **cluster:** bucket autopipelined MSET objects by their first key ([#2207](#2207)) ([b4e3237](b4e3237)) * **cluster:** hash autopipelined Buffer keys by their bytes ([#2208](#2208)) ([a8378bd](a8378bd)), closes [Command#getSlot](https://github.com/Command/issues/getSlot) * **cluster:** match Buffer shard channels when unsubscribing ([#2203](#2203)) ([d997d01](d997d01)) * **cluster:** support zero-argument sunsubscribe with sharded subscribers ([#2137](#2137)) ([13ff050](13ff050)), closes [#2133](#2133) * **cluster:** vendor cluster-key-slot to fix empty hash tag slot bug ([#2172](#2172)) ([b479b76](b479b76)) * decode integer replies up to MAX_SAFE_INTEGER exactly ([#2164](#2164)) ([28940b5](28940b5)) * decode RESP3 attribute replies instead of failing the connection ([#2213](#2213)) ([6a8cb3e](6a8cb3e)), closes [#decodeNestedType](https://github.com/redis/ioredis/issues/decodeNestedType) [#decodeMapAsObject](https://github.com/redis/ioredis/issues/decodeMapAsObject) [#decodeMapKey](https://github.com/redis/ioredis/issues/decodeMapKey) * decode RESP3 doubles with Number ([#2150](#2150)) ([6fb5a10](6fb5a10)) * handle rejections from auto-resubscribe and select on reconnect ([#2187](#2187)) ([de2fbdf](de2fbdf)) * inherit connectTimeout for cluster subscribers ([#2220](#2220)) ([db57c2d](db57c2d)) * keep String.prototype in fast mode after defining VerbatimString ([#2206](#2206)) ([90e1261](90e1261)) * keep subscriber mode while other channel kinds remain subscribed ([#2196](#2196)) ([eb97beb](eb97beb)) * keep the exec error out of its own previousErrors ([#2218](#2218)) ([2c602cf](2c602cf)) * preserve empty keys and match patterns in scan streams ([#2199](#2199)) ([b169c3a](b169c3a)) * reject unfulfilled commands dropped on reconnect ([#2194](#2194)) ([d95d05a](d95d05a)), closes [#2193](#2193) * reset retry attempts when connect() is called after giving up ([#2204](#2204)) ([b597303](b597303)), closes [#1463](#1463) * **types:** accept documented TLS profile options ([#2195](#2195)) ([56a7fb8](56a7fb8)), closes [#1626](#1626) * **types:** add event overloads to Cluster interface ([#2201](#2201)) ([3996eaf](3996eaf)), closes [#2191](#2191) * **types:** add smessage and smessageBuffer event overloads ([#2191](#2191)) ([ee48f4c](ee48f4c)), closes [#2190](#2190) * **types:** allow number for zrange stop argument ([#2163](#2163)) ([932f4c2](932f4c2)) ### Features * bless commands ([#2209](#2209)) ([9181ee0](9181ee0)) * **cluster:** allow configuring node role for Pub/Sub subscriber ([#2176](#2176)) ([6b9130c](6b9130c)) * smart client handoffs ([#2165](#2165)) ([d8ae3cb](d8ae3cb))
|
🎉 This PR is included in version 6.1.0-beta.1 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
JacobLey
added a commit
to JacobLey/iovalkey
that referenced
this pull request
Oct 22, 2026
Auto-pipelining built the slot key with a template string, which decodes a Buffer key as UTF-8. Bytes that are not valid UTF-8 become U+FFFD, so different binary keys could share an auto pipeline even when their slots are served by different nodes, and Pipeline#exec rejected the batch. Hash Buffer keys from their bytes (with keyPrefix prepended), as Command does. Ported from redis/ioredis#2208. Signed-off-by: Jacob Ley <jacobtley@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request fixes auto-pipelining in cluster mode for commands whose key is a binary
Buffer.Bug
executeWithAutoPipeliningbuilds the slot for the auto pipeline with a template string,`${prefix}${key}`. That decodes aBufferkey as UTF-8, so bytes that are not valid UTF-8 becomeU+FFFD, and the slot is computed for the decoded string instead of the bytes that go on the wire.Command#getSlotandPipeline#exechash the raw bytes, so the two disagree.Different binary keys can then land in the same auto pipeline even though their real slots belong to different nodes, and
Pipeline#execrejects the whole batch:Repro with
enableAutoPipelining: trueon a 3-node cluster (slots 0-5000, 5001-9999, 10000-16383):Without auto-pipelining the same calls work. This affects binary keys such as serialized IDs or hashes stored as raw bytes.
Fix
When the first key is a
Buffer, compute the slot fromBuffer.concat([Buffer.from(prefix), key]), the same bytesCommandsends after applyingkeyPrefix. String and number keys still use the existing string path.Tests
Added
should bucket binary Buffer keys by their raw bytestotest/functional/cluster/autopipelining.ts(mock servers, no Redis needed). It fails onmainwith the error above and passes with the fix.test/unitplustest/functional/clusterpass (591 passing), andtsc --noEmit, eslint and prettier are clean.This is a small, isolated bug fix with a focused test, so I opened it directly as CONTRIBUTING allows.
Note
Low Risk
Small, isolated change to cluster auto-pipeline slot routing for Buffer keys, with a focused regression test.
Overview
Fixes cluster auto-pipelining when commands use binary
Bufferkeys. Slot grouping used`${prefix}${key}`, which turns invalid UTF-8 bytes into the same replacement character and can put keys that belong on different nodes into one pipeline;Pipeline#execthen rejects the batch.executeWithAutoPipeliningnow hashesBufferkeys fromBuffer.concat([Buffer.from(prefix), key]), matching howCommandapplieskeyPrefixbeforecalculateSlot. String and number keys are unchanged.Adds a functional test that
Promise.alltwogetcalls withBuffer.from([0xff])andBuffer.from([0xfe])underenableAutoPipelining, with mock handlers on the correct nodes.Reviewed by Cursor Bugbot for commit ca8d279. Bugbot is set up for automated code reviews on this repo. Configure here.