Repository navigation
fix(cluster): hash lone surrogates as the UTF-8 Node sends - #2242
Merged
PavelPashov merged 1 commit intoOct 22, 2026
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
JacobLey
added a commit
to JacobLey/iovalkey
that referenced
this pull request
Oct 22, 2026
Ported from redis/ioredis#2242. Signed-off-by: Jacob Ley <jacobtley@gmail.com>
PavelPashov
approved these changes
Oct 22, 2026
PavelPashov
left a comment
Contributor
There was a problem hiding this comment.
@JacobLey LGTM, thank you for the contribution
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix cluster slot calculation for string keys that contain an unpaired UTF-16 surrogate. It follows up #2172, which vendored
cluster-key-slotintolib/utils/calculateSlot.ts.Bug
toUTF8Array()encodes a lone surrogate as a 3-byte sequence (for example\udc00becomesED B0 80). Node never sends those bytes. When ioredis writes a string argument to the socket, Node replaces each lone surrogate with U+FFFD (EF BF BD). So the client and the server hash different bytes:CLUSTER KEYSLOT"x\ud800y"(lone high)"x\udc00"(lone low)"abc\ud83d"(high at end)"a\udc00\ud800b"(low then high)The right-hand column by capturing the bytes ioredis wrote to a socket, and by running
CLUSTER KEYSLOTon a live Redis 8 server (they match)Consequence
Commands on such a key go to the wrong node, get
MOVED, and are retried. That costs an extra round trip on every command for the key. Every path that computes a slot from a string goes throughcalculateSlot(Command#getSlot, autopipelining including key prefixes,Pipeline,ClusterSubscriberGroup), so the fix covers all of them.Fix
In the 3-byte branch of
toUTF8Array(), a code unit in the surrogate range is replaced with0xFFFDbefore it is encoded. This matches Node's UTF-8 encoding exactly:Node's UTF-8 write passes V8's
kReplaceInvalidUtf8(string_bytes.cc,v8-primitive.h).Valid surrogate pairs are still handled by the 4-byte branch, as before. ASCII and 2-byte characters never reach the new check.
String.prototype.toWellFormed()would also work, buttsconfig.jsontargetslib: ES2021and the manual test workflow still runs Node 12–18. It was also measurably slower. Median ns/op over 2M calls on Node 24:toWellFormed()+ beforeTests
I added cases to
test/unit/calculateSlot.ts: a lone high surrogate, a lone low surrogate, a high surrogate at the end, and a low surrogate followed by a high one. Each asserts thatcalculateSlot(key)equalscalculateSlot(Buffer.from(key)). There is also a check against the server-measured slot 13299, and a control showing that a valid pair is unchanged.Note
Medium Risk
Changes cluster key routing for a rare edge case (malformed surrogate strings); wrong slots previously caused MOVED retries but correct routing is critical for all cluster commands.
Overview
Fixes cluster slot hashing for keys containing unpaired UTF-16 surrogates so ioredis hashes the same UTF-8 bytes Redis sees on the wire.
toUTF8Array()incalculateSlotused to UTF-8-encode lone surrogates directly; Node replaces them with U+FFFD when writing strings (and inBuffer.from). The 3-byte encoding path now maps surrogate code units to0xFFFDbefore encoding, matching V8’s invalid-UTF-8 handling; valid surrogate pairs are unchanged.New unit tests assert
calculateSlot(key)matchescalculateSlot(Buffer.from(key))for several lone-surrogate cases, one liveCLUSTER KEYSLOTvalue, and a valid emoji pair.Reviewed by Cursor Bugbot for commit 3fcf1c8. Bugbot is set up for automated code reviews on this repo. Configure here.