[BUG] ck-server-2.5.0 breaking backward compatibility

[cyqsimon]

I have just updated my server binary to ck-server-linux-amd64-v2.5.0, and realised that it is having backward compatibility issues with ck-client-windows-amd64-v2.4.1.exe and ck-client-2.4.0.apk.

Those two are the ones I tested, but I have reasons to believe it's a decoding error with ck-server introduced in v2.5.0, which leads me to suspect that many or all earlier ck-client will not work correctly with ck-server-2.5.0.

---

My original setup:

( I will use Windows client as an example)

// works fine
[outside-GFW] - [ss-rust] - [ck-server-2.4.1] - [www] - [ck-client-2.4.1] - [ss-windows] - [inside-GFW]

• ck-server running in standalone mode
• ck-client running as a plugin for ss-windows
• ss-rust here refers to the ssserver binary of shadowsocks-rust

Only updating to ck-server-2.5.0 breaks

// breaks. See Symptoms for error messages
[outside-GFW] - [ss-rust] -/breaks here/- [ck-server-2.5.0] - [www] - [ck-client-2.4.1] - [ss-windows] - [inside-GFW]

Only updating to ck-client-2.5.0 does not break

// works fine
[outside-GFW] - [ss-rust] - [ck-server-2.4.1] - [www] - [ck-client-2.5.0] - [ss-windows] - [inside-GFW]

Updating both ck-client and ck-server does not break

// works fine
[outside-GFW] - [ss-rust] - [ck-server-2.5.0] - [www] - [ck-client-2.5.0] - [ss-windows] - [inside-GFW]

---

Symptoms

(as tested with ck-server-linux-amd64-v2.5.0 and ck-client-windows-amd64-v2.4.1.exe)

Client side

• All browser requests routed through Shadowsocks time out.
• ck-client indicates that session is successfully established.

time="/* time redacted */ t+0" level=info msg="Listening on TCP 127.0.0.1:62774 for shadowsocks client"
time="/* time redacted */ t+1" level=info msg="Attempting to start a new session"
time="/* time redacted */ t+1" level=info msg="Session 195156519 established"
// no further logs

• ss-windows indicates that socket is connected.

/* time redacted */ t+0.8129|DEBUG|Shadowsocks.Controller.TCPHandler|connect to www.google.com:443
/* time redacted */ t+0.8129|DEBUG|Shadowsocks.Controller.TCPHandler|Socket connected to ss server: /* addr redacted */
// no further logs

Server side

• ck-server indicates that session is successfully established.

time="/* time redacted */ t+0" level=info msg="Starting standalone mode"
time="/* time redacted */ t+0" level=info msg="Listening on :/* port redacted */"
time="/* time redacted */ t+0" level=info msg="New session" UID="/* UID redacted */" sessionID=195156519

• ss-rust indicates that a package is received, but AEAD decryption failed.

/* time redacted */ t+0.028522907+08:00 ERROR [1123:140063113283328] [shadowsocks::crypto::ring] AEAD decrypt failed, input=b"\x9dP", tag=b"\xf6\xa5<\x01cB\x92\\\xcc\xa9\xbf\t\x8b\xe7\xe3\x13", opening: OpeningKey { algorithm: CHACHA20_POLY1305 }
/* time redacted */ t+0.028860989+08:00 ERROR [1123:140063113283328] [shadowsocks::relay::tcprelay::server] failed to decode Address, may be wrong method or key, from client 127.0.0.1:36520, error: AEAD decrypt error

This combination of error message is leading me to believe that ck-server-2.5.0 introduced a breaking bug that causes incorrect cleartext (from the perspective of ck-server) to be passed to the underlying program, if the cleartext was encoded using a ck-client prior to 2.5.0.

---

Extra info and env

None of the following config files were changed during update.

ck-client config

{
  "Transport": "direct",
  "ProxyMethod": "shadowsocks",
  "EncryptionMethod": "aes-gcm",
  "UID": "/* redacted */",
  "PublicKey": "/* redacted */",
  "ServerName": "/* redacted */",
  "NumConn": 6,
  "BrowserSig": "chrome"
}

ck-server config

{
  "BindAddr": [
    "/* redacted */"
  ],
  "ProxyBook": {
    "shadowsocks": [
      "tcp",
      "/* redacted */"
    ]
  },
  "BypassUID": [
    /* redacted */
  ],
  "RedirAddr": "localhost:443",
  "PrivateKey": "/* redacted */",
  "AdminUID": "/* redacted */",
  "DatabasePath": "/* redacted *//userinfo.db"
}

ss-rust config

{
  "local_address": "0.0.0.0",
  "local_port": 1081,
  "servers": [
    {
      "server": "0.0.0.0",
      "server_port": /* redacted */,
      "password": "/* redacted */",
      "timeout": 300,
      "method": "chacha20-ietf-poly1305",
    }
  ],
  "mode": "tcp_and_udp"
}

Server OS

CentOS 7

[root@VPS ~]# uname -msr
Linux 5.10.1-1.el7.elrepo.x86_64 x86_64

Client OS

Windows 10 Pro N x64 20H2

Shadowsocks

Server: ss-rust shadowsocks 1.8.23

Client: ss-windows Shadowsocks 4.3.3.0

[notsure2]

There was a commit to master about copying frames, can you retest with master branch ?

[cyqsimon]

There was a commit to master about copying frames, can you retest with master branch ?

Sorry I actually do not have Go development environment set up on my VPS, so that will take a while.

How certain are you about this issue being related to this known (and fixed) bug? If you feel relatively certain that this issue is already fixed in master branch, I'm happy to wait until next minor version release.

Thanks for the quick reply though 😄

[notsure2]

@cyqsimon I was able to reproduce your problem, it appears with using aes-gcm encryption mode in Cloak. I will test with master branch now to confirm if it is fixed or not.

[notsure2]

@cyqsimon confirmed. the bug is fixed in the latest master.

[cyqsimon]

Alright. Great to hear! Should we close this issue, or should we retroactively link this issue to the fixing commit? Or should I just leave it here until the next release?

[notsure2]

@cyqsimon until @cbeuw makes a release you can get almost bleeding edge compiled binaries from my fork https://github.com/notsure2/Cloak/releases/tag/v2.5.1-notsure2-2

[cyqsimon]

Awesome. Thank you! 👍

[cyqsimon]

@notsure2 🤔 Odd. I am currently using your binary, and the issue is still persisting.

/* time redacted */ t+0.984990321+08:00 ERROR [1123:140063113283328] [shadowsocks::crypto::ring] AEAD decrypt failed, input=b"\x87\xe7", tag=b"Z\x89\xa7\xe4\x16\x1f\xcf\x98\xec\xd4?b\xe7\x02?\x93", opening: OpeningKey { algorithm: CHACHA20_POLY1305 }
/* time redacted */ t+0.985741698+08:00 ERROR [1123:140063113283328] [shadowsocks::relay::tcprelay::server] failed to decode Address, may be wrong method or key, from client 127.0.0.1:38072, error: AEAD decrypt error

• Server: ck-server-linux-amd64-v2.5.1-notsure2-2
• Client: ck-client-windows-amd64-v2.4.1.exe

[notsure2]

oh I guess I was mistaken, I forgot that my server was already updated to 2.5.0 and I tested with the master branch against it and thought it was fine :-( I guess keep this bug open then

[cyqsimon]

If I use ck-client-windows-amd64-v2.5.1-notsure2-2.exe as my client then things are alright. But the issue's about backward compatibility in the first place so I don't know.

[cyqsimon]

oh I guess I was mistaken, I forgot that my server was already updated to 2.5.0 and I tested with the master branch against it and thought it was fine :-( I guess keep this bug open then

No problem. Thanks for helping!

[cbeuw]

Turns out the culpit was 415523f. I don't yet know why it breaks backwards compatibility but not communication between same versions. Will continue to investigate.

[cbeuw]

@cyqsimon Could you please try this release https://github.com/cbeuw/Cloak/releases/tag/v2.5.1? It works with my setup

[cyqsimon]

@cbeuw

Yes. Can confirm that v2.5.1 server is now backward compatible with v2.4.x client.

Much thanks! 🎉