Skip to content

[SPARK-47172][CORE][3.5] Add support for AES-GCM for RPC encryption #47060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sweisdb wants to merge 3 commits into from
Closed

[SPARK-47172][CORE][3.5] Add support for AES-GCM for RPC encryption #47060

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ab78366
[SPARK-47172][CORE][3.5] Add support for AES-GCM for RPC encryption
sweisdb Jun 21, 2024
aec3208
[SPARK-47172][3.4] Fixing Java 8 build issues
sweisdb Jun 24, 2024
aca0295
[SPARK-47172][3.5] Fixing lint errors
sweisdb Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
[SPARK-47172][CORE][3.5] Add support for AES-GCM for RPC encryption 
This change adds AES-GCM as an optional AES cipher mode for RPC encryption. The current default is using AES-CTR without any authentication. That would allow someone on the network to easily modify RPC contents on the wire and impact Spark behavior. See [SPARK-47172](https://issues.apache.org/jira/browse/SPARK-47172) for more details.

The current default is using AES-CTR without any authentication. That would allow someone on the network to easily modify RPC contents on the wire and impact Spark behavior.

Yes, it adds an additional configuration flag is reflected in the documentation.

Existing unit tests are all ensured to pass. New unit tests are written to explicitly test GCM support and to verify that modifying ciphertext content will cause an exception and fail.

`build/sbt "network-common/test:testOnly"`
`build/sbt "network-common/test:testOnly org.apache.spark.network.crypto.AuthIntegrationSuite"`
`build/sbt "network-common/test:testOnly org.apache.spark.network.crypto.AuthEngineSuite"`

Nope.

Closes #46515 from sweisdb/SPARK-47172.

Authored-by: Steve Weis <[email protected]>
Signed-off-by: Yi Wu <[email protected]>
  • Loading branch information
@sweisdb
sweisdb committed Jun 25, 2024
commit ab783665b6a195586f180c3ecba73b8eaf8ef2b2
21 changes: 15 additions & 6 deletions common/network-common/src/main/java/org/apache/spark/network/crypto/AuthEngine.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ class AuthEngine implements Closeable {
public static final byte[] INPUT_IV_INFO = "inputIv".getBytes(UTF_8);
public static final byte[] OUTPUT_IV_INFO = "outputIv".getBytes(UTF_8);
private static final String MAC_ALGORITHM = "HMACSHA256";
private static final String LEGACY_CIPHER_ALGORITHM = "AES/CTR/NoPadding";
private static final String CIPHER_ALGORITHM = "AES/GCM/NoPadding";
private static final int AES_GCM_KEY_SIZE_BYTES = 16;
private static final byte[] EMPTY_TRANSCRIPT = new byte[0];
private static final int UNSAFE_SKIP_HKDF_VERSION = 1;
Expand Down Expand Up @@ -227,12 +229,19 @@ private TransportCipher generateTransportCipher(
OUTPUT_IV_INFO, // This is the HKDF info field used to differentiate IV values
AES_GCM_KEY_SIZE_BYTES);
SecretKeySpec sessionKey = new SecretKeySpec(derivedKey, "AES");
return new TransportCipher(
cryptoConf,
conf.cipherTransformation(),
sessionKey,
isClient ? clientIv : serverIv, // If it's the client, use the client IV first
isClient ? serverIv : clientIv);
if (LEGACY_CIPHER_ALGORITHM.equalsIgnoreCase(conf.cipherTransformation())) {
return new CtrTransportCipher(
cryptoConf,
sessionKey,
isClient ? clientIv : serverIv, // If it's the client, use the client IV first
isClient ? serverIv : clientIv);
} else if (CIPHER_ALGORITHM.equalsIgnoreCase(conf.cipherTransformation())) {
return new GcmTransportCipher(sessionKey);
} else {
throw new IllegalArgumentException(
String.format("Unsupported cipher mode: %s. %s and %s are supported.",
conf.cipherTransformation(), CIPHER_ALGORITHM, LEGACY_CIPHER_ALGORITHM));
}
}

private byte[] getTranscript(AuthMessage... encryptedPublicKeys) {
Expand Down
Loading