[SPARK-35041][SQL] Revise the overflow in UTF8String

[ulysses-you]

What changes were proposed in this pull request?

Add overflow check before do new byte[].

Why are the changes needed?

Avoid overflow in extreme case.

Does this PR introduce any user-facing change?

Maybe yes, the error msg changed if overflow.

How was this patch tested?

Pass CI.

[ulysses-you]

cc @maropu @MaxGekk

[MaxGekk]

How about:

• spark/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java

  Line 767 in 1be1012

  byte[] newBytes = new byte[numBytes * times];

• I would add an assert here

  spark/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java

  Lines 886 to 887 in 1be1012

  	int size = numBytes - delim.numBytes - idx;
  	byte[] bytes = new byte[size];

  to be sure that size doesn't overflow
• Replace Ints.checkedCast at

  spark/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java

  Line 974 in 1be1012

  final byte[] result = new byte[Ints.checkedCast(totalLength)];

[ulysses-you]

int size = numBytes - delim.numBytes - idx;
byte[] bytes = new byte[size];

The second place seems will never overflow. Any reason to assert not overflow here ?

[MaxGekk]

The second place seems will never overflow. Any reason to assert not overflow here ?

ok. It seems we can guarantee that numBytes > idx + delim.numBytes and size <= Int.MaxValue

[MaxGekk]

Suggested change

	int resultSize = Math.toIntExact((long)numBytes * times);
	byte[] newBytes = new byte[resultSize];
	byte[] newBytes = new byte[Math.multiplyExact(numBytes, times)];

[ulysses-you]

thanks for the carefully review !

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41853/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41853/

[SparkQA]

Test build #137274 has finished for PR 32142 at commit d43643e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41862/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41862/

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41865/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41865/

[SparkQA]

Test build #137283 has finished for PR 32142 at commit 6f48d88.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[MaxGekk]

+1, LGTM. Merging to master.
Thank you, @ulysses-you .

[ulysses-you]

thanks for merging