[SPARK-48583][SQL][TESTS] Replace deprecated classes and methods of commons-io called in Spark
#46935
Conversation
|
Is this the only case related to commons-io? |
For |
|
Yes, if there are other cases related to commons-io, let's fix them together all in one |
|
@wayneguow seems there are other cases related to commons-io in |
@LuciferYang Thank you for pointing that out. I'll fix this and other left in a batch. |
wayneguow
changed the title
FileUtils#writeStringToFile commons-io called in Spark
|
@LuciferYang I made a double check, the following
In the build log, only |
wayneguow
changed the title
commons-io called in Sparkcommons-io called in Spark
wayneguow
changed the title
commons-io called in Sparkcommons-io called in Spark
| countingIn = new CountingInputStream(fsin) | ||
| countingIn = BoundedInputStream.builder() | ||
| .setInputStream(fsin) | ||
| .setMaxCount(fileSplit.getLength) |
There was a problem hiding this comment.
Is this really necessary in the current scenario? Perhaps the unbound one is enough?
There was a problem hiding this comment.
Sounds good, changed to unbound .
|
Merged into master for Spark 4.0. Thanks @wayneguow |
2 participants
What changes were proposed in this pull request?
This pr replaces deprecated classes and methods of
commons-iocalled in Spark：writeStringToFile(final File file, final String data)->writeStringToFile(final File file, final String data, final Charset charset)CountingInputStream->BoundedInputStreamWhy are the changes needed?
Clean up deprecated API usage.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Passed related test cases in
UDFXPathUtilSuiteandXmlSuite.Was this patch authored or co-authored using generative AI tooling?
No.