Skip to content
Closed
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
22bd4bd
Merge pull request #1 from apache/master
DaveDeCaprio Nov 27, 2018
b7f964d
Added a configurable limit on the maximum length of a plan debug string.
DaveDeCaprio Nov 28, 2018
3f4fe1f
Removed unneeded imports
DaveDeCaprio Nov 28, 2018
30e4348
Moved withSizeLimitedWriter to treeString function that uses StringWr…
DaveDeCaprio Nov 28, 2018
5528ca1
Fixed formatting
DaveDeCaprio Nov 28, 2018
3171cf3
Coverted to javadoc style multiline comment
DaveDeCaprio Nov 28, 2018
3ffdc6a
Fixed scalastyle formatting of imports
DaveDeCaprio Nov 28, 2018
45a60fc
Have size limit cut off right at the correct number of characters.
DaveDeCaprio Nov 29, 2018
9678799
Changed name to remove the debug from the config parameter name
DaveDeCaprio Nov 29, 2018
a5af842
Fixed formatting error
DaveDeCaprio Nov 29, 2018
a4be985
Changed length default to Long.MaxValue to turn off behavior unless i…
DaveDeCaprio Dec 3, 2018
f0f75c2
Merge branch 'master' of https://github.com/apache/spark into text-pl…
DaveDeCaprio Dec 3, 2018
1b692a0
Added test case for not limiting plan length and tested with a defaul…
DaveDeCaprio Dec 4, 2018
22fe117
Correctly added test case missed in the previous commit
DaveDeCaprio Dec 4, 2018
be3f265
Added more documentation of the plan length parameter.
DaveDeCaprio Dec 8, 2018
f6d0efc
Merge branch 'master' of https://github.com/apache/spark into text-pl…
DaveDeCaprio Dec 8, 2018
855f540
Removed tab
DaveDeCaprio Dec 8, 2018
e83f5f2
Merge branch 'master' of https://github.com/apache/spark into text-pl…
DaveDeCaprio Jan 5, 2019
db663b7
Merge branch 'master' of https://github.com/apache/spark into text-pl…
DaveDeCaprio Jan 20, 2019
2eecbfa
Added plan size limits to StringConcat
DaveDeCaprio Jan 21, 2019
4082aa3
Scalastyle
DaveDeCaprio Jan 21, 2019
f9085e7
Incorporated changes from code review
DaveDeCaprio Jan 21, 2019
b3d43b7
Missed one error
DaveDeCaprio Jan 21, 2019
e470ab2
Cleaned up append function and added tracking of the full plan length.
DaveDeCaprio Jan 21, 2019
35bc1d5
Got rid of unneeded "availableLength" flag.
DaveDeCaprio Jan 21, 2019
5ec58c8
Fixed errors missed
DaveDeCaprio Jan 21, 2019
bdfaf28
Updated to handle nulls again.
DaveDeCaprio Jan 22, 2019
0cfcb4e
Tabs to spaces
DaveDeCaprio Jan 22, 2019
eb69888
Remove useless test.
DaveDeCaprio Jan 22, 2019
daf02f2
Addressed code review comments.
DaveDeCaprio Mar 8, 2019
7d89388
Missed some Boolean->Unit updates
DaveDeCaprio Mar 8, 2019
4f56e48
Missed some Boolean->Unit updates
DaveDeCaprio Mar 8, 2019
a090fbb
Code formatting
DaveDeCaprio Mar 8, 2019
dcb4eb0
Fixed Fatal Warning.
DaveDeCaprio Mar 8, 2019
b4cb7bf
Fixed failing unit tests
DaveDeCaprio Mar 9, 2019
4fec590
Style/formatting issues from code review
DaveDeCaprio Mar 11, 2019
db0db18
Style/formatting issues from code review
DaveDeCaprio Mar 11, 2019
b5b30f3
Changed plan string length to bytesConf and gave a better message for…
DaveDeCaprio Mar 13, 2019
e4afa26
Tabs to spaces
DaveDeCaprio Mar 13, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Coverted to javadoc style multiline comment
  • Loading branch information
DaveDeCaprio committed Nov 28, 2018
commit 3171cf31ae8001e5c37b31668154bbf2b6b4411f
Original file line number Diff line number Diff line change
Expand Up @@ -20,28 +20,29 @@ package org.apache.spark.sql.catalyst.util
import java.io.Writer

class WriterSizeException(val attemptedSize: Long, val charLimit: Long) extends Exception(
s"Attempted to write $attemptedSize characters to a writer that is limited to $charLimit")
s"Attempted to write $attemptedSize characters to a writer that is limited to $charLimit")

/**
* This class is used to control the size of generated writers. Guarantees that the total number
* of characters written will be less than the specified size.
*
* Checks size before writing and throws a WriterSizeException if the total size would count the limit.
*/
* This class is used to control the size of generated writers. Guarantees that the total number
* of characters written will be less than the specified size.
*
* Checks size before writing and throws a WriterSizeException if the total size would count the
* limit.
*/
class SizeLimitedWriter(underlying: Writer, charLimit: Long) extends Writer {

var charsWritten: Long = 0
var charsWritten: Long = 0

override def write(cbuf: Array[Char], off: Int, len: Int): Unit = {
val newLength = charsWritten + Math.min(cbuf.length - off, len)
if(newLength > charLimit) {
throw new WriterSizeException(newLength, charLimit)
}
charsWritten = newLength
underlying.write(cbuf, off, len)
}
override def write(cbuf: Array[Char], off: Int, len: Int): Unit = {
val newLength = charsWritten + Math.min(cbuf.length - off, len)
if (newLength > charLimit) {
throw new WriterSizeException(newLength, charLimit)
}
charsWritten = newLength
underlying.write(cbuf, off, len)
}

override def flush(): Unit = underlying.flush()
override def flush(): Unit = underlying.flush()

override def close(): Unit = underlying.close()
override def close(): Unit = underlying.close()
}