Commit b3f2911
[SPARK-24945][SQL] Switching to uniVocity 2.7.3
## What changes were proposed in this pull request?
In the PR, I propose to upgrade uniVocity parser from **2.6.3** to **2.7.3**. The recent version includes a fix for the SPARK-24645 issue and has better performance.
Before changes:
```
Parsing quoted values: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
One quoted string 33336 / 34122 0.0 666727.0 1.0X
Wide rows with 1000 columns: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Select 1000 columns 90287 / 91713 0.0 90286.9 1.0X
Select 100 columns 31826 / 36589 0.0 31826.4 2.8X
Select one column 25738 / 25872 0.0 25737.9 3.5X
count() 6931 / 7269 0.1 6931.5 13.0X
```
after:
```
Parsing quoted values: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
One quoted string 33411 / 33510 0.0 668211.4 1.0X
Wide rows with 1000 columns: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Select 1000 columns 88028 / 89311 0.0 88028.1 1.0X
Select 100 columns 29010 / 32755 0.0 29010.1 3.0X
Select one column 22936 / 22953 0.0 22936.5 3.8X
count() 6657 / 6740 0.2 6656.6 13.5X
```
Closes #21892
## How was this patch tested?
It was tested by `CSVSuite` and `CSVBenchmarks`
Author: Maxim Gekk <[email protected]>
Closes #21969 from MaxGekk/univocity-2_7_3.1 parent 7cf16a7 commit b3f2911
File tree
4 files changed
+4
-4
lines changed- dev/deps
- sql/core
4 files changed
+4
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
191 | 191 | | |
192 | 192 | | |
193 | 193 | | |
194 | | - | |
| 194 | + | |
195 | 195 | | |
196 | 196 | | |
197 | 197 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
192 | 192 | | |
193 | 193 | | |
194 | 194 | | |
195 | | - | |
| 195 | + | |
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
215 | | - | |
| 215 | + | |
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
0 commit comments