Skip to content

Commit 5d90583

Browse files
dongjoon-hyunkai-chi
authored andcommitted
[SPARK-26427][BUILD][BACKPORT-2.4] Upgrade Apache ORC to 1.5.4
This is a backport of apache#23364. To make Apache Spark 2.4.1 more robust, this PR aims to update Apache ORC dependency to the latest version 1.5.4 released at Dec. 20. ([Release Notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187])) ``` [ORC-237] OrcFile.mergeFiles Specified block size is less than configured minimum value [ORC-409] Changes for extending MemoryManagerImpl [ORC-410] Fix a locale-dependent test in TestCsvReader [ORC-416] Avoid opening data reader when there is no stripe [ORC-417] Use dynamic Apache Maven mirror link [ORC-419] Ensure to call `close` at RecordReaderImpl constructor exception [ORC-432] openjdk 8 has a bug that prevents surefire from working [ORC-435] Ability to read stripes that are greater than 2GB [ORC-437] Make acid schema checks case insensitive [ORC-411] Update build to work with Java 10. [ORC-418] Fix broken docker build script ``` Build and pass Jenkins. Closes apache#23646 from dongjoon-hyun/SPARK-26427-2.4. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 6674aa2 commit 5d90583

File tree

4 files changed

+14
-10
lines changed

4 files changed

+14
-10
lines changed

dev/deps/spark-deps-hadoop-2.6

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -153,9 +153,9 @@ objenesis-2.5.1.jar
153153
okhttp-3.8.1.jar
154154
okio-1.13.0.jar
155155
opencsv-2.3.jar
156-
orc-core-1.5.2-nohive.jar
157-
orc-mapreduce-1.5.2-nohive.jar
158-
orc-shims-1.5.2.jar
156+
orc-core-1.5.4-nohive.jar
157+
orc-mapreduce-1.5.4-nohive.jar
158+
orc-shims-1.5.4.jar
159159
oro-2.0.8.jar
160160
osgi-resource-locator-1.0.1.jar
161161
paranamer-2.8.jar

dev/deps/spark-deps-hadoop-2.7

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -154,9 +154,9 @@ objenesis-2.5.1.jar
154154
okhttp-3.8.1.jar
155155
okio-1.13.0.jar
156156
opencsv-2.3.jar
157-
orc-core-1.5.2-nohive.jar
158-
orc-mapreduce-1.5.2-nohive.jar
159-
orc-shims-1.5.2.jar
157+
orc-core-1.5.4-nohive.jar
158+
orc-mapreduce-1.5.4-nohive.jar
159+
orc-shims-1.5.4.jar
160160
oro-2.0.8.jar
161161
osgi-resource-locator-1.0.1.jar
162162
paranamer-2.8.jar

dev/deps/spark-deps-hadoop-3.1

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -172,9 +172,9 @@ okhttp-2.7.5.jar
172172
okhttp-3.8.1.jar
173173
okio-1.13.0.jar
174174
opencsv-2.3.jar
175-
orc-core-1.5.2-nohive.jar
176-
orc-mapreduce-1.5.2-nohive.jar
177-
orc-shims-1.5.2.jar
175+
orc-core-1.5.4-nohive.jar
176+
orc-mapreduce-1.5.4-nohive.jar
177+
orc-shims-1.5.4.jar
178178
oro-2.0.8.jar
179179
osgi-resource-locator-1.0.1.jar
180180
paranamer-2.8.jar

pom.xml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@
132132
<hive.version.short>3.0.0.1</hive.version.short>
133133
<derby.version>10.12.1.1</derby.version>
134134
<parquet.version>1.10.0</parquet.version>
135-
<orc.version>1.5.2</orc.version>
135+
<orc.version>1.5.4</orc.version>
136136
<orc.classifier></orc.classifier>
137137
<hive.parquet.version>1.6.0</hive.parquet.version>
138138
<jetty.version>9.3.24.v20180605</jetty.version>
@@ -1814,6 +1814,10 @@
18141814
<classifier>${orc.classifier}</classifier>
18151815
<scope>${orc.deps.scope}</scope>
18161816
<exclusions>
1817+
<exclusion>
1818+
<groupId>javax.xml.bind</groupId>
1819+
<artifactId>jaxb-api</artifactId>
1820+
</exclusion>
18171821
<exclusion>
18181822
<groupId>io.hops</groupId>
18191823
<artifactId>hadoop-common</artifactId>

0 commit comments

Comments
 (0)