Skip to content

[SPARK-23711][SQL] Add fallback generator for UnsafeProjection #21106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
viirya wants to merge 19 commits into from
Closed

[SPARK-23711][SQL] Add fallback generator for UnsafeProjection #21106

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fe2a1cd
Add the base of fallback logic fo UnsafeProjection.
viirya Apr 19, 2018
36f90cf
Address comment.
viirya Apr 20, 2018
68213fb
Unify with UnsafeProjectionCreator.
viirya Apr 26, 2018
6cdaec4
Add a SQL config to control fallback mode under test.
viirya May 6, 2018
32e2766
Merge remote-tracking branch 'upstream/master' into SPARK-23711
viirya May 6, 2018
aeef468
Add test.
viirya May 6, 2018
1f5cc17
Don't access SQLConf on task execution.
viirya May 7, 2018
4c04d14
Change SQL conf naming.
viirya May 11, 2018
67f8701
Rename CodegenObjectFactory to CodeGeneratorWithInterpreted.
viirya May 13, 2018
e01fe97
Use public.
viirya May 14, 2018
de88f88
Change comment.
viirya May 14, 2018
f883c2b
Change mode name.
viirya May 16, 2018
716d88f
Address comments.
viirya May 17, 2018
8c56ae4
Simplify test suite.
viirya May 17, 2018
5a735af
Don't use hardcoded configs.
viirya May 18, 2018
5569646
Merge remote-tracking branch 'upstream/master' into SPARK-23711
viirya May 22, 2018
ed525f6
Simplify SQL conf getting.
viirya May 22, 2018
c9ec817
For comment.
viirya May 22, 2018
5527595
Task context should not have null properties.
viirya May 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Task context should not have null properties.
  • Loading branch information
@viirya
viirya committed May 22, 2018
commit 5527595b83b9cd394ba6d52e82711b777259f569
3 changes: 2 additions & 1 deletion sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeKVExternalSorterSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,8 @@ class UnsafeKVExternalSorterSuite extends SparkFunSuite with SharedSQLContext {
// Make sure we can successfully create a UnsafeKVExternalSorter with a `BytesToBytesMap`
// which has duplicated keys and the number of entries exceeds its capacity.
try {
TaskContext.setTaskContext(new TaskContextImpl(0, 0, 0, 0, 0, taskMemoryManager, null, null))
val context = new TaskContextImpl(0, 0, 0, 0, 0, taskMemoryManager, new Properties(), null)
TaskContext.setTaskContext(context)
new UnsafeKVExternalSorter(
schema,
schema,
Expand Down