Skip to content

Conversation

@gatorsmile
Copy link
Member

What changes were proposed in this pull request?

This PR is the follow-up PR of #28003

  • add a migration guide
  • add an end-to-end test case.

Why are the changes needed?

The original PR made the major behavior change in the user-facing RESET command.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added a new end-to-end test


- The `org.apache.spark.ExecutorPlugin` interface and related configuration has been replaced with
`org.apache.spark.plugin.SparkPlugin`, which adds new functionality. Plugins using the old
`org.apache.spark.api.plugin.SparkPlugin`, which adds new functionality. Plugins using the old
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a typo fix.

@SparkQA
Copy link

SparkQA commented Apr 19, 2020

Test build #121489 has finished for PR 28265 at commit c261896.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu
Copy link
Member

maropu commented Apr 20, 2020

Yea, the fix looks fine if the relevant test failure fixed.


assert(session.sessionState.conf.getConfString("spark.app.name") === "test-app-SPARK-31234")
assert(session.sessionState.conf.getConf(GLOBAL_TEMP_DATABASE) === "globalTempDB-SPARK-31234")
assert(session.sessionState.conf.getConf(GLOBAL_TEMP_DATABASE) === "globaltempdb-spark-31234")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This difference between Spark 2.4 and Spark 3.0 is caused by #24979

@gatorsmile
Copy link
Member Author

cc @cloud-fan

@cloud-fan
Copy link
Contributor

LGTM

@SparkQA
Copy link

SparkQA commented Apr 20, 2020

Test build #121501 has finished for PR 28265 at commit a63ad80.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu
Copy link
Member

maropu commented Apr 20, 2020

retest this please

@SparkQA
Copy link

SparkQA commented Apr 20, 2020

Test build #121506 has finished for PR 28265 at commit a63ad80.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

retest this please

@SparkQA
Copy link

SparkQA commented Apr 20, 2020

Test build #121529 has finished for PR 28265 at commit a63ad80.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

gatorsmile added a commit that referenced this pull request Apr 20, 2020
…QL Configuration

### What changes were proposed in this pull request?
This PR is the follow-up PR of #28003

- add a migration guide
- add an end-to-end test case.

### Why are the changes needed?
The original PR made the major behavior change in the user-facing RESET command.

### Does this PR introduce any user-facing change?
No

### How was this patch tested?
Added a new end-to-end test

Closes #28265 from gatorsmile/spark-31234followup.

Authored-by: gatorsmile <[email protected]>
Signed-off-by: gatorsmile <[email protected]>
(cherry picked from commit 6c792a7)
Signed-off-by: gatorsmile <[email protected]>
@gatorsmile
Copy link
Member Author

Thanks! Merged to master and 3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants