Skip to content

[SPARK-24332][SS][MESOS]Fix places reading 'spark.network.timeout' as milliseconds #21382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
zsxwing wants to merge 1 commit into from
Closed

[SPARK-24332][SS][MESOS]Fix places reading 'spark.network.timeout' as milliseconds #21382

zsxwing wants to merge 1 commit into from

Conversation

@zsxwing
Copy link
Member

@zsxwing zsxwing commented May 21, 2018

What changes were proposed in this pull request?

This PR replaces getTimeAsMs with getTimeAsSeconds to fix the issue that reading "spark.network.timeout" using a wrong time unit when the user doesn't specify a time out.

How was this patch tested?

Jenkins

@SparkQA
Copy link

SparkQA commented May 21, 2018

Test build #90908 has finished for PR 21382 at commit 145dd3b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@felixcheung
Copy link
Member

felixcheung commented May 22, 2018

@susanxhuynh

felixcheung
felixcheung approved these changes May 22, 2018
View reviewed changes
Copy link
Member

@felixcheung felixcheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there test for config?

@zsxwing
Copy link
Member Author

zsxwing commented May 22, 2018 •
edited
Loading

@felixcheung spark.network.timeout is one of the most weird configs as it may have different default values in different places. I think this is why it's not added to the global org.apache.spark.internal.config. This makes us hard to write a test for it.

@zsxwing
Copy link
Member Author

zsxwing commented May 24, 2018

Thanks! Merging to master.

@asfgit asfgit closed this in 53c06dd May 24, 2018
@zsxwing zsxwing deleted the fix-network-timeout-conf branch May 24, 2018 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felixcheung felixcheung felixcheung approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zsxwing @SparkQA @felixcheung