Skip to content

[SPARK-32121][SHUFFLE] Support Windows OS in ExecutorDiskUtils #28940

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pan3793 wants to merge 10 commits into from
Closed

[SPARK-32121][SHUFFLE] Support Windows OS in ExecutorDiskUtils #28940

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
e8ea16a
[SPARK-32121][SHUFFLE][TEST] Fix ExternalShuffleBlockResolverSuite fa…
pan3793 Jun 28, 2020
187e813
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] fix checkstyle
pan3793 Jun 28, 2020
9b24a14
[SPARK-32121][SHUFFLE][TEST] use different MULTIPLE_SEPARATORS in Win…
pan3793 Jun 29, 2020
8cd8bd1
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] fix checkstyle
pan3793 Jun 29, 2020
06024f4
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] fix checkstyle
pan3793 Jun 29, 2020
6a201a5
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] simplify code by commons-lang3
pan3793 Jun 29, 2020
50a742d
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] improve comments
pan3793 Jun 29, 2020
56f3be5
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] minor refactor
pan3793 Jun 30, 2020
aac01ca
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] code style
pan3793 Jun 30, 2020
a80bd6c
[SPARK-32121][SHUFFLE] improve comments
pan3793 Jul 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[FOLLOWUP][SPARK-32121][SHUFFLE][TEST] code style
  • Loading branch information
@pan3793
pan3793 committed Jun 30, 2020
commit aac01ca11c3c024e8a75753e43a217cadb1d8c46
16 changes: 7 additions & 9 deletions ...fle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,19 +148,17 @@ public void jsonSerializationOfExecutorRegistration() throws IOException {
@Test
public void testNormalizeAndInternPathname() {
String sep = File.separator;
String expectedPathname1 = sep + "foo" + sep + "bar" + sep + "baz";
String expectedPathname2 = "foo" + sep + "bar" + sep + "baz";
String expectedPathname3 = sep + "foo\\" + sep + "bar" + sep + "baz";
assertPathsMatch("/foo", "bar", "baz", expectedPathname1);
assertPathsMatch("//foo/", "bar/", "//baz", expectedPathname1);
assertPathsMatch("foo", "bar", "baz///", expectedPathname2);
assertPathsMatch("/foo/", "/bar//", "/baz", expectedPathname1);
String expectedPathname = sep + "foo" + sep + "bar" + sep + "baz";
assertPathsMatch("/foo", "bar", "baz", expectedPathname);
assertPathsMatch("//foo/", "bar/", "//baz", expectedPathname);
assertPathsMatch("/foo/", "/bar//", "/baz", expectedPathname);
assertPathsMatch("foo", "bar", "baz///", "foo" + sep + "bar" + sep + "baz");
assertPathsMatch("/", "", "", sep);
assertPathsMatch("/", "/", "/", sep);
if (SystemUtils.IS_OS_WINDOWS) {
assertPathsMatch("/foo\\/", "bar", "baz", expectedPathname1);
assertPathsMatch("/foo\\/", "bar", "baz", expectedPathname);
} else {
assertPathsMatch("/foo\\/", "bar", "baz", expectedPathname3);
assertPathsMatch("/foo\\/", "bar", "baz", sep + "foo\\" + sep + "bar" + sep + "baz");
}
}

Expand Down