Skip to content

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Jun 23, 2020

What changes were proposed in this pull request?

Spark 3.0 accidentally dropped R < 3.5. It is built by R 3.6.3 which not support R < 3.5:

Error in readRDS(pfile) : cannot read workspace version 3 written by R 3.6.3; need R 3.5.0 or newer version.

In fact, with SPARK-31918, we will have to drop R < 3.5 entirely to support R 4.0.0. This is inevitable to release on CRAN because they require to make the tests pass with the latest R.

Why are the changes needed?

To show the supported versions correctly, and support R 4.0.0 to unblock the releases.

Does this PR introduce any user-facing change?

In fact, no because Spark 3.0.0 already does not work with R < 3.5.
Compared to Spark 2.4, yes. R < 3.5 would not work.

How was this patch tested?

Jenkins should test it out.

@HyukjinKwon
Copy link
Member Author

Jenkins R's 3.4 IIRC. I will see the test results first, and let the appropriate person know and update the env.

@SparkQA
Copy link

SparkQA commented Jun 23, 2020

Test build #124421 has finished for PR 28908 at commit d48ecf9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@shivaram
Copy link
Contributor

LGTM. Thanks @HyukjinKwon

@dongjoon-hyun
Copy link
Member

Thanks, @HyukjinKwon . BTW, if we accidentally dropped it, how can we mention it in this case? This PR should target Apache Spark 3.0.1 with releasenotes label? (For now, it's just a normal JIRA issue.)
cc @gatorsmile and @srowen

@viirya
Copy link
Member

viirya commented Jun 23, 2020

As we accidentally dropped R < 3.5 in Spark 3.0.0, can we update the doc like https://spark.apache.org/docs/latest/index.html to make it less confusing?

@HyukjinKwon
Copy link
Member Author

@viirya, it will be ported back to branch-3.0 and 3.0.1 will contains the doc fixes there. It should be fine with 3.0.1.

@HyukjinKwon
Copy link
Member Author

Sure, I added releasenotes label in the JIRA.

@HyukjinKwon
Copy link
Member Author

Oh, just as a note to my self, Jenkins R version seems to be 3.5.2:

using R version 3.5.2 (2018-12-20)

Nice.

HyukjinKwon added a commit that referenced this pull request Jun 24, 2020
### What changes were proposed in this pull request?

Spark 3.0 accidentally dropped R < 3.5. It is built by R 3.6.3 which not support R < 3.5:

```
Error in readRDS(pfile) : cannot read workspace version 3 written by R 3.6.3; need R 3.5.0 or newer version.
```

In fact, with SPARK-31918, we will have to drop R < 3.5 entirely to support R 4.0.0. This is inevitable to release on CRAN because they require to make the tests pass with the latest R.

### Why are the changes needed?

To show the supported versions correctly, and support R 4.0.0 to unblock the releases.

### Does this PR introduce _any_ user-facing change?

In fact, no because Spark 3.0.0 already does not work with R < 3.5.
Compared to Spark 2.4, yes. R < 3.5 would not work.

### How was this patch tested?

Jenkins should test it out.

Closes #28908 from HyukjinKwon/SPARK-32073.

Authored-by: HyukjinKwon <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit b62e253)
Signed-off-by: HyukjinKwon <[email protected]>
HyukjinKwon added a commit that referenced this pull request Jun 24, 2020
Spark 3.0 accidentally dropped R < 3.5. It is built by R 3.6.3 which not support R < 3.5:

```
Error in readRDS(pfile) : cannot read workspace version 3 written by R 3.6.3; need R 3.5.0 or newer version.
```

In fact, with SPARK-31918, we will have to drop R < 3.5 entirely to support R 4.0.0. This is inevitable to release on CRAN because they require to make the tests pass with the latest R.

To show the supported versions correctly, and support R 4.0.0 to unblock the releases.

In fact, no because Spark 3.0.0 already does not work with R < 3.5.
Compared to Spark 2.4, yes. R < 3.5 would not work.

Jenkins should test it out.

Closes #28908 from HyukjinKwon/SPARK-32073.

Authored-by: HyukjinKwon <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit b62e253)
Signed-off-by: HyukjinKwon <[email protected]>
@HyukjinKwon
Copy link
Member Author

Merged to master, branch-3.0 and branch-2.4.

@shivaram
Copy link
Contributor

Oh, just as a note to my self, Jenkins R version seems to be 3.5.2:

using R version 3.5.2 (2018-12-20)

Nice.

Hah - we should file a JIRA to upgrade it to 4.0 or at least 3.6.3?

@HyukjinKwon
Copy link
Member Author

I think so .. I will at least target to test 4.0 in AppVeyor at #28909 for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants