[SPARK-34979][PYTHON][DOC] Add PyArrow installation note for PySpark aarch64 user #32363
Conversation
|
Test build #137995 has finished for PR 32363 at commit
|
|
Kubernetes integration test unable to build dist. exiting with code: 1 |
| If using JDK 11, set ``-Dio.netty.tryReflectionSetAccessible=true`` for Arrow related features and refer | ||
| to |downloading|_. | ||
|
|
||
| Note for AArch64(Arm64) user: The PyArrow is required by PySpark SQL, but the |
There was a problem hiding this comment.
Let's rewrite this to fix a number of typos:
Note for AArch64 (ARM64) users: PyArrow is required by PySpark SQL, but PyArrow support for AArch64 is introduced in PyArrow 4.0.0. If PySpark installation fails on AArch64 due to PyArrow installation errors, you can install PyArrow >= 4.0.0 as below:
There was a problem hiding this comment.
done, Thanks for your help. : )
|
Test build #138003 has finished for PR 32363 at commit
|
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Merged to master. |
4 participants
What changes were proposed in this pull request?
This patch adds a note for aarch64 user to install the specific pyarrow>=4.0.0.
Why are the changes needed?
The pyarrow aarch64 support is introduced in PyArrow 4.0.0, and it has been published 27.Apr.2021.
See more in SPARK-34979.
Does this PR introduce any user-facing change?
Yes, this doc can help user install arrow on aarch64.
How was this patch tested?
doc test passed.