Commit 58a4c0c
committed
[SPARK-26080][PYTHON] Skips Python resource limit on Windows in Python worker
## What changes were proposed in this pull request?
`resource` package is a Unix specific package. See https://docs.python.org/2/library/resource.html and https://docs.python.org/3/library/resource.html.
Note that we document Windows support:
> Spark runs on both Windows and UNIX-like systems (e.g. Linux, Mac OS).
This should be backported into branch-2.4 to restore Windows support in Spark 2.4.1.
## How was this patch tested?
Manually mocking the changed logics.
Closes #23055 from HyukjinKwon/SPARK-26080.
Lead-authored-by: hyukjinkwon <gurwls223@apache.org>
Co-authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit 9cda9a8)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>1 parent 3ec03ec commit 58a4c0c
2 files changed
+14
-7
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
190 | 190 | | |
191 | 191 | | |
192 | 192 | | |
| 193 | + | |
| 194 | + | |
193 | 195 | | |
194 | 196 | | |
195 | 197 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
26 | 31 | | |
27 | 32 | | |
28 | 33 | | |
| |||
268 | 273 | | |
269 | 274 | | |
270 | 275 | | |
271 | | - | |
272 | | - | |
273 | | - | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
274 | 279 | | |
275 | 280 | | |
276 | 281 | | |
| |||
283 | 288 | | |
284 | 289 | | |
285 | 290 | | |
286 | | - | |
287 | | - | |
288 | | - | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
289 | 294 | | |
290 | 295 | | |
291 | 296 | | |
| |||
0 commit comments