Skip to content

Conversation

@itholic
Copy link
Contributor

@itholic itholic commented Nov 29, 2024

What changes were proposed in this pull request?

This PR proposes to support Interrupt(Tag|All) for PySpark

Why are the changes needed?

To improve the compatibility between Spark Connect and Spark Classic.

Does this PR introduce any user-facing change?

New APIs are added

  • InterruptTag
  • InterruptAll

How was this patch tested?

Added UTs

Was this patch authored or co-authored using generative AI tooling?

No

@itholic
Copy link
Contributor Author

itholic commented Jan 3, 2025

Sorry I accidentally deleted comment from @xupefei while resolving some complex conflict that occurred in my development environment. Is there any way to restore it?

(The comment was basically a side note for the #48906 and I aware of it, so it's not a big problem, though)

@itholic itholic marked this pull request as ready for review January 3, 2025 07:49
@itholic itholic changed the title [WIP][SPARK-50357][PYTHON] Support Interrupt(Tag|All) APIs for PySpark [SPARK-50357][PYTHON] Support Interrupt(Tag|All) APIs for PySpark Jan 3, 2025
@itholic
Copy link
Contributor Author

itholic commented Jan 3, 2025

I think it is ready for the review now. cc @HyukjinKwon and also cc @xupefei FYI

@github-actions github-actions bot added the DOCS label Jan 6, 2025
@itholic itholic closed this in 459483a Jan 6, 2025
@itholic
Copy link
Contributor Author

itholic commented Jan 6, 2025

Merged to master. Thanks @HyukjinKwon and @xupefei for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants