[SPARK-34065][INFRA] Cancel the duplicated jobs only in PRs at GitHub Actions

[HyukjinKwon]

What changes were proposed in this pull request?

This is kind of a followup of #31104 but I decided to track it separately with a separate JIRA.

Currently the jobs are being canceled in main repo branches. If a commit is merged, for example, to master branch before the test finishes, it cancels the previous builds. This is a problem because we cannot, for example, detect logical conflict properly. We should only cancel the jobs in PRs:

This PR proposes to don't do this in the main repo branch commits but only do it in PRs.

Why are the changes needed?

• To keep the test coverage
• To run the test in the synced master branch instead of relying on the builds made in each PR with an outdated master branch
• To detect test failures from logical conflicts from merging two conflicting PRs at the same time.

Does this PR introduce any user-facing change?

No, dev-only.

How was this patch tested?

I manually tested in

• Cancel jobs only in PRs HyukjinKwon/spark#27
• Cancel jobs only in PRs HyukjinKwon/spark#28

I added Yi Wu as a co-author since he helped verifying the current fix in the PR above.

I checked that it does not cancel in the main repo branch:

I checked it cancels in PRs:

[HyukjinKwon]

cc @mik-laj and @dongjoon-hyun FYI

[dongjoon-hyun]

Nice!

[HyukjinKwon]

@dongjoon-hyun, I referred to https://securitylab.github.com/research/github-actions-preventing-pwn-requests and found this way is slightly better because it's at least easier to understand. Likewise, I tested it in my fork, and updated PR description.

[HyukjinKwon]

Let me merge this in to recover the test coverage. Let's see if it works out of the box in the main repo too.

[HyukjinKwon]

Merged to master.

[HyukjinKwon]

Seems working fine:

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38498/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38498/

[SparkQA]

Test build #133909 has finished for PR 31121 at commit 31cd9f7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #133911 has finished for PR 31121 at commit 4b32e9f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.