Skip to content

[SPARK-47777] fix python streaming data source connect test #46906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
chaoqin-li1123 wants to merge 3 commits into from
Closed

[SPARK-47777] fix python streaming data source connect test #46906

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
94f6248
[SPARK-47777][PYTHON][SS][TESTS] Add spark connect test for python st…
chaoqin-li1123 Apr 11, 2024
f0c62d6
Merge branch 'master' of github.com:apache/spark into fix_include
chaoqin-li1123 Jun 10, 2024
720a3a2
fix
chaoqin-li1123 Jun 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fix
  • Loading branch information
@chaoqin-li1123
chaoqin-li1123 committed Jun 10, 2024
commit 720a3a29ff02fe96403a044b3691add7bab787fa
3 changes: 1 addition & 2 deletions python/pyspark/sql/streaming/python_streaming_source_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
from typing import IO, Iterator, Tuple

from pyspark.accumulators import _accumulatorRegistry
from pyspark.java_gateway import local_connect_and_auth
from pyspark.errors import IllegalArgumentException, PySparkAssertionError, PySparkRuntimeError
from pyspark.serializers import (
read_int,
Expand All @@ -37,7 +36,7 @@
StructType,
)
from pyspark.sql.worker.plan_data_source_read import records_to_arrow_batches
from pyspark.util import handle_worker_exception
from pyspark.util import handle_worker_exception, local_connect_and_auth
from pyspark.worker_util import (
check_python_version,
read_command,
Expand Down