-
Notifications
You must be signed in to change notification settings - Fork 2.1k
[Fix][Connector-V2][Postgres-CDC] Fix PostgreSQL GEOMETRY handling with JDBC sink #10186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
|
Add unit tests for the Geography type |
|
Please update the document |
...ache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/psql/PostgresJdbcRowConverter.java
Outdated
Show resolved
Hide resolved
…andling and add tests
| if (PG_INET.equalsIgnoreCase(sourceType) | ||
| if (sourceType != null | ||
| && (PG_GEOMETRY.equalsIgnoreCase(sourceType) | ||
| || PG_GEOGRAPHY.equalsIgnoreCase(sourceType))) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zhangshenghang
The sourceType here comes from source. Are you sure there is no problem? The logic code here only applies to pgsql-cdc ->pgsql. The sourceType should come from sink to apply to all sources. Please take a look at this PR: #10076
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing this out, the original implementation was indeed taking sourceType from the source tableSchema, so the special handling mainly worked for the pgsql-cdc -> pgsql case.
I’ve updated PostgresJdbcRowConverter.toExternal to resolve the type from the sink first:
- use
databaseTableSchema(sink schema) by column name when it’s available - fall back to the original
tableSchemasourceType array only for backward compatibility
Purpose of this pull request
Fix issue #10155 where PostgreSQL-CDC → JDBC sink failed on GEOMETRY columns because Debezium Geometry/Geography values were serialized as Struct{wkb=...,srid=...} strings and then written as VARCHAR into geometry fields. This PR converts Debezium geometry to hex WKB strings and writes them using PGobject, and adds unit/e2e tests for the full path.
Does this PR introduce any user-facing change?
Yes. For PostgreSQL-CDC + JDBC sink jobs, GEOMETRY/GEOGRAPHY columns can now be synchronized correctly without type mismatch errors. There is no configuration change; behavior is fixed for existing pipelines that previously failed on geometry types.
How was this patch tested?
Added/updated unit tests:
SeaTunnelRowDebeziumDeserializationConvertersTest to verify Debezium Geometry is converted to \x... hex WKB string.
PostgresJdbcRowConverterTest to verify sourceType=geometry fields are written via PGobject("geometry", value) instead of setString.
Extended e2e tests:
Check list
New License Guide
incompatible-changes.mdto describe the incompatibility caused by this PR.