Skip to content

Conversation

@theory
Copy link
Contributor

@theory theory commented Nov 20, 2025

It converts the internal integer to a decimal string relative to the scale and precision. Not a blocker for my projects (I copied the code), but I am getting an error when using -D BUILD_SHARED_LIBS:

undefined symbol: _ZN4absllsERSoNS_6int128E

Presumably -D BUILD_SHARED_LIBS needs to also build and install the absl library.

@theory theory marked this pull request as ready for review November 20, 2025 21:08
theory added a commit to ClickHouse/pg_clickhouse that referenced this pull request Nov 20, 2025
The mappings already work for the HTTP engine, although its inserts were
limited to `float8`, now fixed to use proper numeric formatting.

Add the mappings for the binary engine. Use strings to convert between
the Postgres and ClickHouse formats to as to avoid differences in their
implementations: ClickHouse stores data as `Int32`, `Int64`, or
`Int128`, while Postgres...well, I have no idea what the internals are,
but using the `numeric_in` and `numeric_out` functions avoid all issues.

Conversion from `Decimal` values returned from ClickHouse requires
converting its internal `Int128` into a string. The clickhouse-cpp
library doesn't provide this functionality, [yet], so implement it here.

Unfortunately, this approach doesn't work with clickhouse-cpp compiled
into a dynamic library, presumably because it doesn't also compile and
install the `absl` dynamic library. So make the build static by default
on all platforms for now.

Add a test that covers both the HTTP and binary engines.

  [yet]: ClickHouse/clickhouse-cpp#451
@mshustov mshustov requested review from Copilot and slabko November 24, 2025 09:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a GetString method to the ColumnDecimal class that converts internal integer representations to human-readable decimal strings with proper scale and precision formatting.

Key Changes:

  • Added GetString method to ColumnDecimal class that formats decimal values as strings
  • Added comprehensive test coverage for the new method across various decimal types and edge cases

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
clickhouse/columns/decimal.h Declares the new GetString method with documentation
clickhouse/columns/decimal.cpp Implements decimal-to-string conversion logic with proper handling of scale, precision, and negative values
ut/client_ut.cpp Adds extensive test cases validating the string conversion for positive, negative, and edge-case decimal values

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

It converts the internal integer to a decimal string relative to the
scale and precision.
@slabko
Copy link
Contributor

slabko commented Dec 1, 2025

I’m a bit on the fence about that. There’s a problem with this approach -- it’s five times slower than using SELECT toString(...) FROM ... and getting the same value. However, as soon as there is a GetString(...) function, it will become everyone’s default choice.

@theory
Copy link
Contributor Author

theory commented Dec 1, 2025

I’m a bit on the fence about that. There’s a problem with this approach -- it’s five times slower than using SELECT toString(...) FROM ... and getting the same value. However, as soon as there is a GetString(...) function, it will become everyone’s default choice.

I'll look into whether I can use that syntax in pg_clickhouse, but ISTM that including stringification makes a more complete API.

Would it be more efficient to use a template to treat UIint32, UInt64, and UInt128 separately?

@theory
Copy link
Contributor Author

theory commented Dec 5, 2025

I spent some time trying to use toString() in the queries built by pg_clickhouse to convert to strings on the back end. I got it working nicely for Postgres NUMERIC columns. For example, for this Postgres query:

SELECT * FROM dec_http.decimals ORDER BY id;

pg_clickhouse would send it as:

SELECT id, toString("dec"), toString(dec32), toString(dec64), toString(dec128)
FROM decimal_test.decimals ORDER BY id ASC NULLS LAST

But alas, the code is unaware of the data type of SQL expressions. So this Postgres query:

SELECT avg(dec) FROM dec_http.decimals;

Gets converted to:

SELECT avg(toString("dec")) FROM decimal_test.decimals

Which is obviously incorrect. It's far better to just get the Decimal value from a Block and to convert it to a string, as all the evaluation is done and ClickHouse has returned the proper value for the overall expression.

I assume I could figure out how to fiddle with the Postgres query AST enough at some point to work out where and how to do the casting, but it would take quite a bit of effort. At this point, I think it's simpler to just rely on what ClickHouse sends back. And for better or worse, to convert a Decimal to a Postgres NUMERIC, I need a string.

Copy link
Contributor

@slabko slabko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted earlier, if we introduce such a function, its performance should be on par with ClickHouse’s toString; a convenience API should not be several times slower than the next obvious alternative.

A more serious issue, however, is that this PR breaks shared-library builds. Running:

cmake -S . -B build -D BUILD_SHARED_LIBS=YES
cmake --build build

results in:

/usr/bin/ld: ../contrib/absl/absl/libabsl_int128.a(int128.cc.o): relocation R_X86_64_32S against symbol `_ZTVSt9basic_iosIcSt11char_traitsIcEE@@GLIBCXX_3.4' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status

@theory
Copy link
Contributor Author

theory commented Dec 9, 2025

A more serious issue, however, is that this PR breaks shared-library builds

Does -fPIC help?

@slabko slabko marked this pull request as draft December 9, 2025 17:29
@slabko
Copy link
Contributor

slabko commented Dec 10, 2025

The lack of -fPIC is the reason it doesn’t compile. absl needs to be compiled with -fPIC in order to use operator<<. So yes, -fPIC does help.
See #454

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants