[RELEASE] v2.0.0-beta9

[jarohen]

A couple of breaking changes to tell you about here, but first: COPY!

COPY

XTDB now supports Postgres's COPY functionality, adding the ability to quickly load large quantities of data into XT.

COPY my_table FROM STDIN WITH (FORMAT = 'transit-json')

• If you're using JDBC, you'll want to check out the CopyManager documentation, accessible through the XTDB connection using getCopyAPI.
  • If you're using Clojure, and XT's JDBC wrapper, you can use xtdb.next.jdbc/copy-in
• If you're using Postgres.js, check out its copy API

XTDB supports Transit initially - either 'transit-json' or 'transit-msgpack'. We're looking to add support for Arrow too - this will reduce the (de-)serialisation required in transaction submission to near-zero.

Clojure API implementation

Previously, the XTDB Clojure API used an internal API to submit transactions and run queries. This worked well, but it meant that there were two code-paths - one that you'd use on an in-process node, and one when you graduated to a remote node - opening up the possibility for differences between the two.

We now implement the Clojure API as a client of the Postgres wire protocol, so that you're running the same code regardless of whether your node is local or remote. As a result, we've also been able to remove some of the abstractions that were previously required because of the two code-paths.

There are two related breaking changes to tell you about:

• The Clojure HTTP client is no longer available - we'd encourage users to expose the Postgres wire-server port instead and connect via xt/client. (The server is still available, for users connecting directly via HTTP.)
• xt/execute-tx will now throw an exception if the transaction fails to commit, in line with both JDBC and next.jdbc.

Elsewhere:

A couple of smaller changes to draw your attention to:

• XT now has xt.metrics_gauges, xt.metrics_timers and xt.metrics_counters tables containing various monitoring metrics for the node. In a production system, you'll most likely still want to set up Prometheus with your choice of dashboard - but, at a pinch, these tables contain the raw metrics.
• (breaking) We now send java.util.Dates over pgwire as timestamps-with-tz. This is a deviation from JDBC tradition, which treats them as timestamps-without-tz.
• XT now uses a lesser-known Cognitect library called 'anomalies' to return more structured errors - you should see this either as ex-data from the xtdb.api functions, or in the 'detail' messages in your Postgres error responses.

As always, for a full list of closed cards, see the milestone.

Cheers!

James & the XT team

---

This discussion was created from the release v2.0.0-beta9.