Adding Pulsar transport

[chiluap]

Hello there,

I started adding Apache Pulsar Transport based on
https://github.com/fsharplang-ru/pulsar-client-dotnet
using the Apache Kafka Transport as base.

Being quite new to this, I got Publisher and Subscriber working, I expect the integration to be far from perfect =).

If someone has the time to give me some hints, my efforts can be found at:
https://github.com/chiluap/CAP/tree/pulsar-transport

Looking forward to a pull-reguest adding Pulsar to CAP's CAPabilities.

Thanks for reading

c.

[yang-xiaodong]

Hello @chiluap ,

I have no idea about Apache Pulsar, I need some time to learn. Is Pulsar popular?

[chiluap]

Pulsar is gaining traction.
Google-Trends
oreilly-Survey
We settled on Pulsar after comparing it against Kafka and NATS.

[chiluap]

Hi @yang-xiaodong ,

If you are interested, docs can be found here

And here's my docker-compose for local testing.
(don't forget to create the data dir =))

version: '2'

services:
  pulsar:
    image: apachepulsar/pulsar:latest
    environment:
      TZ: Europe/Berlin
    ports:
      - 6650:6650
      - 8080:8080
    volumes:
        - ./data:/pulsar/data
    command: ['bin/pulsar', 'standalone']
  pulsar-dashboard:
    image: bbonnin/pulsar-express
    environment:
      PE_CONNECTION_URL: http://pulsar:8080
    ports:
        - 3000:3000

[jeffdoolittle]

Kafka isn't really a message bus or broker. It's a distributed commit log with consumer indexing.

Pulsar offers standard bus/broker capabilities and can function as a distributed commit log with consumer indexing.

[chiluap]

@jeffdoolittle Thank you for the clarification.
For us Pulsar seems like the better fit being more flexible with automatic re-balancing, instant subscriptions (reader), and separation between broker and storage for scaling.

[rs-infg]

As part of the 2.8 release, Pulsar has evolved the exactly-once semantic from guaranteed message deduplication on a single topic to atomic produce and acknowledgment over multiple topics via Transaction API.

Read more here: https://streamnative.io/en/blog/release/2021-06-14-exactly-once-semantics-with-transactions-in-pulsar

[paulvanbladel]

As part of the 2.8 release, Pulsar has evolved the exactly-once semantic from guaranteed message deduplication on a single topic to atomic produce and acknowledgment over multiple topics via Transaction API.

Read more here: https://streamnative.io/en/blog/release/2021-06-14-exactly-once-semantics-with-transactions-in-pulsar

Intresting. So does this mean outbox pattern is not necessary any longer for pulsar?
Thx

[yang-xiaodong]

@paulvanbladel This article describes the features of Pulsar's own Transaction API and transaction support for Exactly-Once, which is different from ADO.NET's transaction.