What makes this different from the "official" kubernetes/contrib/statefulsets/kafka?

[coco98]

Could you add sections to the README about what makes this different from https://github.com/kubernetes/contrib/tree/master/statefulsets/kafka?

Filing this issue in case the maintainers already have some points in mind.
I'm planning to take both for a spin and I can contribute once I have my thoughts in place too!

[solsson]

I will try to add such a section, "Alternatives" maybe.

There's also https://github.com/kubernetes/charts/tree/master/incubator/kafka

I hope this repo will be a community where small organizations (who can't have dedicated Kafka ops people or consultants) can come together. A folder inside some contrib/charts repo is a poor start for such a community. I recommend that after you've taken this for a spin, if you decide to go for a kafka setup like this, you fork the repo (private or public) and have your customizations there. With a the other templates you would start from a copy, but have no use of the upstream repo's changes. We've seen valuable contributions here recently, like the discussion in #13.

I'm very interested to hear your thoughts after the initial setups.

[arthurk]

@solsson I just wanted to say thanks for creating and maintaining this Kubernetes files, they're up-to-date (unlike the kubernetes contrib files, don't require helm and work great!

[solsson]

Thanks a lot @arthurk!

I think there is an important discussion to be had in this repo: on the level of automation. Kubernetes could be used to implement the kind of tooling that I imagine every large kafka ops team has. The "Operator" pattern would fit very nicely for automated Kafka cluster health. Built-in supervision like readiness and service discovery might solve things that often overlap features built in to other clustered backends.

Now when working with automation and monitoring I've re-read parts of the Kafka Book from this perspective. For example #95 e6a7aec is interesting as it tries to capture some of that knowledge.