Skip to content

Conversation

@teutonet
Copy link
Contributor

Add pod anti affinity to enforce that postgres pods does not spawn on the same k8s node.

@coveralls
Copy link

coveralls commented Feb 15, 2019

Coverage Status

Coverage remained the same at 23.705% when pulling 986d3ab on teutostack:pr_pod_affinity into 2e9b653 on zalando-incubator:master.

@redbaron
Copy link
Contributor

related #479

@redbaron
Copy link
Contributor

In some cases it is OK to have 2 PG pods on the same node, for instance when testing switchover/failover

@teutonet
Copy link
Contributor Author

@redbaron You're right with #479, but it is a big change and it will take some time to get it done. Are there any timelines regarding this issue?

For production DBs it would be a worst case scenario to have 2 PG pods on the same node, once the node dies.

@Jan-M
Copy link
Member

Jan-M commented Feb 15, 2019

This def. needs a toggle, maybe incl. a toggle per cluster.

While initially this may sound bad for availability it all depends.

In our K8S infrastructure on AWS this is also largely not happening for a single Postgres cluster as pods are bound to different AZs via the different EBS volumes.

But if your storage is EBS it may sometimes in a single AZ not matter so much if pods are on the same host.

Regarding #479 I am looking forward to ideas. In our initial impl. in Python this was somehow neater having the statefulset spec just injected as a String template.

@teutonet
Copy link
Contributor Author

@Jan-M I think it's a good idea to have a switch in the cluster config to enable pod anti affinity and a addtional property to set the pod affinity TopologyKey, default could be kubernetes.io/hostname.

 * use `cluster_labels` for pod anti affinity
 * use `node_readiness_label` when defined
 * added property `enable_pod_antiaffinity` to operator config
 * added property `pod_antiaffinity_topology_key`
@teutonet
Copy link
Contributor Author

@Jan-M Switch for pod anti affinity is implemented as well as the TopologyKey.

Copy link
Member

@sdudoladov sdudoladov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Could you also update the admin docs and the operator params' reference to mention the new option ?

  2. pls also propagate the option to/from the CRD-based operator configuration

 * use `cluster_labels` for pod anti affinity
 * use `node_readiness_label` when defined
 * added property `enable_pod_antiaffinity` to operator config
 * added property `pod_antiaffinity_topology_key`
@sdudoladov sdudoladov merged commit 26a7fdf into zalando:master Feb 21, 2019
@sdudoladov
Copy link
Member

lgtm. thank you, @ocaner-biz

@FxKu FxKu added this to the v1.2 milestone May 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants