Skip to content

Change the instance name for standard pod scraping to be unique #261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
beorn7 merged 2 commits into from
May 18, 2020
Merged

Change the instance name for standard pod scraping to be unique #261

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
26402f2
Change the instance name for standard pod scraping to be unique
beorn7 May 15, 2020
0ccca40
Add a `container` and `pod` target label
beorn7 May 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Add a container and pod target label 
This allows joining with cAdvisor metrics.
  • Loading branch information
@beorn7
beorn7 committed May 15, 2020
commit 0ccca409a20a56d209ecfa1fc6a8ba76bef89798
23 changes: 18 additions & 5 deletions prometheus-ksonnet/lib/prometheus-config.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,23 @@
replacement: '$1',
},

// But also include the namespace as a separate label, for routing alerts
// But also include the namespace, container, pod as separate labels,
// for routing alerts and joining with cAdvisor metrics.
{
source_labels: ['__meta_kubernetes_namespace'],
action: 'replace',
target_label: 'namespace',
},
{
source_labels: ['__meta_kubernetes_pod_name'],
action: 'replace',
target_label: 'pod', // Not 'pod_name', which disappeared in K8s 1.16.
},
{
source_labels: ['__meta_kubernetes_container_name'],
action: 'replace',
target_label: 'container', // Not 'container_name', which disappeared in K8s 1.16.
},

// Rename instances to the concatenation of pod:container:port.
// All three components are needed to guarantee a unique instance label.
Expand Down Expand Up @@ -169,10 +180,12 @@
],
},

// A separate scrape config for kube-state-metrics which doesn't add a namespace
// label, instead taking the namespace label from the exported timeseries. This
// prevents the exported namespace label being renamed to exported_namespace, and
// allows us to route alerts based on namespace.
// A separate scrape config for kube-state-metrics which doesn't
// add namespace, container, and pod labels, instead taking
// those labels from the exported timeseries. This prevents them
// being renamed to exported_namespace etc. and allows us to
// route alerts based on namespace and join KSM metrics with
// cAdvisor metrics.
{
job_name: '%s/kube-state-metrics' % $._config.namespace,
kubernetes_sd_configs: [{
Expand Down