Implement kubernetes caching backend

[jkroepke]

Fixes #3

Tested with azure-resourcemanager-exporter, using k8scm://opsstack/azure-resourcemanager-exporter-cache/

Kubernetes v1.22 is required as minimum here, since I'm using Server-Side-Apply to managed the data fields separately to resolve any concurrency issues.

Logs:

2023-05-16T16:51:27.523Z        info    collector/cache.go:177  restoring state from cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/resourceHealth.json   {"collector": "resourceHealth"}
2023-05-16T16:51:27.524Z        info    collector/cache.go:177  restoring state from cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/quota.json    {"collector": "quota"}
2023-05-16T16:51:27.524Z        info    collector/cache.go:177  restoring state from cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/general.json  {"collector": "general"}
2023-05-16T16:51:27.527Z        info    collector/cache.go:215  restored state from cache: "k8scm://opsstack/azure-resourcemanager-exporter-cache/general.json" (expiring 2023-05-16 16:51:46.131671796 +0000 UTC)        {"collector": "general"}
2023-05-16T16:51:27.527Z        info    collector/cache.go:215  restored state from cache: "k8scm://opsstack/azure-resourcemanager-exporter-cache/quota.json" (expiring 2023-05-16 16:51:50.984695132 +0000 UTC)  {"collector": "quota"}
2023-05-16T16:51:27.528Z        info    collector/cache.go:215  restored state from cache: "k8scm://opsstack/azure-resourcemanager-exporter-cache/resourceHealth.json" (expiring 2023-05-16 16:51:47.856535755 +0000 UTC) {"collector": "resourceHealth"}
2023-05-16T16:51:27.528Z        info    collector/cache.go:177  restoring state from cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/resource.json {"collector": "resource"}
2023-05-16T16:51:27.535Z        info    collector/cache.go:215  restored state from cache: "k8scm://opsstack/azure-resourcemanager-exporter-cache/resource.json" (expiring 2023-05-16 16:51:46.646073796 +0000 UTC)       {"collector": "resource"}
2023-05-16T16:52:27.591Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/general.json (expiring 2023-05-16 16:53:27.532214593 +0000 UTC)       {"collector": "general"}
2023-05-16T16:52:28.017Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/resource.json (expiring 2023-05-16 16:53:27.968933969 +0000 UTC)      {"collector": "resource"}
2023-05-16T16:52:30.334Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/resourceHealth.json (expiring 2023-05-16 16:53:30.188369511 +0000 UTC)        {"collector": "resourceHealth"}
2023-05-16T16:52:32.828Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/quota.json (expiring 2023-05-16 16:53:32.816557471 +0000 UTC){"collector": "quota"}
2023-05-16T16:53:27.674Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/general.json (expiring 2023-05-16 16:54:27.600089385 +0000 UTC)       {"collector": "general"}
2023-05-16T16:53:28.437Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/resource.json (expiring 2023-05-16 16:54:28.376659594 +0000 UTC)      {"collector": "resource"}
2023-05-16T16:53:33.319Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/resourceHealth.json (expiring 2023-05-16 16:54:33.269111721 +0000 UTC)        {"collector": "resourceHealth"}
2023-05-16T16:53:33.363Z        info    collector/cache.go:241  saved state to cache: k8scm://opsstack/azure-resourcemanager-exporter-cache/quota.json (expiring 2023-05-16 16:54:33.324268304 +0000 UTC){"collector": "quota"}

The Kubernetes RBAC role can be restricted to that ConfigMap only and doesn't need higher permissions

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: azure-resourcemanager-exporter
rules:
- apiGroups:
  - ""
  resourceNames:
  - azure-resourcemanager-exporter-cache
  resources:
  - configmaps
  verbs:
  - get
  - patch
  - create
  - update
  - delete

Limitation: The total amount of "cache" is limited to 1MB. The data will be stored as gzip compressed string.

[jkroepke]

@mblaschke this is ready to review, i tested this on a local build with the latest azure-resourcemanager-exporter-cache from main.

I tested this with the data from 4 different subscriptions with this configuration:

---
azure:
  subscriptions:
        - ""
        - ""
        - ""
        - ""
  locations: []

  resourceTags: []
  resourceGroupTags: []

collectors:
  general:
    scrapeTime: 1m

  resource: 
    scrapeTime: 1h

  quota:
    scrapeTime: 1h

  defender:
    scrapeTime: 1h

  resourceHealth:
    scrapeTime: 1h

  iam:
    scrapeTime: 1h

  graph:
    scrapeTime: 1h

  costs:
    scrapeTime: 1h
    queries:
      - name: resource
        dimensions: [ResourceId]
        valueField: PreTaxCost
        timeFrames: [MonthToDate]

  portscan:
    scanner:
      parallel: 2
      threads: 1000
      timeout: 5
      ports: [22,80,443,3389]