diff --git a/_data/reference.yml b/_data/reference.yml index 108635adab8db..444bbf8e50540 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -65,10 +65,8 @@ toc: - title: Setup Tools section: - - title: Kubeadm - path: /docs/admin/kubeadm/ + - docs/admin/kubeadm.md - title: Kubefed - path: /docs/admin/kubefed/ section: - docs/admin/kubefed.md - docs/admin/kubefed_options.md diff --git a/docs/admin/kubeadm.md b/docs/admin/kubeadm.md index 7b617e60b9225..8ce7084946178 100644 --- a/docs/admin/kubeadm.md +++ b/docs/admin/kubeadm.md @@ -4,11 +4,16 @@ approvers: - luxas - errordeveloper - jbeda -title: kubeadm Setup Tool Reference Guide +title: Kubeadm +notitle: true --- +# kubeadm Setup Tool Reference Guide This document provides information on how to use kubeadm's advanced options. +* TOC +{:toc} + Running `kubeadm init` bootstraps a Kubernetes cluster. This consists of the following steps: @@ -27,8 +32,9 @@ following steps: dropping it in the cert directory (configured via `--cert-dir`, by default `/etc/kubernetes/pki`), this step is skipped. -1. Outputting a kubeconfig file for the kubelet to use to connect to the API - server, as well as an additional kubeconfig file for administration. +1. kubeadm outputs a kubeconfig file for the master's kubelet to use to connect + to the API server, as well as an additional kubeconfig file for cluster + administrators. 1. kubeadm generates Kubernetes static Pod manifests for the API server, controller manager and scheduler. It places them in @@ -46,8 +52,9 @@ following steps: Running `kubeadm join` on each node in the cluster consists of the following steps: -1. kubeadm downloads root CA information from the API server. It uses the token - to verify the authenticity of that data. +1. kubeadm downloads root CA information from the API server. By default, it + uses the bootstrap token and the CA key hash to verify the authenticity of + that data. The root CA can also be discovered directly via a file or URL. 1. kubeadm creates a local key pair. It prepares a certificate signing request (CSR) and sends that off to the API server for signing. The bootstrap token @@ -72,121 +79,123 @@ It is usually sufficient to run `kubeadm init` without any flags, but in some cases you might like to override the default behaviour. Here we specify all the flags that can be used to customise the Kubernetes installation. +**Options for `kubeadm init`:** + - `--apiserver-advertise-address` -This is the address the API Server will advertise to other members of the -cluster. This is also the address used to construct the suggested `kubeadm -join` line at the end of the init process. If not set (or set to 0.0.0.0) then -IP for the default interface will be used. + This is the address the API Server will advertise to other members of the + cluster. This is also the address used to construct the suggested `kubeadm + join` line at the end of the init process. If not set (or set to 0.0.0.0) then + IP for the default interface will be used. -This address is also added to the certificate that the API Server uses. + This address is also added to the certificate that the API Server uses. - `--apiserver-bind-port` -The port that the API server will bind on. This defaults to 6443. + The port that the API server will bind on. This defaults to 6443. - `--apiserver-cert-extra-sans` -Additional hostnames or IP addresses that should be added to the Subject -Alternate Name section for the certificate that the API Server will use. If you -expose the API Server through a load balancer and public DNS you could specify -this with - -``` ---apiserver-cert-extra-sans=kubernetes.example.com,kube.example.com,10.100.245.1 -``` + Additional hostnames or IP addresses that should be added to the Subject + Alternate Name section for the certificate that the API Server will use. If you + expose the API Server through a load balancer and public DNS you could specify + this with + + ``` + --apiserver-cert-extra-sans=kubernetes.example.com,kube.example.com,10.100.245.1 + ``` - `--cert-dir` -The path where to save and store the certificates. The default is -"/etc/kubernetes/pki". + The path where to save and store the certificates. The default is + "/etc/kubernetes/pki". - `--config` -A kubeadm specific [config file](#config-file). This can be used to specify an -extended set of options including passing arbitrary command line flags to the -control plane components. - -**Note**: When providing configuration values using _both_ a configuration file -and flags, the file will take precedence. For example, if a file exists with: - -```yaml -apiVersion: kubeadm.k8s.io/v1alpha1 -kind: MasterConfiguration -token: 1234 -``` - -and the user ran `kubeadm init --config file.yaml --token 5678`, -the chosen token value will be `1234`. + A kubeadm specific [config file](#config-file). This can be used to specify an + extended set of options including passing arbitrary command line flags to the + control plane components. + + **Note**: When providing configuration values using _both_ a configuration file + and flags, the file will take precedence. For example, if a file exists with: + + ```yaml + apiVersion: kubeadm.k8s.io/v1alpha1 + kind: MasterConfiguration + token: 1234 + ``` + + and the user ran `kubeadm init --config file.yaml --token 5678`, + the chosen token value will be `1234`. - `--kubernetes-version` (default 'latest') the kubernetes version to initialise -The **v1.6** version of kubeadm only supports building clusters that are at -least **v1.6.0**. There are many reasons for this including kubeadm's use of -RBAC, the Bootstrap Token system, and enhancements to the Certificates API. With -this flag you can try any future version of Kubernetes. Check [releases -page](https://github.com/kubernetes/kubernetes/releases) for a full list of -available versions. + The **v1.6** version of kubeadm only supports building clusters that are at + least **v1.6.0**. There are many reasons for this including kubeadm's use of + RBAC, the Bootstrap Token system, and enhancements to the Certificates API. With + this flag you can try any future version of Kubernetes. Check [releases + page](https://github.com/kubernetes/kubernetes/releases) for a full list of + available versions. - `--pod-network-cidr` -For certain networking solutions the Kubernetes master can also play a role in -allocating network ranges (CIDRs) to each node. This includes many cloud -providers and flannel. You can specify a subnet range that will be broken down -and handed out to each node with the `--pod-network-cidr` flag. This should be a -minimum of a /16 so controller-manager is able to assign /24 subnets to each -node in the cluster. If you are using flannel with [this -manifest](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) -you should use `--pod-network-cidr=10.244.0.0/16`. Most CNI based networking -solutions do not require this flag. + For certain networking solutions the Kubernetes master can also play a role in + allocating network ranges (CIDRs) to each node. This includes many cloud + providers and flannel. You can specify a subnet range that will be broken down + and handed out to each node with the `--pod-network-cidr` flag. This should be a + minimum of a /16 so controller-manager is able to assign /24 subnets to each + node in the cluster. If you are using flannel with [this + manifest](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) + you should use `--pod-network-cidr=10.244.0.0/16`. Most CNI based networking + solutions do not require this flag. - `--service-cidr` (default '10.96.0.0/12') -You can use the `--service-cidr` flag to override the subnet Kubernetes uses to -assign pods IP addresses. If you do, you will also need to update the -`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file to reflect this -change else DNS will not function correctly. + You can use the `--service-cidr` flag to override the subnet Kubernetes uses to + assign pods IP addresses. If you do, you will also need to update the + `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file to reflect this + change else DNS will not function correctly. - `--service-dns-domain` (default 'cluster.local') -By default, `kubeadm init` deploys a cluster that assigns services with DNS -names `..svc.cluster.local`. You can use the -`--service-dns-domain` to change the DNS name suffix. Again, you will need to -update the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file -accordingly else DNS will not function correctly. - -**Note**: This flag has an effect (it's needed for the kube-dns Deployment -manifest and the API Server's serving certificate) but not as you might expect, -since you will have to modify the arguments to the kubelets in the cluster for -it to work fully. Specifying DNS parameters using this flag only is not enough. -Rewriting the kubelet's CLI arguments is out of scope for kubeadm as it should -be agnostic to how you run the kubelet. However, making all kubelets in the -cluster pick up information dynamically via the API _is_ in scope and is a -[planned feature](https://github.com/kubernetes/kubeadm/issues/28) for upcoming -releases. + By default, `kubeadm init` deploys a cluster that assigns services with DNS + names `..svc.cluster.local`. You can use the + `--service-dns-domain` to change the DNS name suffix. Again, you will need to + update the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file + accordingly else DNS will not function correctly. + + **Note**: This flag has an effect (it's needed for the kube-dns Deployment + manifest and the API Server's serving certificate) but not as you might expect, + since you will have to modify the arguments to the kubelets in the cluster for + it to work fully. Specifying DNS parameters using this flag only is not enough. + Rewriting the kubelet's CLI arguments is out of scope for kubeadm as it should + be agnostic to how you run the kubelet. However, making all kubelets in the + cluster pick up information dynamically via the API _is_ in scope and is a + [planned feature](https://github.com/kubernetes/kubeadm/issues/28) for upcoming + releases. - `--skip-preflight-checks` -By default, kubeadm runs a series of preflight checks to validate the system -before making any changes. Advanced users can use this flag to bypass these if -necessary. + By default, kubeadm runs a series of preflight checks to validate the system + before making any changes. Advanced users can use this flag to bypass these if + necessary. - `--token` -By default, `kubeadm init` automatically generates the token used to initialise -each new node. If you would like to manually specify this token, you can use the -`--token` flag. The token must be of the format `[a-z0-9]{6}\.[a-z0-9]{16}`. A -compatible random token can be generated `kubeadm token generate`. Tokens can -be managed through the API after the cluster is created. See the [section on -managing tokens](#manage-tokens) below. + By default, `kubeadm init` automatically generates the token used to initialise + each new node. If you would like to manually specify this token, you can use the + `--token` flag. The token must be of the format `[a-z0-9]{6}\.[a-z0-9]{16}`. A + compatible random token can be generated `kubeadm token generate`. Tokens can + be managed through the API after the cluster is created. See the [section on + managing tokens](#manage-tokens) below. - `--token-ttl` -This sets an expiration time for the token. This is specified as a duration -from the current time. After this time the token will no longer be valid and -will be removed. A value of 0 specifies that the token never expires. 0 is the -default. See the [section on managing tokens](#manage-tokens) below. + This sets an expiration time for the token. This is specified as a duration + from the current time. After this time the token will no longer be valid and + will be removed. A value of 0 specifies that the token never expires. The + default is 24 hours. See the [section on managing tokens](#manage-tokens) below. ### `kubeadm join` @@ -194,15 +203,24 @@ When joining a kubeadm initialized cluster, we need to establish bidirectional trust. This is split into discovery (having the Node trust the Kubernetes master) and TLS bootstrap (having the Kubernetes master trust the Node). -There are 2 main schemes for discovery. The first is to use a shared token along -with the IP address of the API server. The second is to provide a file (a subset -of the standard kubeconfig file). This file can be a local file or downloaded -via an HTTPS URL. The forms are `kubeadm join --discovery-token -abcdef.1234567890abcdef 1.2.3.4:6443`, `kubeadm join --discovery-file -path/to/file.conf` or `kubeadm join --discovery-file https://url/file.conf`. +There are two main schemes for discovery: + + - **Using a shared token** along with the IP address of the API server and a + hash of the root CA key: + + `kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 1.2.3.4:6443` + + - **Using a file** (a subset of the standard kubeconfig file). This file can + be a local file or downloaded via an HTTPS URL: + + `kubeadm join --discovery-file path/to/file.conf` + + `kubeadm join --discovery-file https://url/file.conf` + Only one form can be used. If the discovery information is loaded from a URL, HTTPS must be used and the host installed CA bundle is used to verify the -connection. +connection. For details on the security tradeoffs of these mechanisms, see the +[security model](#security-model) section below. The TLS bootstrap mechanism is also driven via a shared token. This is used to temporarily authenticate with the Kubernetes master to submit a certificate @@ -216,60 +234,83 @@ can be used instead of specifying the each token individually. Here's an example on how to use it: -`kubeadm join --token=abcdef.1234567890abcdef 192.168.1.1:6443` +`kubeadm join --token=abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 192.168.1.1:6443` -Specific options: +**Options for `kubeadm join`:** - `--config` -Extended options a specified in the [kubeadm specific config file](#config-file). + Extended options are specified in the [kubeadm specific config file](#config-file). - `--skip-preflight-checks` -By default, kubeadm runs a series of preflight checks to validate the system -before making any changes. Advanced users can use this flag to bypass these if -necessary. + By default, kubeadm runs a series of preflight checks to validate the system + before making any changes. Advanced users can use this flag to bypass these if + necessary. - `--discovery-file` -A local file path or HTTPS URL. The file specified must be a kubeconfig file -with nothing but an unnamed cluster entry. This is used to find both the -location of the API server to join along with a root CA bundle to use when -talking to that server. - -This might look something like this: - -``` yaml -apiVersion: v1 -clusters: -- cluster: - certificate-authority-data: - server: https://10.138.0.2:6443 - name: "" -contexts: [] -current-context: "" -kind: Config -preferences: {} -users: [] -``` + A local file path or HTTPS URL. The file specified must be a kubeconfig file + with nothing but an unnamed cluster entry. This is used to find both the + location of the API server to join along with a root CA bundle to use when + talking to that server. + + This might look something like this: + + ``` yaml + apiVersion: v1 + clusters: + - cluster: + certificate-authority-data: + server: https://10.138.0.2:6443 + name: "" + contexts: [] + current-context: "" + kind: Config + preferences: {} + users: [] + ``` - `--discovery-token` -The discovery token is used along with the address of the API server (as an -unnamed argument) to download and verify information about the cluster. The -most critical part of the cluster information is the root CA bundle used to -verify the identity of the server during subsequent TLS connections. + The discovery token is used along with the address of the API server (as an + unnamed argument) to download and verify information about the cluster. The + most critical part of the cluster information is the root CA bundle used to + verify the identity of the server during subsequent TLS connections. + +- `--discovery-token-ca-cert-hash` + + The CA key hash is used to verify the full root CA certificate discovered during + token-based bootstrapping. It has the format `sha256:`. By + default, the hash value is returned in the `kubeadm join` command printed at the + end of `kubeadm init`. It is in a standard format (see + [RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4)) and can also be + calculated by 3rd party tools or provisioning systems. For example, using the + OpenSSL CLI: + `openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>&/dev/null | openssl dgst -sha256 -hex` + + _Skipping this flag is allowed in Kubernetes 1.8, but makes certain spoofing + attacks possible._ See the [security model](#security-model) for details. + Pass `--discovery-token-unsafe-skip-ca-verification` to silence warnings (which + will become errors in Kubernetes 1.9). + +- `--discovery-token-unsafe-skip-ca-verification` + + Disable the warning/error that occurs when `--discovery-token-ca-cert-hash` is + not provided. Passing this flag is an acknowledgement of the + [security tradeoffs](#security-model) involved in skipping this verification + (which may or may not be appropriate in your environment). - `--tls-bootstrap-token` -The token used to authenticate to the API server for the purposes of TLS -bootstrapping. + The token used to authenticate to the API server for the purposes of TLS + bootstrapping. - `--token=` -Often times the same token is used for both `--discovery-token` and -`--tls-bootstrap-token`. This option specifies the same token for both. Other -flags override this flag if present. + Often times the same token is used for both `--discovery-token` and + `--tls-bootstrap-token`. This option specifies the same token for both. Other + flags override this flag if present. ## Using kubeadm with a configuration file {#config-file} @@ -414,34 +455,53 @@ created cluster (`/etc/kubernetes/admin.conf`). You can specify an alternate kubeconfig file for credentials with the `--kubeconfig` to the following commands. -* `kubeadm token list` Lists the tokens along with when they expire and what the - approved usages are. -* `kubeadm token create` Creates a new token. - * `--description` Set the description on the new token. - * `--ttl duration` Set expiration time of the token as a delta from "now". - Default is 0 for no expiration. The unit of the duration is seconds. - * `--usages` Set the ways that the token can be used. The default is +* `kubeadm token list` - List tokens (along with their expirations, usages, and groups). + +* `kubeadm token create` - Create a new token. + + * `--description ` + + Set the human-readable description for the new token. + + * `--ttl ` + + Set the expiration time-to-live of the token relative to the current time. + Default is 24 hours. A value of 0 means "never expire". The default unit + of the duration is seconds but other units can be specified (e.g., `15m`, `1h`). + + * `--usages [,...]` + + Set the ways that the token can be used. The default is `signing,authentication`. These are the usages as described above. -* `kubeadm token delete |.` Delete a token. - The token can either be identified with just an ID or with the entire token - value. Only the ID is used; the token is still deleted if the secret does not - match. -In addition, you can use the `kubeadm token generate` command to locally creates -a new token. This token is of the correct form for specifying with the -`--token` argument to `kubeadm init`. + * `--groups [,...]` + + Add extra bootstrap groups that the new token will authenticate as. Can be + specified multiple times. Each extra group must start with + `system:bootstrappers:`. This is an advanced feature meant for custom + bootstrap scenarios where you want to change how CSR approval works for + different groups of nodes. + +* `kubeadm token delete |.` - Delete a token. + + The token can either be identified with just an ID or with the + entire token value. Only the ID is used; the token is still deleted if the + secret does not match. + +* `kubeadm token generate` - Generate a token locally. + + Locally generate a token but do not create it on the server. This token is of + the correct form for specifying with the `--token` argument to `kubeadm init`. For the gory details on how the tokens are implemented (including managing them -outside of kubeadm) see the [Bootstrap Token -docs](/docs/admin/bootstrap-tokens/). +outside of kubeadm) see the [Bootstrap Token docs](/docs/admin/bootstrap-tokens/). ## Automating kubeadm Rather than copying the token you obtained from `kubeadm init` to each node, as -in the [basic kubeadm tutorial](/docs/admin/kubeadm/), you can -parallelize the token distribution for easier automation. To implement this -automation, you must know the IP address that the master will have after it is -started. +in the [basic kubeadm tutorial](/docs/admin/kubeadm/), you can parallelize the +token distribution for easier automation. To implement this automation, you must +know the IP address that the master will have after it is started. 1. Generate a token. This token must have the form `<6 character string>.<16 character string>`. More formally, it must match the regex: @@ -460,6 +520,91 @@ started. Once the cluster is up, you can grab the admin credentials from the master node at `/etc/kubernetes/admin.conf` and use that to talk to the cluster. +Note that this style of bootstrap has some relaxed security guarantees because +it does not allow the root CA hash to be validated with +`--discovery-token-ca-cert-hash` (since it's not generated when the nodes are +provisioned). For details, see the [security model](#security-model). + +## Security model +The kubeadm discovery system has several options, each with security tradeoffs. +The right method for your environment depends on how you provision nodes and the +security expectations you have about your network and node lifecycles. + +### Token-based discovery with CA pinning +_This is the default mode in Kubernetes 1.8._ In this mode, kubeadm downloads +the cluster configuration (including root CA) and validates it using the token +as well as validating that the root CA public key matches the provided hash and +that the API server certificate is valid under the root CA. + +**Example `kubeadm join` command:** + + - `kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 1.2.3.4:6443` + +**Advantages:** + + - Allows bootstrapping nodes to securely discover a root of trust for the + master even if other worker nodes or the network are compromised. + + - Convenient to execute manually since all of the information required fits + into a single `kubeadm join` command that is easy to copy and paste. + +**Disadvantages:** + + - The CA hash is not normally known until the master has been provisioned, + which can make it more difficult to build automated provisioning tools that + use kubeadm. + +### Token-based discovery without CA pinning +_This was the default in Kubernetes 1.7 and earlier_, but comes with some +important caveats. This mode relies only on the symmetric token to sign +(HMAC-SHA256) the discovery information that establishes the root of trust for +the master. It's still possible in Kubernetes 1.8 and above using the +`--discovery-token-unsafe-skip-ca-verification` flag, but you should consider +using one of the other modes if possible. + +**Example `kubeadm join` command:** + + - `kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-verification 1.2.3.4:6443` + +**Advantages:** + + - Still protects against many network-level attacks. + + - The token can be generated ahead of time and shared with the master and + worker nodes, which can then bootstrap in parallel without coordination. This + allows it to be used in many provisioning scenarios. + +**Disadvantages:** + + - If an attacker is able to steal a bootstrap token via some vulnerability, + they can use that token (along with network-level access) to impersonate the + master to other bootstrapping nodes. This may or may not be an appropriate + tradeoff in your environment. + +### File or HTTPS-based discovery +This provides an out-of-band way to establish a root of trust between the master +and bootstrapping nodes. Consider using this mode if you are building automated provisioning +using kubeadm. + +**Example `kubeadm join` commands:** + + - `kubeadm join --discovery-file path/to/file.conf` (local file) + + - `kubeadm join --discovery-file https://url/file.conf` (remote HTTPS URL) + +**Advantages:** + - Allows bootstrapping nodes to securely discover a root of trust for the + master even if other worker nodes or the network are compromised. + +**Disadvantages:** + - Requires that you have some way to carry the discovery information from + the master to the bootstrapping nodes. This might be possible, for example, + via your cloud provider or provisioning tool. The information in this file is + not secret, but HTTPS or equivalent is required to ensure its integrity. + + - Less convenient to use manually since the file is difficult to copy and paste + between nodes. + ## Use Kubeadm with other CRI runtimes Since [Kubernetes 1.6 release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#node-components-1), Kubernetes container runtimes have been transferred to using CRI by default. Currently, the build-in container runtime is Docker which is enabled by build-in `dockershim` in `kubelet`. @@ -492,17 +637,17 @@ the following images are required for the cluster works will be automatically pulled by the kubelet if they don't exist locally while `kubeadm init` is initializing your master: -| Image Name | v1.6 release branch version | v1.7 release branch version +| Image Name | v1.7 release branch version | v1.8 release branch version |---|---|---| -| gcr.io/google_containers/kube-apiserver-${ARCH} | v1.6.x | v1.7.x -| gcr.io/google_containers/kube-controller-manager-${ARCH} | v1.6.x | v1.7.x -| gcr.io/google_containers/kube-scheduler-${ARCH} | v1.6.x | v1.7.x -| gcr.io/google_containers/kube-proxy-${ARCH} | v1.6.x | v1.7.x +| gcr.io/google_containers/kube-apiserver-${ARCH} | v1.7.x | v1.8.x +| gcr.io/google_containers/kube-controller-manager-${ARCH} | v1.7.x | v1.8.x +| gcr.io/google_containers/kube-scheduler-${ARCH} | v1.7.x | v1.8.x +| gcr.io/google_containers/kube-proxy-${ARCH} | v1.7.x | v1.8.x | gcr.io/google_containers/etcd-${ARCH} | 3.0.17 | 3.0.17 | gcr.io/google_containers/pause-${ARCH} | 3.0 | 3.0 -| gcr.io/google_containers/k8s-dns-sidecar-${ARCH} | 1.14.1 | 1.14.4 -| gcr.io/google_containers/k8s-dns-kube-dns-${ARCH} | 1.14.1 | 1.14.4 -| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.1 | 1.14.4 +| gcr.io/google_containers/k8s-dns-sidecar-${ARCH} | 1.14.4 | 1.14.4 +| gcr.io/google_containers/k8s-dns-kube-dns-${ARCH} | 1.14.4 | 1.14.4 +| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.4 | 1.14.4 Here `v1.7.x` means the "latest patch release of the v1.7 branch". @@ -534,7 +679,7 @@ A breakdown of what/why: tells the kubelet where the API server is. This file also has the kubelet's credentials. * `--require-kubeconfig=true` the kubelet should fail fast if the kubeconfig file - is not present. This makes the kubelet crashloop during the time the service is + is not present. This makes the kubelet crash loop during the time the service is started until `kubeadm init` is run. * `--pod-manifest-path=/etc/kubernetes/manifests` specifies from where to read Static Pod manifests used for spinning up the control plane @@ -562,7 +707,7 @@ A breakdown of what/why: systemctl restart kubelet ``` -## Cloudprovider integrations (experimental) +## Cloud provider integrations (experimental) Enabling specific cloud providers is a common request. This currently requires manual configuration and is therefore not yet fully supported. If you wish to do @@ -573,7 +718,7 @@ installed on the host, for example for volume mounting/unmounting, install those packages. Specify the `--cloud-provider` flag for the kubelet and set it to the cloud of your -choice. If your cloudprovider requires a configuration file, create the file +choice. If your cloud provider requires a configuration file, create the file `/etc/kubernetes/cloud-config` on every node. The exact format and content of that file depends on the requirements imposed by your cloud provider. If you use the `/etc/kubernetes/cloud-config` file, you must append it to the kubelet