Skip to content

Commit b792c62

Browse files
2b3c5112b3c511
andauthored
Helm README.md (#26)
* helm * add license * helm add readme.md --------- Co-authored-by: 2b3c511 <[email protected]>
1 parent d67cfc5 commit b792c62

File tree

8 files changed

+302
-0
lines changed

8 files changed

+302
-0
lines changed

helm/README.md

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
22+
# IoTDB CLuster on Kubernetes
23+
24+
## Deployment
25+
26+
### Deployment environment preparation
27+
28+
Make sure that the current environment has the following tools installed and can successfully connect to the corresponding Kubernetes cluster.
29+
30+
* kubectl
31+
* helm (version >= v3.0.0)
32+
33+
### Parameter Configuration
34+
35+
Modify the configuration parameters in the `helm/values.yaml` file according to the corresponding requirements. Specifically:
36+
37+
1. Verify the current cluster's storage medium and specify the corresponding `storageClass`. IoTDB will create a `pv` on the corresponding data storage medium based on the `storageClass`.
38+
39+
```
40+
storage:
41+
className: local-storage # Set this value to the storage medium available in the current Kubernetes environment
42+
```
43+
44+
2. Confirm the scale of the cluster
45+
```
46+
datanode:
47+
# The number of DataNodes nodeCount: 3
48+
nodeCount: 3
49+
# The size of each datanode's corresponding data directory needs to be specified based on the designed data volume, and it cannot be modified afterward. In production environments, it is recommended to provide sufficient space based on the size of the data.
50+
storageCapacity: 20Gi
51+
52+
confignode:
53+
# The number of ConfigNodes nodeCount: 3
54+
nodeCount: 3
55+
#The size of the PVC used by confignode is generally not more than 10Gi, and the default value can be used directly.
56+
storageCapacity: 10Gi
57+
```
58+
59+
3. Verify the number of replicas
60+
61+
The IoTDB cluster can provide the ability to store multiple copies of data. You can choose a flexible storage medium based on your own usage. For example, if the storage medium corresponding to the storageClass does not provide data replication capabilities, you can set the metadata replication to 3 and the data replication to 3 to ensure data availability. If the storage medium corresponding to the storageClass has data redundancy, no data replication is needed at the IoTDB cluster level, and the metadata replication can be set to 1, and the data replication can be set to 1 or 2.
62+
63+
```
64+
confignode:
65+
# Number of metadata replicas
66+
schemaReplicationFactor: 1
67+
# Number of data replicas
68+
dataReplicationFactor: 2
69+
```
70+
71+
72+
### Start the deployment
73+
74+
Go to the `./helm/` directory and exec `install` command to install the IoTDB Cluster in a Kubernetes environment.
75+
76+
```
77+
helm install iotdb-cluster .
78+
```
79+
80+
### Wait for the deployment to complete
81+
82+
You can use the `kubectl` command to view the progress of the deployment. When all the pods are in the Running state, the system is considered to be in a healthy state, as shown in the following figure: <img src='../images/getpods.png'>
83+
84+
## Connect to the cluster
85+
86+
### To connect to a cluster in a Kubernetes environment
87+
88+
The address of a DataNode is `datanode-<N>.datanode-svc.iotdb-cluster`,with port number `6667`,You need to modify the value of `<N>` in the address according to the index of the DataNode you want to connect to. For example, `datanode-0.datanode-svc.iotdb-cluster` represents the address of DataNode-0.
89+
90+
### Connect to the cluster from outside the cluster
91+
92+
If you want to connect to the cluster from outside via a program, you need to map the datanode-svc in the cluster to an address that can be accessed from outside the cluster; this may depend on the cloud characteristics of the current Kubernetes cluster.
93+
For example, if your Kubernetes cluster has the "LoadBalancer" feature, you can map the 6667 port of the datanode to the outside of the cluster.
94+
95+
If you only need to perform local testing, you can map the 6667 port of the node datanode to a certain port on a Kubernetes cluster node through the `nodePort` method, and connect to the datanode by the mapped port and the IP address of the node. You can refer to
96+
```
97+
# node-port.yaml
98+
apiVersion: v1
99+
kind: Service
100+
metadata:
101+
namespace: iotdb-cluster
102+
name: datanode-nodeport
103+
spec:
104+
type: NodePort
105+
ports:
106+
- port: 6667
107+
targetPort: 6667
108+
nodePort: 32667
109+
selector:
110+
app: datanode
111+
```
112+
113+
## Demonstration Effects
114+
115+
### Cluster Deployment and Operational Status
116+
117+
We will show the operation status of each component in the cluster after deploying with 3 ConfigNodes and 3 DataNodes as an example.
118+
119+
1. Status of Pods
120+
121+
<img src='../images/getpods.png'>
122+
123+
2. Use of PVC
124+
125+
<img src='../images/getpvc.png'>
126+
127+
Each confignode and datanode will use a separate PVC and persist its data to the corresponding PVC.
128+
129+
### Connect and access the cluster within the cluster
130+
131+
1. Use the CLI to establish a connection to the cluster's internal address of the datanode.
132+
133+
<img src='../images/logininner.png'>
134+
135+
136+
2. Show the node status in the cluster.
137+
138+
<img src='../images/showcluster.png'>
139+
140+
It can be seen that the node addresses in the cluster are all internal FQDNs in Kubernetes.
141+
142+
3. Show the top 10 time series in the cluster (after test data has been written)
143+
144+
<img src='../images/showtimeseries.png'>
145+
146+
3. Perform data queries
147+
148+
<img src='../images/select10.png'>

helm/README_zh.md

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
22+
# IoTDB CLuster on Kubernetes
23+
24+
## 部署方法
25+
26+
### 部署环境准备
27+
28+
确保当前环境安装了以下工具并能成功连接到对应的 Kubernetes 集群
29+
30+
* kubectl
31+
* helm (version >= v3.0.0)
32+
33+
### 参数配置
34+
35+
根据对应的需求修改 `helm/values.yaml` 里的配置参数。具体为:
36+
37+
1. 确认当前集群的存储介质,并指定相应的`storageClass`。IoTDB 将会根据`storageClass` 在对应的数据存储介质上创建`pv`
38+
39+
```
40+
storage:
41+
className: local-storage # 将该值指定为当前Kubernetes 中可用的存储介质
42+
```
43+
44+
2. 确认集群规模
45+
```
46+
datanode:
47+
# datanode 的节点数量
48+
nodeCount: 3
49+
# 每个 datanode 对应的数据目录的大小,需要根据设计的数据量指定,且指定后不可修改,生产环境中,建议根据数据量的大小给定充足的空间
50+
storageCapacity: 20Gi
51+
52+
confignode:
53+
# confignode 的节点数量
54+
nodeCount: 3
55+
# confignode 使用的pvc的大小,一般不会超过10Gi,可直接使用默认值
56+
storageCapacity: 10Gi
57+
```
58+
59+
3. 确认数据副本数
60+
61+
IoTDB 集群可以提供数据的多副本存储能力。可根据自身使用的存储介质进行灵活的选择。例如,storageClass对应的数据存储介质本身不提供数据的副本能力,则可以将元数据的副本设置为3、数据的副本也设置为3的方式来保证数据的可用性;如果storageClass对应的数据存储介质做了数据冗余,那无需在IoTDB 集群层面进行数据的多副本存储,可将元数据副本数置为1,数据副本数置为1或2。
62+
63+
```
64+
confignode:
65+
# 元数据副本数
66+
schemaReplicationFactor: 1
67+
# 数据副本数
68+
dataReplicationFactor: 2
69+
```
70+
71+
72+
### 启动部署
73+
74+
进入 ./helm/ 目录,执行install 命令,将 IoTDB Cluster 安装到 Kubernetes 环境中
75+
76+
```
77+
helm install iotdb-cluster .
78+
```
79+
80+
### 等待部署完毕
81+
82+
可以通过`kubectl` 命令来查看部署的进展,当所有pod 为 Running 状态时,系统就处于可以状态了, 如下图所示
83+
<img src='../images/getpods.png'>
84+
85+
86+
## 连接到集群
87+
88+
### 在 Kubernetes 环境中来连接集群
89+
90+
DataNode 的地址为 `datanode-<N>.datanode-svc.iotdb-cluster`, 端口号为`6667`;需要根据想要连接的datanode的序号,修改地址中\<N\> 的值; 例如,`datanode-0.datanode-svc.iotdb-cluster` 表示datanode-0 的地址。
91+
92+
### 在集群外部连接到集群
93+
94+
如果希望通过集群外部的程序连接到集群,则需要将集群中的 datanode-svc 映射到集群外可以访问的地址;这可能取决于当前 Kubernetes 集群的云特性。
95+
96+
例如,如果您的 Kubernetes 集群具有`LoadBalancer` 的功能,则您可以将 datanode 的6667端口映射到集群外部;
97+
98+
如果仅仅是为了进行本地测试,则可以通过`nodePort` 的方式将节点datanode的6667端口映射到Kubernetes集群节点的某一端口,并通过映射后的端口和节点的ip地址对datanode进行连接。可参考
99+
100+
```
101+
# node-port.yaml
102+
apiVersion: v1
103+
kind: Service
104+
metadata:
105+
namespace: iotdb-cluster
106+
name: datanode-nodeport
107+
spec:
108+
type: NodePort
109+
ports:
110+
- port: 6667
111+
targetPort: 6667
112+
nodePort: 32667
113+
selector:
114+
app: datanode
115+
```
116+
117+
118+
## 演示效果
119+
120+
### 集群部署及运行状态
121+
122+
下面,以3个ConfigNode、3个DataNodes的部署方案为例,展示一下集群部署后各组件的运行状况
123+
124+
1. Pod运行情况
125+
126+
<img src='../images/getpods.png'>
127+
128+
2. pvc 使用情况
129+
130+
<img src='../images/getpvc.png'>
131+
132+
每个confignode\datanode 都会独立使用一个pvc,并将其数据持久化到对应的pvc中
133+
134+
135+
### 在集群内部连接并访问集群
136+
137+
1. 使用cli通过datanode的集群内部地址建立连接
138+
139+
<img src='../images/logininner.png'>
140+
141+
142+
2. 展示集群中的节点情况
143+
144+
<img src='../images/showcluster.png'>
145+
146+
可以看到,集群中的节点地址均为Kubernetes 中内部FQDN
147+
148+
3. 展示集群中的前十条时间序列(已经进行了测试数据写入)
149+
150+
<img src='../images/showtimeseries.png'>
151+
152+
3. 进行数据查询
153+
154+
<img src='../images/select10.png'>

images/getpods.png

40.3 KB
Loading

images/getpvc.png

60.6 KB
Loading

images/logininner.png

43.1 KB
Loading

images/select10.png

70.6 KB
Loading

images/showcluster.png

71.9 KB
Loading

images/showtimeseries.png

166 KB
Loading

0 commit comments

Comments
 (0)