Skip to content

Commit cf011a0

Browse files
author
David Scott
committed
RRDs: initial attempt to merge content from the Xen wiki
The XenServer SDK and the Xen wiki both have partial, not well structured descriptions of how the RRD metrics gathering system can be used. Hopefully we can turn this into a more useful document. Signed-off-by: David Scott <[email protected]>
1 parent 143fbda commit cf011a0

File tree

1 file changed

+175
-61
lines changed

1 file changed

+175
-61
lines changed

‎xen-api/metrics.md‎

Lines changed: 175 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -3,66 +3,50 @@ title: Metrics
33
layout: default
44
---
55

6-
Getting XenServer Performance Statistics
7-
---------------------------------------------
6+
Getting Performance Statistics
7+
==============================
88

9-
XenServer records statistics about the performance of various
10-
aspects of your XenServer installation. The metrics are stored
9+
[xcp-rrdd](https://github.com/xapi-project/xcp-rrdd)
10+
records statistics about the host and the VMs running on top.
11+
The metrics are stored
1112
persistently for long term access and analysis of historical trends.
12-
Where storage is available to a VM, the statistics are written to disk
13-
when a VM is shut down. Statistics are stored in RRDs (Round Robin
14-
Databases), which are maintained for individual VMs (including the
15-
control domain) and the server. RRDs are resident on the server on which
13+
Statistics are stored in
14+
[RRDs](http://oss.oetiker.ch/rrdtool/)
15+
(Round Robin Databases).
16+
RRDs are fixed-size structures containing data at multiple resolutions.
17+
'Data sources' are sampled every few seconds and points are added to
18+
the highest resolution RRD. Periodically each high-frequency RRD is
19+
'consolidated' (e.g. averaged) to produce a data point for a lower-frequency RRD.
20+
21+
RRDs are resident on the server on which
1622
the VM is running, or the pool master when the VM is not running. The
1723
RRDs are also backed up every day.
1824

19-
> **Warning**
20-
>
21-
> In earlier versions of the XenServer API, instantaneous
22-
> performance metrics could be obtained using the `VM_metrics`,
23-
> `VM_guest_metrics`, `host_metrics` methods and associated methods.
24-
> These methods has been deprecated in favor of using the http handler
25-
> described in this chapter to download the statistics from the RRDs on
26-
> the VMs and servers. Note that by default the legacy metrics will
27-
> return zeroes. To revert to periodic statistical polling as present in
28-
> earlier versions of XenServer, set the
29-
> `other-config:rrd_update_interval=` parameters on your host to one of
30-
> the following values, and restart your host:
31-
>
32-
> never
33-
> : This is the default, meaning no periodic polling is performed.
34-
>
35-
> 1
36-
> : Polling is performed every 5 seconds.
37-
>
38-
> 2
39-
> : Polling is performed every minute.
40-
>
41-
> By default, the older metrics APIs will not return any values, and so
42-
> this key must be enabled to run monitoring clients which use the
43-
> legacy monitoring protocol.
25+
Granularity
26+
-----------
4427

4528
Statistics are persisted for a maximum of one year, and are stored at
4629
different granularities. The average and most recent values are stored
4730
at intervals of:
4831

4932
- 5 seconds for the past 10 minutes
50-
5133
- one minute for the past 2 hours
52-
5334
- one hour for the past week
54-
5535
- one day for the past year
5636

5737
RRDs are saved to disk as uncompressed XML. The size of each RRD when
5838
written to disk ranges from 200KiB to approximately 1.2MiB when the RRD
5939
stores the full year of statistics.
6040

61-
> **Warning**
62-
>
63-
> If statistics cannot be written to disk, for example when a disk is
64-
> full, statistics will be lost and the last saved version of the RRD
65-
> will be used.
41+
By default each RRD contains only averaged data. To record minimum and
42+
maximum values, set the Pool-wide flag:
43+
44+
```sh
45+
xe pool-param-set uuid= other-config:create_min_max_in_new_VM_RRDs
46+
```
47+
48+
Downloading
49+
===========
6650

6751
Statistics can be downloaded over HTTP in XML format, for example using
6852
`wget`. See [](http://oss.oetiker.ch/rrdtool/doc/rrddump.en.html) and
@@ -71,43 +55,173 @@ about the XML format. HTTP authentication can take the form of a
7155
username and password or a session token. Parameters are appended to the
7256
URL following a question mark (?) and separated by ampersands (&).
7357

58+
Statistics may be downloaded all at once, including all history, or as
59+
deltas suitable for interactive graphing.
60+
61+
Downloading statistics all at once
62+
----------------------------------
63+
64+
To obtain a full dump of RRD data for a host use:
65+
66+
```sh
67+
wget http://user:password@host/host_rrd
68+
```
69+
70+
The RRD data is in the XML export format used by rrdtool and looks like this:
71+
72+
```xml
73+
<?xml version="1.0"?>
74+
<rrd>
75+
<version>0003</version>
76+
<step>5</step>
77+
<lastupdate>1213616574</lastupdate>
78+
<ds>
79+
<name>memory_total_kib</name>
80+
<type>GAUGE</type>
81+
<minimal_heartbeat>300.0000</minimal_heartbeat>
82+
<min>0.0</min>
83+
<max>Infinity</max>
84+
<last_ds>2070172</last_ds>
85+
<value>9631315.6300</value>
86+
<unknown_sec>0</unknown_sec>
87+
</ds>
88+
<ds>
89+
<!-- other dss - the order of the data sources is important
90+
and defines the ordering of the columns in the archives below -->
91+
</ds>
92+
<rra>
93+
<cf>AVERAGE</cf>
94+
<pdp_per_row>1</pdp_per_row>
95+
<params>
96+
<xff>0.5000</xff>
97+
</params>
98+
<cdp_prep> <!-- This is for internal use -->
99+
<ds>
100+
<primary_value>0.0</primary_value>
101+
<secondary_value>0.0</secondary_value>
102+
<value>0.0</value>
103+
<unknown_datapoints>0</unknown_datapoints>
104+
</ds>
105+
...other dss - internal use only...
106+
</cdp_prep>
107+
<database>
108+
<row>
109+
<v>2070172.0000</v> <!-- columns correspond to the DSs defined above -->
110+
<v>1756408.0000</v>
111+
<v>0.0</v>
112+
<v>0.0</v>
113+
<v>732.2130</v>
114+
<v>0.0</v>
115+
<v>782.9186</v>
116+
<v>0.0</v>
117+
<v>647.0431</v>
118+
<v>0.0</v>
119+
<v>0.0001</v>
120+
<v>0.0268</v>
121+
<v>0.0100</v>
122+
<v>0.0</v>
123+
<v>615.1072</v>
124+
</row>
125+
...
126+
</rra>
127+
... other archives ...
128+
</rrd>
129+
130+
```
131+
132+
To obtain a full dump of RRD data of a VM with uuid `x`:
133+
134+
```sh
135+
wget http://user:password@host/vm_rrd?uuid=x
136+
```
137+
138+
Note that it is quite expensive to download full RRDs as they contain
139+
lots of historical information. For interactive displays clients should
140+
download deltas instead.
141+
142+
143+
Downloading deltas
144+
------------------
145+
74146
To obtain an update of all VM statistics on a host, the URL would be of
75147
the form:
76148

77-
http://:@/rrd_updates?start=
149+
```sh
150+
wget http://user:password@host/rrd_updates?start=0
151+
```
78152

79153
This request returns data in an rrdtool `xport` style XML format, for
80154
every VM resident on the particular host that is being queried. To
81155
differentiate which column in the export is associated with which VM,
82156
the `legend` field is prefixed with the UUID of the VM.
83157

158+
An example `rrd_updates` output:
159+
160+
```xml
161+
<xport>
162+
<meta>
163+
<start>1213578000</start>
164+
<step>3600</step>
165+
<end>1213617600</end>
166+
<rows>12</rows>
167+
<columns>12</columns>
168+
<legend>
169+
<entry>AVERAGE:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu1</entry> <!-- nb - each data source might have multiple entries for different consolidation functions -->
170+
<entry>AVERAGE:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu0</entry>
171+
<entry>AVERAGE:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:memory</entry>
172+
<entry>MIN:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu1</entry>
173+
<entry>MIN:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu0</entry>
174+
<entry>MIN:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:memory</entry>
175+
<entry>MAX:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu1</entry>
176+
<entry>MAX:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu0</entry>
177+
<entry>MAX:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:memory</entry>
178+
<entry>LAST:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu1</entry>
179+
<entry>LAST:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:cpu0</entry>
180+
<entry>LAST:vm:ecd8d7a0-1be3-4d91-bd0e-4888c0e30ab3:memory</entry>
181+
</legend>
182+
</meta>
183+
<data>
184+
<row>
185+
<t>1213617600</t>
186+
<v>0.0</v> <!-- once again, the order or the columns is defined by the legend above -->
187+
<v>0.0282</v>
188+
<v>209715200.0000</v>
189+
<v>0.0</v>
190+
<v>0.0201</v>
191+
<v>209715200.0000</v>
192+
<v>0.0</v>
193+
<v>0.0445</v>
194+
<v>209715200.0000</v>
195+
<v>0.0</v>
196+
<v>0.0243</v>
197+
<v>209715200.0000</v>
198+
</row>
199+
...
200+
</data>
201+
</xport>
202+
```
203+
204+
84205
To obtain host updates too, use the query parameter `host=true`:
85206

86-
http://:@/rrd_updates?start=&host=true
207+
```sh
208+
wget http://user:password@host/rrd_updates?start=0&host=true
209+
```
87210

88211
The step will decrease as the period decreases, which means that if you
89212
request statistics for a shorter time period you will get more detailed
90213
statistics.
91214

92-
**Additional rrd\_updates parameters**
93-
94-
cf=ave|min|max
95-
: the data consolidation mode
96-
97-
interval=interval
98-
: the interval between values to be reported
99-
100-
> **Note**
101-
>
102-
> By default only `ave` statistics are available. To obtain `min` and
103-
> `max` statistics for a VM, run the following command:
104-
>
105-
> xe pool-param-set uuid= other-config:create_min_max_in_new_VM_RRDs
215+
To download updates containing only the averages, or minimums or maximums,
216+
add the parameter `cf=ave|min|max` e.g.
106217

107-
To obtain all statistics for a host:
218+
```sh
219+
wget http://user:password@host/rrd_updates?start=0&cf=max
220+
```
108221

109-
http:///host_rrd
222+
To request a different update interval, add the parameter `interval=seconds` e.g.
110223

111-
To obtain all statistics for a VM:
112224

113-
http:///vm_rrd?uuid=
225+
```sh
226+
wget http://user:password@host/rrd_updates?start=0&interval=5
227+
```

0 commit comments

Comments
 (0)