From 84265b1f014d8376c9c2a20208a3e5d548f7436e Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:19:25 +0100 Subject: [PATCH 1/7] Updated to include new input plugin doc for Windows System Statistics. Applies to #2139. Signed-off-by: Eric D. Schabell --- SUMMARY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/SUMMARY.md b/SUMMARY.md index 12245f3ba..fe8ecb414 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -125,6 +125,7 @@ * [Windows Event logs (winlog)](pipeline/inputs/windows-event-log.md) * [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md) * [Windows exporter metrics](pipeline/inputs/windows-exporter-metrics.md) + * [Windows System Statistics (winstat)](pipeline/inputs/windows-system-statistics.md) * [Parsers](pipeline/parsers.md) * [Configuring parsers](pipeline/parsers/configuring-parser.md) * [Decoders](pipeline/parsers/decoders.md) From 8b9641dec0b3d64aa6d3faf9d80564a96331d824 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:20:15 +0100 Subject: [PATCH 2/7] Updated to include new input plugin doc for Windows System Statistics. Applies to #2139. Signed-off-by: Eric D. Schabell --- pipeline/inputs/windows-system-statistics.md | 99 ++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 pipeline/inputs/windows-system-statistics.md diff --git a/pipeline/inputs/windows-system-statistics.md b/pipeline/inputs/windows-system-statistics.md new file mode 100644 index 000000000..8a5344ee2 --- /dev/null +++ b/pipeline/inputs/windows-system-statistics.md @@ -0,0 +1,99 @@ +# Windows System Statistics (winstat) + +The _Windows System Statistics_ (`winstat`) input plugin collects system-level statistics from Windows environments, including CPU usage, memory consumption, disk I/O, and network activity. This plugin uses Windows Performance Counters to gather real-time system metrics. + +{% hint style="info" %} + +This plugin is only available on Windows operating systems and requires appropriate permissions to access Windows Performance Counters. + +{% endhint %} + +## Configuration parameters + +The plugin supports the following configuration parameters: + +| Key | Description | Default | +|:----------------|:--------------------------------------------------------------------------------------------------------|:--------| +| `Interval_Sec` | Polling interval in seconds. | `1` | +| `Interval_NSec` | Polling interval in nanoseconds. | `0` | +| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` | + +## Metrics collected + +The `winstat` plugin collects the following system statistics: + +| Metric Category | Description | +|:----------------|:----------------------------------------------------------------------------| +| CPU | CPU usage percentage, including user and system time | +| Memory | Memory usage including total, available, and used memory | +| Disk | Disk I/O statistics including read/write operations and throughput | +| Network | Network interface statistics including bytes sent/received and packet counts | + +## Get started + +To collect Windows system statistics, you can run the plugin from the command line or through the configuration file: + +### Command line + +You can run the plugin from the command line: + +```shell +fluent-bit -i winstat -o stdout +``` + +Which returns information similar to the following: + +```text +... +[0] winstat: [1699123456.123456789, {"cpu.usage"=>15.3, "cpu.user"=>10.2, "cpu.system"=>5.1, "memory.total"=>8192, "memory.available"=>4096, "memory.used"=>4096, "disk.read_bytes"=>1024000, "disk.write_bytes"=>512000, "network.bytes_sent"=>2048000, "network.bytes_recv"=>1024000}] +[1] winstat: [1699123457.123456789, {"cpu.usage"=>16.1, "cpu.user"=>11.0, "cpu.system"=>5.1, "memory.total"=>8192, "memory.available"=>4080, "memory.used"=>4112, "disk.read_bytes"=>1025000, "disk.write_bytes"=>515000, "network.bytes_sent"=>2050000, "network.bytes_recv"=>1025000}] +... +``` + +### Configuration file + +In your main configuration file append the following: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: winstat + tag: winstat + interval_sec: 1 + interval_nsec: 0 + + outputs: + - name: stdout + match: '*' +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name winstat + Tag winstat + Interval_Sec 1 + Interval_NSec 0 + +[OUTPUT] + Name stdout + Match * +``` + +{% endtab %} +{% endtabs %} + +Total interval (sec) = `Interval_Sec` + (`Interval_NSec` / 1000000000) + +For example: `1.5s` = `1s` + `500000000ns` + +## Notes + +- The `winstat` plugin requires Windows Performance Counters access. Ensure Fluent Bit is running with appropriate permissions. +- This plugin is Windows-only and will not work on Linux, macOS, or other operating systems. +- For Prometheus-based metrics collection on Windows, consider using the [Windows Exporter Metrics](windows-exporter-metrics.md) input plugin instead. From 2bdfa9a414523e8b9e71df7c719f3fa71d838189 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:27:37 +0100 Subject: [PATCH 3/7] Lint fix for winstat input plugin doc fix. Signed-off-by: Eric D. Schabell --- pipeline/inputs/windows-system-statistics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pipeline/inputs/windows-system-statistics.md b/pipeline/inputs/windows-system-statistics.md index 8a5344ee2..963bf5c58 100644 --- a/pipeline/inputs/windows-system-statistics.md +++ b/pipeline/inputs/windows-system-statistics.md @@ -1,4 +1,4 @@ -# Windows System Statistics (winstat) +# Windows system statistics (winstat) The _Windows System Statistics_ (`winstat`) input plugin collects system-level statistics from Windows environments, including CPU usage, memory consumption, disk I/O, and network activity. This plugin uses Windows Performance Counters to gather real-time system metrics. @@ -95,5 +95,5 @@ For example: `1.5s` = `1s` + `500000000ns` ## Notes - The `winstat` plugin requires Windows Performance Counters access. Ensure Fluent Bit is running with appropriate permissions. -- This plugin is Windows-only and will not work on Linux, macOS, or other operating systems. +- This plugin is Windows-only and won't work on Linux, macOS, or other operating systems. - For Prometheus-based metrics collection on Windows, consider using the [Windows Exporter Metrics](windows-exporter-metrics.md) input plugin instead. From 027c91307520f00c46b029209c9a8f7c4d82ed49 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:48:12 +0100 Subject: [PATCH 4/7] Added new Plot output plugin doc. Applies to #2142. Signed-off-by: Eric D. Schabell --- pipeline/outputs/plot.md | 131 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 pipeline/outputs/plot.md diff --git a/pipeline/outputs/plot.md b/pipeline/outputs/plot.md new file mode 100644 index 000000000..ab7a54bfc --- /dev/null +++ b/pipeline/outputs/plot.md @@ -0,0 +1,131 @@ +--- +description: Generate data file for GNU Plot +--- + +# Plot + +The _Plot_ output plugin generates data files in a format compatible with [GNU Plot](http://www.gnuplot.info/) (gnuplot), a command-line graphing tool. This plugin allows you to export your telemetry data for visualization and analysis using gnuplot. + +## Configuration parameters + +This plugin supports the following parameters: + +| Key | Description | Default | +|:--- |:----------- |:------- | +| `File` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. If the file cannot be opened, the plugin falls back to writing to STDOUT. | _none_ | +| `Key` | Specify the key name from the record to extract as the value. The value must be a numeric type (integer or float). If not specified, the plugin uses the first field from the record. | _none_ | + +## Output format + +The Plot output plugin generates data files in a format suitable for gnuplot. The output format is space-separated values with two columns: timestamp and value. + +The output format is: + +```text +timestamp value +``` + +Where: +- `timestamp` is a floating-point Unix timestamp +- `value` is the numeric value extracted from the specified key (or the first field if `Key` is not specified) + +The plugin only supports numeric values (integers or floats). If the specified key is not found or the value is not numeric, an error is logged and the record is skipped. + +## Get started + +You can run the plugin from the command line or through the configuration file. + +### Command line + +From the command line you can generate plot data files with the following options: + +```shell +fluent-bit -i cpu -o plot -p file=cpu_data.dat -p key=cpu_p +``` + +This example extracts the `cpu_p` field from CPU metrics and writes timestamp-value pairs to `cpu_data.dat`. + +### Configuration file + +In your main configuration file append the following: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: cpu + tag: cpu + + outputs: + - name: plot + match: '*' + file: cpu_data.dat + key: cpu_p +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name cpu + Tag cpu + +[OUTPUT] + Name plot + Match * + File cpu_data.dat + Key cpu_p +``` + +{% endtab %} +{% endtabs %} + +## Example usage with gnuplot + +After generating the data file with Fluent Bit, you can use gnuplot to visualize the data: + +1. Generate the data file: + +```shell +fluent-bit -i cpu -o plot -p file=cpu_data.dat -p key=cpu_p -f 1 +``` + +This command collects CPU metrics, extracts the `cpu_p` field (CPU percentage), and writes timestamp-value pairs to `cpu_data.dat`. The output file will contain lines like: + +```text +1704067200.123456 25.5 +1704067201.123456 30.2 +1704067202.123456 28.7 +``` + +2. Create a gnuplot script (e.g., `plot.gp`): + +```text +set terminal png +set output "cpu_usage.png" +set xlabel "Time" +set ylabel "CPU Usage (%)" +set xdata time +set timefmt "%s" +set format x "%H:%M:%S" +plot "cpu_data.dat" using 1:2 with lines title "CPU Usage" +``` + +3. Run gnuplot: + +```shell +gnuplot plot.gp +``` + +This will generate a PNG image file showing the CPU usage over time. + +## Notes + +- The `Key` parameter is optional. If not specified, the plugin uses the first field from the record. +- Only numeric values (integers or floats) are supported. Non-numeric values will cause the record to be skipped with an error logged. +- If the specified `Key` is not found in a record, an error is logged and that record is skipped. +- If the output file cannot be opened (e.g., due to permissions), the plugin automatically falls back to writing to STDOUT. +- The output file is opened in append mode, so new data is added to existing files. From e4dc7ed02b55abf05e8a3fbd99e8a72a136ca99c Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:48:41 +0100 Subject: [PATCH 5/7] Added new entry for Plot output plugin doc. Applies to #2142. Signed-off-by: Eric D. Schabell --- SUMMARY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/SUMMARY.md b/SUMMARY.md index fe8ecb414..53d543d06 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -205,6 +205,7 @@ * [OpenTelemetry](pipeline/outputs/opentelemetry.md) * [Output Formats](pipeline/outputs/output_formats.md) * [Parseable](pipeline/outputs/parseable.md) + * [Plot](pipeline/outputs/plot.md) * [PostgreSQL](pipeline/outputs/postgresql.md) * [Prometheus exporter](pipeline/outputs/prometheus-exporter.md) * [Prometheus remote write](pipeline/outputs/prometheus-remote-write.md) From 6cca26cd4d6ccdf9855325f55b87c1387061cc44 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:54:19 +0100 Subject: [PATCH 6/7] Fixed linting issues for plot output plugin doc. Signed-off-by: Eric D. Schabell --- pipeline/outputs/plot.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/pipeline/outputs/plot.md b/pipeline/outputs/plot.md index ab7a54bfc..165457b29 100644 --- a/pipeline/outputs/plot.md +++ b/pipeline/outputs/plot.md @@ -4,7 +4,7 @@ description: Generate data file for GNU Plot # Plot -The _Plot_ output plugin generates data files in a format compatible with [GNU Plot](http://www.gnuplot.info/) (gnuplot), a command-line graphing tool. This plugin allows you to export your telemetry data for visualization and analysis using gnuplot. +The _Plot_ output plugin generates data files in a format compatible with [GNU Plot](http://www.gnuplot.info/) (`gnuplot`), a command-line graphing tool. This plugin allows you to export your telemetry data for visualization and analysis using `gnuplot`. ## Configuration parameters @@ -17,7 +17,7 @@ This plugin supports the following parameters: ## Output format -The Plot output plugin generates data files in a format suitable for gnuplot. The output format is space-separated values with two columns: timestamp and value. +The Plot output plugin generates data files in a format suitable for `gnuplot`. The output format is space-separated values with two columns: timestamp and value. The output format is: @@ -83,9 +83,9 @@ pipeline: {% endtab %} {% endtabs %} -## Example usage with gnuplot +## Example usage with `gnuplot` -After generating the data file with Fluent Bit, you can use gnuplot to visualize the data: +After generating the data file with Fluent Bit, you can use `gnuplot` to visualize the data: 1. Generate the data file: @@ -101,7 +101,7 @@ This command collects CPU metrics, extracts the `cpu_p` field (CPU percentage), 1704067202.123456 28.7 ``` -2. Create a gnuplot script (e.g., `plot.gp`): +2. Create a `gnuplot` script (e.g., `plot.gp`): ```text set terminal png @@ -114,7 +114,7 @@ set format x "%H:%M:%S" plot "cpu_data.dat" using 1:2 with lines title "CPU Usage" ``` -3. Run gnuplot: +3. Run `gnuplot`: ```shell gnuplot plot.gp @@ -127,5 +127,5 @@ This will generate a PNG image file showing the CPU usage over time. - The `Key` parameter is optional. If not specified, the plugin uses the first field from the record. - Only numeric values (integers or floats) are supported. Non-numeric values will cause the record to be skipped with an error logged. - If the specified `Key` is not found in a record, an error is logged and that record is skipped. -- If the output file cannot be opened (e.g., due to permissions), the plugin automatically falls back to writing to STDOUT. +- If the output file can't be opened (for example, due to permissions), the plugin automatically falls back to writing to STDOUT. - The output file is opened in append mode, so new data is added to existing files. From 7184d48d4883f0b838b0d7d059dfb2c6da5d831a Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 24 Nov 2025 12:56:50 +0100 Subject: [PATCH 7/7] Fixed linting issues for plot output plugin doc. Signed-off-by: Eric D. Schabell --- pipeline/outputs/plot.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/plot.md b/pipeline/outputs/plot.md index 165457b29..27183b26f 100644 --- a/pipeline/outputs/plot.md +++ b/pipeline/outputs/plot.md @@ -101,7 +101,7 @@ This command collects CPU metrics, extracts the `cpu_p` field (CPU percentage), 1704067202.123456 28.7 ``` -2. Create a `gnuplot` script (e.g., `plot.gp`): +2. Create a `gnuplot` script (for example, `plot.gp`): ```text set terminal png