Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/plugins/codecs/multiline.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ Available configuration options:
|=======================================================================
|Setting |Input type|Required|Default value
| <<plugins-codecs-multiline-charset>> |<<string,string>>, one of `["ASCII-8BIT", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "US-ASCII", "UTF-8", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "GB2312", "IBM437", "IBM737", "IBM775", "CP850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "IBM860", "IBM861", "IBM862", "IBM863", "IBM864", "IBM865", "IBM866", "IBM869", "Windows-1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "CP951", "stateless-ISO-2022-JP", "eucJP-ms", "CP51932", "GB12345", "ISO-2022-JP", "ISO-2022-JP-2", "CP50220", "CP50221", "Windows-1252", "Windows-1250", "Windows-1256", "Windows-1253", "Windows-1255", "Windows-1254", "TIS-620", "Windows-874", "Windows-1257", "Windows-31J", "MacJapanese", "UTF-7", "UTF8-MAC", "UTF-16", "UTF-32", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "BINARY", "CP437", "CP737", "CP775", "IBM850", "CP857", "CP860", "CP861", "CP862", "CP863", "CP864", "CP865", "CP866", "CP869", "CP1258", "Big5-HKSCS:2008", "eucJP", "euc-jp-ms", "eucKR", "eucTW", "EUC-CN", "eucCN", "CP936", "ISO2022-JP", "ISO2022-JP2", "ISO8859-1", "CP1252", "ISO8859-2", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "CP1256", "ISO8859-7", "CP1253", "ISO8859-8", "CP1255", "ISO8859-9", "CP1254", "ISO8859-10", "ISO8859-11", "CP874", "ISO8859-13", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "CP65000", "CP65001", "UTF-8-MAC", "UTF-8-HFS", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "external", "locale"]`|No|`"UTF-8"`
| <<plugins-codecs-multiline-max_bytes>> |<<bytes,bytes>>|No|`"10 MiB"`
| <<plugins-codecs-multiline-max_lines>> |<<number,number>>|No|`500`
| <<plugins-codecs-multiline-multiline_tag>> |<<string,string>>|No|`"multiline"`
| <<plugins-codecs-multiline-negate>> |<<boolean,boolean>>|No|`false`
| <<plugins-codecs-multiline-pattern>> |<<string,string>>|Yes|
Expand Down Expand Up @@ -129,6 +131,28 @@ or in another character set other than `UTF-8`.

This only affects "plain" format logs since JSON is `UTF-8` already.

[[plugins-codecs-multiline-max_bytes]]
===== `max_bytes`

* Value type is <<bytes,bytes>>
* Default value is `"10 MiB"`

The accumulation of events can make logstash exit with an out of memory error
if event boundaries are not correctly defined. This settings make sure to flush
multiline events after reaching a number of bytes, it is used in combination
max_lines.

[[plugins-codecs-multiline-max_lines]]
===== `max_lines`

* Value type is <<number,number>>
* Default value is `500`

The accumulation of events can make logstash exit with an out of memory error
if event boundaries are not correctly defined. This settings make sure to flush
multiline events after reaching a number of lines, it is used in combination
max_bytes.

[[plugins-codecs-multiline-multiline_tag]]
===== `multiline_tag`

Expand Down
14 changes: 7 additions & 7 deletions docs/plugins/codecs/s3_plain.asciidoc
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
[[plugins-codecs-s3_plain]]
=== s3_plain


NOTE: This is a community-maintained plugin! It does not ship with Logstash by default, but it is easy to install by running `bin/plugin install logstash-codec-s3_plain`.


The "s3_plain" codec is used for backward compatibility with previous version of the S3 Output


&nbsp;

==== Synopsis

This plugin supports the following configuration options:
This plugin has no configuration options.


Required configuration options:
Complete configuration example:

[source,json]
--------------------------
s3_plain {
}
}
--------------------------



==== Details

&nbsp;

2 changes: 1 addition & 1 deletion docs/plugins/filters/date.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ This is useful in case the time zone cannot be extracted from the value,
and is not the platform default.
If this is not specified the platform default will be used.
Canonical ID is good as it takes care of daylight saving time for you
For example, `America/Los_Angeles` or `Europe/France` are valid IDs.
For example, `America/Los_Angeles` or `Europe/Paris` are valid IDs.

[[plugins-filters-date-type]]
===== `type` (DEPRECATED)
Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/filters/json.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -194,15 +194,15 @@ The configuration for the JSON filter:
[source,ruby]
source => source_field

For example, if you have JSON data in the @message field:
For example, if you have JSON data in the `message` field:
[source,ruby]
filter {
json {
source => "message"
}
}

The above would parse the json from the @message field
The above would parse the json from the `message` field

[[plugins-filters-json-tags]]
===== `tags` (DEPRECATED)
Expand Down
10 changes: 9 additions & 1 deletion docs/plugins/filters/mutate.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,15 @@ Convert a field's value to a different type, like turning a string to an
integer. If the field value is an array, all members will be converted.
If the field is a hash, no action will be taken.

Valid conversion targets are: integer, float, string.
If the conversion type is `boolean`, the acceptable values are:

* **True:** `true`, `t`, `yes`, `y`, and `1`
* **False:** `false`, `f`, `no`, `n`, and `0`

If a value other than these is provided, it will pass straight through
and log a warning message.

Valid conversion targets are: integer, float, string, and boolean.

Example:
[source,ruby]
Expand Down
10 changes: 10 additions & 0 deletions docs/plugins/filters/split.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Available configuration options:
| <<plugins-filters-split-periodic_flush>> |<<boolean,boolean>>|No|`false`
| <<plugins-filters-split-remove_field>> |<<array,array>>|No|`[]`
| <<plugins-filters-split-remove_tag>> |<<array,array>>|No|`[]`
| <<plugins-filters-split-target>> |<<string,string>>|No|
| <<plugins-filters-split-terminator>> |<<string,string>>|No|`"\n"`
|=======================================================================

Expand Down Expand Up @@ -202,6 +203,15 @@ would remove a sad, unwanted tag as well.
Only handle events with all of these tags.
Optional.

[[plugins-filters-split-target]]
===== `target`

* Value type is <<string,string>>
* There is no default value for this setting.

The field within the new event which the value is split into.
If not set, target field defaults to split field name.

[[plugins-filters-split-terminator]]
===== `terminator`

Expand Down
1 change: 1 addition & 0 deletions docs/plugins/filters/translate.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ For example, if we have configured `fallback => "no match"`, using this dictiona
Then, if logstash received an event with the field `foo` set to `bar`, the destination
field would be set to `bar`. However, if logstash received an event with `foo` set to `nope`,
then the destination field would still be populated, but with the value of `no match`.
This configuration can be dynamic and include parts of the event using the `%{field}` syntax.

[[plugins-filters-translate-field]]
===== `field`
Expand Down
18 changes: 9 additions & 9 deletions docs/plugins/inputs/couchdb_changes.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@



This CouchDB input allows you to automatically stream events from the
This CouchDB input allows you to automatically stream events from the
CouchDB http://guide.couchdb.org/draft/notifications.html[_changes] URI.
Moreover, any "future" changes will automatically be streamed as well making it easy to synchronize
your CouchDB data with any target destination

### Upsert and delete
You can use event metadata to allow for document deletion.
You can use event metadata to allow for document deletion.
All non-delete operations are treated as upserts

### Starting at a Specific Sequence
Expand Down Expand Up @@ -149,7 +149,7 @@ The format of input data (plain, json, json_event)

Logstash connects to CouchDB's _changes with feed=continuous
The heartbeat is how often (in milliseconds) Logstash will ping
CouchDB to ensure the connection is maintained. Changing this
CouchDB to ensure the connection is maintained. Changing this
setting is not recommended unless you know what you are doing.

[[plugins-inputs-couchdb_changes-host]]
Expand All @@ -166,7 +166,7 @@ IP or hostname of your CouchDB instance
* Value type is <<boolean,boolean>>
* Default value is `true`

Future feature! Until implemented, changing this from the default
Future feature! Until implemented, changing this from the default
will not do anything.

Ignore attachments associated with CouchDB documents.
Expand All @@ -181,7 +181,7 @@ If unspecified, Logstash will attempt to read the last sequence number
from the `sequence_path` file. If that is empty or non-existent, it will
begin with 0 (the beginning).

If you specify this value, it is anticipated that you will
If you specify this value, it is anticipated that you will
only be doing so for an initial read under special circumstances
and that you will unset this value afterwards.

Expand Down Expand Up @@ -215,7 +215,7 @@ will cause unexpected results.
* Value type is <<password,password>>
* Default value is `nil`

Password, if authentication is needed to connect to
Password, if authentication is needed to connect to
CouchDB

[[plugins-inputs-couchdb_changes-port]]
Expand Down Expand Up @@ -283,11 +283,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand All @@ -297,7 +297,7 @@ when sent to another Logstash server.
* Value type is <<string,string>>
* Default value is `nil`

Username, if authentication is needed to connect to
Username, if authentication is needed to connect to
CouchDB


4 changes: 2 additions & 2 deletions docs/plugins/inputs/elasticsearch.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -297,11 +297,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/inputs/eventlog.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -142,11 +142,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/inputs/exec.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -153,11 +153,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
21 changes: 15 additions & 6 deletions docs/plugins/inputs/file.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Available configuration options:
|Setting |Input type|Required|Default value
| <<plugins-inputs-file-add_field>> |<<hash,hash>>|No|`{}`
| <<plugins-inputs-file-codec>> |<<codec,codec>>|No|`"plain"`
| <<plugins-inputs-file-delimiter>> |<<string,string>>|No|`"\n"`
| <<plugins-inputs-file-discover_interval>> |<<number,number>>|No|`15`
| <<plugins-inputs-file-exclude>> |<<array,array>>|No|
| <<plugins-inputs-file-path>> |<<array,array>>|Yes|
Expand Down Expand Up @@ -94,13 +95,21 @@ The codec used for input data. Input codecs are a convenient method for decoding



[[plugins-inputs-file-delimiter]]
===== `delimiter`

* Value type is <<string,string>>
* Default value is `"\n"`

set the new line delimiter, defaults to "\n"

[[plugins-inputs-file-discover_interval]]
===== `discover_interval`

* Value type is <<number,number>>
* Default value is `15`

How often we expand globs to discover new files to watch.
How often (in seconds) we expand globs to discover new files to watch.

[[plugins-inputs-file-exclude]]
===== `exclude`
Expand Down Expand Up @@ -197,9 +206,9 @@ has no effect.
* Value type is <<number,number>>
* Default value is `1`

How often we stat files to see if they have been modified. Increasing
this interval will decrease the number of system calls we make, but
increase the time to detect new log lines.
How often (in seconds) we stat files to see if they have been modified.
Increasing this interval will decrease the number of system calls we make,
but increase the time to detect new log lines.

[[plugins-inputs-file-tags]]
===== `tags`
Expand All @@ -222,11 +231,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/inputs/ganglia.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -144,11 +144,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/inputs/gelf.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -178,11 +178,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
7 changes: 3 additions & 4 deletions docs/plugins/inputs/generator.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,7 @@ This can help with processing later.
* Value type is <<number,number>>
* Default value is `1`

Set this to the number of threads you want this input to spawn.
This is the same as declaring the input multiple times


[[plugins-inputs-generator-type]]
===== `type`
Expand All @@ -189,11 +188,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/inputs/github.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -165,11 +165,11 @@ Add a `type` field to all events handled by this input.
Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can
also use the type to search for it in the web interface.
also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for
example when you send an event from a shipper to an indexer) then
a new input will not override the existing type. A type set at
a new input will not override the existing type. A type set at
the shipper stays with that event for its life even
when sent to another Logstash server.

Expand Down
Loading