Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Sync with latest code comments
  • Loading branch information
suyograo committed May 7, 2015
commit fae0b343c52453c62d3b5cf12a479bfce3ea24e1
58 changes: 47 additions & 11 deletions docs/plugins/outputs/elasticsearch.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ With the default `protocol` setting ("node"), this plugin will join your
Elasticsearch cluster as a client node, so it will show up in Elasticsearch's
cluster status.

You can learn more about Elasticsearch at <http://www.elasticsearch.org>
You can learn more about Elasticsearch at <https://www.elastic.co/products/elasticsearch>

## Operational Notes
==== Operational Notes

If using the default `protocol` setting ("node"), your firewalls might need
to permit port 9300 in *both* directions (from Logstash to Elasticsearch, and
to permit port `9300` in *both* directions (from Logstash to Elasticsearch, and
Elasticsearch to Logstash)

&nbsp;
Expand Down Expand Up @@ -61,15 +61,16 @@ Available configuration options:
| <<plugins-outputs-elasticsearch-cacert>> |a valid filesystem path|No|
| <<plugins-outputs-elasticsearch-cluster>> |<<string,string>>|No|
| <<plugins-outputs-elasticsearch-codec>> |<<codec,codec>>|No|`"plain"`
| <<plugins-outputs-elasticsearch-document_id>> |<<string,string>>|No|`nil`
| <<plugins-outputs-elasticsearch-discovery>> |<<string,string>>, one of `["zen", "ec2"]`|No|`"zen"`
| <<plugins-outputs-elasticsearch-document_id>> |<<string,string>>|No|
| <<plugins-outputs-elasticsearch-document_type>> |<<string,string>>|No|
| <<plugins-outputs-elasticsearch-ec2_protocol>> |<<string,string>>, one of `["http", "https"]`|No|
| <<plugins-outputs-elasticsearch-embedded>> |<<boolean,boolean>>|No|`false`
| <<plugins-outputs-elasticsearch-embedded_http_port>> |<<string,string>>|No|`"9200-9300"`
| <<plugins-outputs-elasticsearch-flush_size>> |<<number,number>>|No|`5000`
| <<plugins-outputs-elasticsearch-host>> |<<array,array>>|No|
| <<plugins-outputs-elasticsearch-idle_flush_time>> |<<number,number>>|No|`1`
| <<plugins-outputs-elasticsearch-index>> |<<string,string>>|No|`"logstash-%{+YYYY.MM.dd}"`
| <<plugins-outputs-elasticsearch-index_type>> |<<string,string>>|No|
| <<plugins-outputs-elasticsearch-manage_template>> |<<boolean,boolean>>|No|`true`
| <<plugins-outputs-elasticsearch-max_retries>> |<<number,number>>|No|`3`
| <<plugins-outputs-elasticsearch-node_name>> |<<string,string>>|No|
Expand All @@ -78,6 +79,7 @@ Available configuration options:
| <<plugins-outputs-elasticsearch-protocol>> |<<string,string>>, one of `["node", "transport", "http"]`|No|
| <<plugins-outputs-elasticsearch-retry_max_interval>> |<<number,number>>|No|`5`
| <<plugins-outputs-elasticsearch-retry_max_items>> |<<number,number>>|No|`5000`
| <<plugins-outputs-elasticsearch-routing>> |<<string,string>>|No|
| <<plugins-outputs-elasticsearch-s3_protocol>> |<<string,string>>, one of `["http", "https"]`|No|
| <<plugins-outputs-elasticsearch-sniffing>> |<<boolean,boolean>>|No|`false`
| <<plugins-outputs-elasticsearch-ssl>> |<<boolean,boolean>>|No|`false`
Expand Down Expand Up @@ -208,15 +210,32 @@ for discovery.

The codec used for output data. Output codecs are a convenient method for encoding your data before it leaves the output, without needing a separate filter in your Logstash pipeline.

[[plugins-outputs-elasticsearch-discovery]]
===== `discovery`

* Value can be any of: `zen`, `ec2`
* Default value is `"zen"`



[[plugins-outputs-elasticsearch-document_id]]
===== `document_id`

* Value type is <<string,string>>
* Default value is `nil`
* There is no default value for this setting.

The document ID for the index. Useful for overwriting existing entries in
Elasticsearch with the same ID.

[[plugins-outputs-elasticsearch-document_type]]
===== `document_type`

* Value type is <<string,string>>
* There is no default value for this setting.

The document type to write events to. Generally you should try to write only
similar events to the same 'type'. String expansion `%{foo}` works here.

[[plugins-outputs-elasticsearch-ec2_protocol]]
===== `ec2_protocol`

Expand All @@ -234,7 +253,9 @@ Elasticsearch with the same ID.
Run the Elasticsearch server embedded in this process.
This option is useful if you want to run a single Logstash process that
handles log processing and indexing; it saves you from needing to run
a separate Elasticsearch process.
a separate Elasticsearch process. An example use case is
proof-of-concept testing.
WARNING: This is not recommended for production use!

[[plugins-outputs-elasticsearch-embedded_http_port]]
===== `embedded_http_port`
Expand Down Expand Up @@ -279,7 +300,10 @@ This is only required if the normal multicast/cluster discovery stuff won't
work in your environment.

The plugin will use multicast discovery to connect to Elasticsearch
when using `protocol => node` without setting a host.
when using `protocol => node` without setting a host. When setting unicast
hosts for `node` protocol, it is important to confirm that at least one non-client
node is listed in the `:host` list. client nodes are filtered by default in zen-discovery.
You need to set `discovery.zen.master_election.filter_client` to `false` to override this behavior.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#multicast[Multicast Discovery Docs]

Expand Down Expand Up @@ -315,14 +339,17 @@ Indexes may not contain uppercase characters.
For weekly indexes ISO 8601 format is recommended, eg. logstash-%{+xxxx.ww}

[[plugins-outputs-elasticsearch-index_type]]
===== `index_type`
===== `index_type` (DEPRECATED)

* DEPRECATED WARNING: This configuration item is deprecated and may not be available in future versions.
* Value type is <<string,string>>
* There is no default value for this setting.

The index type to write events to. Generally you should try to write only
similar events to the same 'type'. String expansion `%{foo}` works here.

Deprecated in favor of `document_type` field.

[[plugins-outputs-elasticsearch-manage_template]]
===== `manage_template`

Expand Down Expand Up @@ -431,6 +458,15 @@ Set max interval between bulk retries

Set retry policy for events that failed to send

[[plugins-outputs-elasticsearch-routing]]
===== `routing`

* Value type is <<string,string>>
* There is no default value for this setting.

A routing override to be applied to all processed events.
This can be dynamic using the `%{foo}` syntax.

[[plugins-outputs-elasticsearch-s3_protocol]]
===== `s3_protocol`

Expand All @@ -454,7 +490,7 @@ Asks host for the list of all cluster nodes and adds them to the hosts list
* Value type is <<boolean,boolean>>
* Default value is `false`

SSL Configurations (HTTP only)
SSL Configurations (only valid when protocol is HTTP)

Enable SSL

Expand Down Expand Up @@ -535,7 +571,7 @@ Optional.
* Value type is <<string,string>>
* There is no default value for this setting.

Username and password (HTTP only)
Username and password (only valid when protocol is HTTP; this setting works with HTTP or HTTPS auth)

[[plugins-outputs-elasticsearch-workers]]
===== `workers`
Expand Down