Elasticsearch Load
CertifiedBulk load from Kestra storage
Elasticsearch Load
Bulk load from Kestra storage
Reads ION-serialized records from a Kestra internal storage file and indexes them in bulk. Uses the parent chunk size; set removeIdKey to keep or drop the id field after use.
type: io.kestra.plugin.elasticsearch.LoadExamples
id: elasticsearch_load
namespace: company.team
inputs:
- id: file
type: FILE
tasks:
- id: load
type: io.kestra.plugin.elasticsearch.Load
connection:
hosts:
- "http://localhost:9200"
from: "{{ inputs.file }}"
index: "my_index"
Properties
connection *RequiredNon-dynamic
Elasticsearch connection
Connection settings shared by tasks; hosts are required.
io.kestra.plugin.elasticsearch.ElasticsearchConnection
1Elasticsearch hosts
List of HTTP(S) endpoints including scheme and port, e.g. https://elasticsearch.com: 9200; at least one is required.
Basic authentication
Optional HTTP basic auth credentials rendered at runtime.
io.kestra.plugin.elasticsearch.ElasticsearchConnection-BasicAuth
Basic auth password
Password for HTTP basic authentication.
Basic auth username
Username for HTTP basic authentication.
Custom HTTP headers
Headers sent on every request in Name: Value format, e.g. Authorization: Token XYZ.
Request path prefix
Base path prepended to every Elasticsearch endpoint, e.g. /my/path. Use only when the cluster is served behind a proxy that requires a prefix; leave empty otherwise.
Fail on warning headers
When true, any response containing Elasticsearch warning headers is treated as an error.
8Target Elasticsearch server major version
Major version used for compatibility headers (Accept and Content-Type). Set to 8 for Elasticsearch 8 clusters or 9 for Elasticsearch 9 clusters.
Trust all SSL certificates
Skips certificate validation for HTTPS connections; use only with self-signed certificates in non-production.
from *Requiredstring
Source file
Kestra internal storage URI containing bulk payload; supports dynamic rendering.
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
index *Requiredstring
The Elasticsearch index
chunk integerstring
1000Bulk chunk size
Number of operations per bulk request; default 1000.
idKey string
Field used as document id
Name of the field to use as _id; required when assigning ids from input rows.
opType string
INDEXCREATEUPDATEDELETEOperation type
Intended bulk op type; currently only index is applied.
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
removeIdKey booleanstring
trueRemove id field from document
When true (default), drops the idKey field from the indexed document body.
routing string
Custom shard routing
Optional routing key hashed to pick the shard instead of using the document id.
Outputs
size integer
Indexed document count
Total operations sent across all bulk requests.
Metrics
records counter
recordsNumber of records loaded
requests.count counter
Number of bulk requests sent
requests.duration timer
Duration of bulk requests
