Compare Standalone, Medium, and High-Availability Deployments
For the complete documentation index, see llms.txt. For a full content snapshot, see llms-full.txt. Append .md to any kestra.io/docs/* URL for plain Markdown.

Kestra is a Java application distributed as an executable. It supports multiple deployment options:

Kestra’s plugin system allows you to choose the dependency types that best match your requirements.

Small-sized deployment

Kestra Standalone Architecture

For small-scale deployments, you can use the Kestra standalone server, which runs all server components as threads inside a single process. This architecture has no scaling capability but behaves identically to a distributed cluster — the same components run, collocated in one JVM.

For quick local experimentation, Kestra also offers a local mode (server local) that reduces the footprint further: it uses an embedded H2 database with no external dependencies, requiring no infrastructure setup.

In standalone mode, a database is the only dependency. Supported databases include:

  • PostgreSQL
  • MySQL
  • H2

Medium-sized deployment

Kestra Architecture

For medium-scale deployments where high availability is not required, Kestra can be run with a relational database (PostgreSQL or MySQL) as the only dependency. H2 is not recommended in distributed setups.

  • Supported databases: PostgreSQL and MySQL
  • All server components communicate through the database queue
  • Each server role runs as its own process and can be scaled independently
  • Workers communicate with the Worker Controller via gRPC; they never access the queue or database directly

If components are distributed across multiple hosts, use a shared internal storage implementation such as Google Cloud Storage, AWS S3, or Azure Blob Storage.

High-availability deployment

Kestra High Availability Architecture

For high throughput and full horizontal and vertical scaling, replace the database queue with Kafka and Elasticsearch. This architecture removes single points of failure and enables scaling of all server components.

As with medium deployments, a distributed internal storage solution is required if components run on different hosts.

Kafka

Kafka is the queue backbone of the high-availability deployment. The Executor, Scheduler, Worker Controller, Webserver, and Indexer emit to and subscribe from named Kafka topics — no two roles call each other directly.

Workers do not subscribe to Kafka topics. They connect to the Worker Controller via gRPC, and all job dispatch, result intake, and broadcast events travel over that stream.

Executors scale horizontally — each instance subscribes to the queue and processes the executions assigned to it. Because the executor performs lightweight orchestration work (state transitions, dispatch decisions), it typically requires minimal resources.

Elasticsearch

Elasticsearch acts as the search and read backend for Kestra’s webserver, providing fast retrieval and aggregation of flows, executions, and logs. It is used exclusively by the API and UI.

The Indexer subscribes to Kafka topics and writes to Elasticsearch, keeping the search index in sync. Because the queue and search index are separate, executions continue processing even if Elasticsearch is temporarily unavailable.

Was this page helpful?