Speed Up Python Dependency Management with uv

Manage Python dependencies in Kestra using uv.

uv is a fast Python package and project manager written in Rust. It combines tools like virtualenv, poetry, and pip into one unified interface and is pre-installed in Kestra’s default Python image kestrapy.

uv is useful in Kestra for managing virtual environments with the Process Task Runner or when you need explicit control over dependency resolution speed.

Install Dependencies

By default, Kestra has uv installed to our default Python image kestrapy, so anytime you use a Python Commands or Script task with the Docker Task Runner, it will be preinstalled.

If you’re using a different image or you’d prefer to use the Process Task Runner, you can also install uv using beforeCommands with pip install uv.

id: python_example
namespace: company.team

tasks:
  - id: code_process
    type: io.kestra.plugin.scripts.python.Script
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    beforeCommands:
      - pip install uv 2> /dev/null
    script: |
      print("Hello, World!")

By default, uv will look for a virtual environment to install dependencies into, but this is not required when using the Docker Task Runner as that provides the isolation we would get from a virtual environment. To override this, we can add the --system flag to our install command.

id: python_example
namespace: company.team

tasks:
  - id: code
    type: io.kestra.plugin.scripts.python.Script
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
    beforeCommands:
      - uv pip install pandas --system 2> /dev/null
    script: |
      import pandas as pd
      from kestra import Kestra

      df = pd.read_csv('https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv')
      total_revenue = df['total'].sum()
      Kestra.outputs({"total": total_revenue})

If you’re using the Process Task Runner, you can use uv to create a virtual environment with uv venv. Once this has completed, you can run uv pip install, and it will automatically install these dependencies to this virtual environment without needing to activate the virtual environment.

id: python_example
namespace: company.team

tasks:
  - id: code_process
    type: io.kestra.plugin.scripts.python.Script
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    beforeCommands:
      - pip install uv 2> /dev/null
      - uv venv 2> /dev/null
      - uv pip install pandas kestra 2> /dev/null
      - . .venv/bin/activate
    script: |
      import pandas as pd
      from kestra import Kestra

      df = pd.read_csv('https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv')
      total_revenue = df['total'].sum()
      Kestra.outputs({"total": total_revenue})

Install with a custom Docker image

If you have multiple workflows using uv, you can install it on the Kestra server by creating a custom Docker image for Kestra. Here’s an example of a Dockerfile which is based off the Kestra image but installs uv on top of it.

FROM kestra/kestra:latest
USER root
RUN pip install uv
CMD ["server", "standalone"]

Learn more about installing pip package dependencies at server startup.