Azure Create

Azure Create

Certified
Enterprise Edition

Create and run a job on Azure Batch

Submits a job with its tasks to an Azure Batch pool and waits for completion.

yaml
type: io.kestra.plugin.ee.azure.batch.job.Create

Run a command on Azure Batch

yaml
id: azure_batch_job
namespace: company.team

tasks:
  - id: create
    type: io.kestra.plugin.ee.azure.batch.job.Create
    endpoint: "https://yourbatch.region.batch.azure.com"
    account: "yourbatch"
    accessKey: "{{ secret('AZURE_BATCH_ACCESS_KEY') }}"
    poolId: "kestra-pool"
    job:
      id: my-job
    tasks:
      - id: echo
        commands:
          - 'echo "Hello from Azure Batch"'
Properties

Service endpoint URL

Base HTTPS endpoint of the target service.

The job to create

Definitions
id*Requiredstring
Max length64

Job ID

Unique within the Batch account (<=64 chars, alphanumeric, hyphen, underscore); case-insensitive uniqueness

displayNamestring
Max length1024

Display name

Optional friendly name (up to 1024 Unicode chars); not required to be unique

labelsobject

Job labels

Key/value metadata applied to the job

maxParallelTasksintegerstring

Max parallel tasks

Set to -1 for unlimited (default) or a positive integer; controls concurrent task scheduling

priorityintegerstring

Job priority

Integer from -1000 to 1000; defaults to 0

The ID of the pool

The list of tasks to be run

Definitions
commands*Requiredarray
SubTypestring

Commands

Command lines executed inside the interpreter; rendered with expressions then joined and passed to interpreter/args

id*Requiredstring
Max length64

Task ID

Unique within the job (<=64 chars, alphanumeric, hyphen, underscore); case-insensitive; generated if omitted

constraints

Execution constraints

maxTaskRetryCountintegerstring

Max retry count

Number of retries after non-zero exit: 0 disables retries; -1 retries indefinitely; default is Azure Batch default

maxWallClockTimestring

Max wall clock time

Duration limit from task start; Batch terminates the task if it exceeds this. Null means no limit.

retentionTimestring

Retention time

Minimum time to keep the task working directory after completion; default is 7 days

containerSettings

Container settings

Container image and run options; required if the pool is container-enabled, omit otherwise

imageName*Requiredstring

Container image

Full image reference (e.g. repo/name: tag); defaults to : latest if no tag provided

containerRunOptionsstring

Container run options

Extra arguments passed to docker create alongside Batch-managed options

registry

Container registry

Registry credentials; omit when provided at pool creation

identityReference

User-assigned identity for ACR

Use managed identity instead of username/password when supported by the registry

passwordstring

Registry password

registryServerstring

Registry server

Container registry hostname; defaults to docker.io

userNamestring

Registry username

workingDirectorystring
Possible Values
TASK_WORKING_DIRECTORYCONTAINER_IMAGE_DEFAULT

Working directory scope

Where the container starts; defaults to TASK_WORKING_DIRECTORY. Options: TASK_WORKING_DIRECTORY or CONTAINER_IMAGE_DEFAULT.

displayNamestring
Max length1024

Display name

Optional friendly name (<=1024 Unicode chars); not required to be unique

environmentsobject
SubTypestring

Environment variables

Key/value environment map rendered before execution

interpreterstring
Default/bin/sh

Command interpreter

Shell used to run the task; defaults to /bin/sh

interpreterArgsarray
SubTypestring
Default[ "-c" ]

Interpreter arguments

Arguments prepended to the command; defaults to ['-c']

outputDirsarray
SubTypestring

Output directories

Directories to sync to internal storage; use {{ outputDirs.key }} to write files under that directory

outputFilesarray
SubTypestring

Output files

Names of files to upload from the working directory to internal storage; use {{ outputFiles.key }} to write paths

requiredSlotsintegerstring

Required scheduling slots

Number of slots needed on a node; default 1 and must remain 1 for multi-instance tasks

resourceFilesarray

Resource files

Files staged to the node before execution; subject to Azure Batch request size limits

autoStorageContainerNamestring

Auto storage container name

Mutually exclusive with storageContainerUrl and httpUrl; one of the three is required

blobPrefixstring

Blob prefix filter

Filters blobs when using autoStorageContainerName or storageContainerUrl; downloads only names starting with this prefix

fileModestring

File mode (octal)

Linux-only permission bits; defaults to 0770 on Linux; ignored on Windows nodes

filePathstring

Target path on node

Destination relative to task working dir; required for httpUrl (includes filename); optional dir for container downloads

httpUrlstring

HTTP/HTTPS file URL

Direct file URL; mutually exclusive with storageContainerUrl and autoStorageContainerName; must be readable from compute nodes

identityReference

User-assigned identity reference

Identity used to access storage when using storageContainerUrl or httpUrl

resourceIdstring

User-assigned identity resource ID

ARM resource ID of the managed identity used for storage access

storageContainerUrlstring

Blob container URL

SAS/identity/public URL to a container; mutually exclusive with autoStorageContainerName and httpUrl; must allow read+list from compute nodes

uploadFilesarray

Upload files

Files Azure Batch uploads from the node after execution; for multi-instance tasks only the primary node uploads

destination*Required

Upload destination

container*Required

Azure Blob destination

filePatternstring

File pattern to upload

Supports absolute or task-relative paths with wildcards (*, **, ?, [set]); env vars expanded before matching; dotfiles require explicit match

uploadOptions
Default{ "uploadCondition": "taskcompletion" }

Upload options

Controls when the upload runs and other transfer settings

uploadConditionstring
Defaulttaskcompletion
Possible Values
TASK_SUCCESSTASK_FAILURETASK_COMPLETION

Upload condition

When to upload files (e.g., TASK_COMPLETION, TASK_SUCCESS); defaults to TASK_COMPLETION

DefaultPT1S

The frequency with which the task checks whether the job is completed

Defaulttrue

Whether the job should be deleted upon completion

Warning, if the job is not deleted, a retry of the task could resume an old failed attempt of the job.

The maximum total wait duration

If null, there is no timeout and the task is delegated to Azure Batch.

Reference (ref) of the pluginDefaults to apply to this task.

Defaulttrue

Whether to reconnect to the current job if it already exists

Defaultfalse

Whether to synchronize working directory from remote runner back to local one after run.

SubTypestring

The output files' URIs in Kestra's internal storage

The values from the output of the commands

More Plugins in this Category

VMware (EE)
VMware
Enterprise

Tasks that automate VMware vSphere operations within Kestra workflows.

Infrastructure

33 tasks

3 blueprints

Canonical (EE)
Canonical
Enterprise

Tasks to interact with Canonical services, enabling automation and infrastructure management through API-driven workflows.

CloudInfrastructure

5 tasks