Azure Create
CertifiedRun tasks in an Azure Batch job
Azure Create
Run tasks in an Azure Batch job
Creates or resumes a job on a target pool, uploads task definitions, waits for completion, and optionally deletes the job afterwards. Defaults: resume=true, delete=true, completionCheckInterval=PT1S; set maxDuration to cap wait time.
type: io.kestra.plugin.azure.batch.job.CreateExamples
id: azure_batch_job_create
namespace: company.team
tasks:
- id: create
type: io.kestra.plugin.azure.batch.job.Create
endpoint: https://my.francecentral.batch.azure.com
account: "{{ secret('AZURE_ACCOUNT') }}"
accessKey: "{{ secret('AZURE_ACCESS_KEY') }}"
poolId: "{{ secret('AZURE_POOL_ID') }}"
job:
id: myjob
tasks:
- id: env
commands:
- 'echo t1=$ENV_STRING'
environments:
ENV_STRING: "{{ inputs.first }}"
- id: echo
commands:
- 'echo t2={{ inputs.second }} 1>&2'
- id: for
commands:
- 'for i in $(seq 10); do echo t3=$i; done'
- id: vars
commands:
- echo '::{"outputs":{"extract":"'$(cat files/in/in.txt)'"}::'
resourceFiles:
- httpUrl: https://unittestkt.blob.core.windows.net/tasks/***?sv=***&se=***&sr=***&sp=***&sig=***
filePath: files/in/in.txt
- id: output
commands:
- 'mkdir -p outs/child/sub'
- 'echo 1 > outs/1.txt'
- 'echo 2 > outs/child/2.txt'
- 'echo 3 > outs/child/sub/3.txt'
outputFiles:
- outs/1.txt
outputDirs:
- outs/child
Use a container to start the task, the pool must use a microsoft-azure-batch publisher.
id: azure_batch_job_create
namespace: company.team
tasks:
- id: create
type: io.kestra.plugin.azure.batch.job.Create
endpoint: https://***.francecentral.batch.azure.com
account: "{{ secret('AZURE_ACCOUNT') }}"
accessKey: "{{ secret('AZURE_ACCESS_KEY') }}"
poolId: <pool-id>
job:
id: <job-name>
tasks:
- id: echo
commands:
- 'python --version'
containerSettings:
imageName: python
Properties
accessKey *Requiredstring
account *Requiredstring
endpoint *Requiredstring
Service endpoint URL
Base HTTPS endpoint of the target service.
job *Required
Job definition
Job metadata and scheduling limits passed to Azure
poolId *Requiredstring
Pool ID
Existing Batch pool where the job runs; must be active
tasks *RequiredNon-dynamicarray
Tasks to run
Ordered list of task definitions executed within the job
completionCheckInterval string
PT1SCompletion check interval
Poll interval for job status while waiting; defaults to PT1S
delete booleanstring
trueDelete job on completion
If true (default), the job is deleted after tasks finish; keep false to inspect failures but retries may reuse the old job
maxDuration string
Maximum wait duration
Timeout while waiting for all tasks to finish; null means wait indefinitely
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
resume booleanstring
trueResume existing job if found
When true (default), reuses a matching in-progress job instead of creating a new one
syncWorkingDirectory booleanstring
falseWhether to synchronize working directory from remote runner back to local one after run.
Outputs
outputFiles object
Output files stored in internal storage
vars object
Captured outputs from task logs
Metrics
cpu.kernel.duration timer
The CPU kernel time.
cpu.user.duration timer
The CPU user time.
io.read.gib.count counter
The number of gibibytes read from I/O by the task.
io.read.ops.count counter
The number of read I/O operations performed by the task.
io.write.gib.count counter
The number of gibibytes written to I/O by the task.
io.write.ops.count counter
The number of write I/O operations performed by the task.
wall.clock.duration timer
The wall clock time.
