AWS Run
CertifiedEnterprise EditionRun a job on AWS Batch
AWS Run
Run a job on AWS Batch
Submits a job to an AWS Batch job queue and waits for it to complete.
type: io.kestra.plugin.ee.aws.batch.RunExamples
Run an existing AWS Batch job definition on an existing queue (pure orchestration)
id: aws_batch_run_existing_job
namespace: company.team
description: >
Orchestrate an existing AWS Batch job definition on an existing job queue (pure orchestration).
No resource creation / modification / deletion.
inputs:
- id: aws_region
type: STRING
defaults: eu-central-1
- id: job_queue_arn
type: STRING
defaults: arn:aws:batch:eu-central-1:123456789012:job-queue/prod-etl-queue
- id: job_definition_arn
type: STRING
defaults: arn:aws:batch:eu-central-1:123456789012:job-definition/prod-etl:42
tasks:
- id: run_batch_job
type: io.kestra.plugin.ee.aws.batch.Run
region: "{{ inputs.aws_region }}"
jobQueue: "{{ inputs.job_queue_arn }}"
jobDefinition: "{{ inputs.job_definition_arn }}"
jobName: "prod-etl-{{ execution.id }}"
parameters:
run_id: "{{ execution.id }}"
environment: "prod"
containerEnvironment:
KESTRA_EXECUTION_ID: "{{ execution.id }}"
wait: true
completionCheckInterval: PT10S
completionCheckMaxInterval: PT1M
completionCheckBackoffMultiplier: 1.5
maxWait: PT2H
failOnFailedStatus: true
streamLogs: true
logGroupName: /aws/batch/job
logStreamPrefix: "prod-etl-"
Properties
jobDefinition *Requiredstring
AWS Batch job definition
Name or ARN (optionally with revision) of the existing AWS Batch job definition.
jobQueue *Requiredstring
AWS Batch job queue
Name or ARN of the existing AWS Batch job queue where the job will be submitted.
region *Requiredstring
AWS region
AWS region where the Batch job will be submitted (for example: eu-central-1).
accessKeyId string
AWS access key ID
AWS access key ID used for authentication. If not set, the default AWS credentials provider chain is used.
completionCheckBackoffMultiplier numberstring
1.5Completion check backoff multiplier
Backoff multiplier applied after each completion check (must be greater than or equal to 1).
completionCheckInterval string
PT10SCompletion check interval
Initial polling interval used to check the job status when waiting for completion.
completionCheckMaxInterval string
PT1MMaximum completion check interval
Upper bound for adaptive polling interval while waiting for completion.
containerCommand array
Container command override
Optional container command override applied through AWS Batch containerOverrides.command.
containerEnvironment object
Container environment overrides
Optional environment variables applied through AWS Batch containerOverrides.environment.
endpointOverride string
AWS endpoint override
Optional custom endpoint for AWS Batch (useful for testing or private endpoints).
failOnFailedStatus booleanstring
trueFail on FAILED job status
Whether the task should fail if the AWS Batch job ends with status FAILED.
jobName string
AWS Batch job name
Optional name for the submitted job. Defaults to a name derived from the Kestra execution ID.
logGroupName string
/aws/batch/jobCloudWatch log group name
CloudWatch Logs group used for live tailing when streamLogs is enabled.
logStreamPrefix string
CloudWatch log stream prefix
CloudWatch Logs stream prefix used for live tailing when streamLogs is enabled. Defaults to jobName.
maxWait string
PT6HMaximum wait duration
Maximum amount of time to wait for the AWS Batch job to reach a terminal state.
parameters object
AWS Batch job parameters
Key-value parameters passed to the job, overriding defaults defined in the job definition.
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
secretKeyId string
AWS secret access key
AWS secret access key used for authentication. If not set, the default AWS credentials provider chain is used.
sessionToken string
AWS session token
Optional AWS session token for temporary credentials.
streamLogs booleanstring
falseStream CloudWatch logs
Whether to stream CloudWatch logs while waiting for completion. Log tailing starts only when wait=true and streamLogs=true. Best effort: failures in log streaming do not fail the task.
stsEndpointOverride string
AWS STS endpoint override
Optional custom endpoint for AWS STS.
stsRoleArn string
AWS STS role ARN
ARN of the AWS IAM role to assume using STS.
stsRoleExternalId string
AWS STS external ID
External ID used when assuming a role in another AWS account.
stsRoleSessionDuration string
AWS STS session duration
Duration of the assumed role session (default: PT15M).
stsRoleSessionName string
AWS STS session name
Session name used when assuming an AWS STS role.
wait booleanstring
trueWait for job completion
Whether the task should wait for the AWS Batch job to complete before finishing.
Outputs
createdAt integer
Job creation time
Timestamp (epoch milliseconds) when the AWS Batch job was created.
jobDefinition string
AWS Batch job definition
Name or ARN (with revision) of the AWS Batch job definition used to run the job.
jobId string
AWS Batch job ID
Unique identifier assigned by AWS Batch to the submitted job.
jobName string
AWS Batch job name
Name of the AWS Batch job as submitted by the task.
jobQueue string
AWS Batch job queue
Name or ARN of the AWS Batch job queue on which the job was submitted.
startedAt integer
Job start time
Timestamp (epoch milliseconds) when the AWS Batch job started running.
status string
Job status
Final or current status of the AWS Batch job (for example: SUBMITTED, RUNNING, SUCCEEDED, FAILED).
statusReason string
Job status reason
Additional information explaining the job status, as reported by AWS Batch.
stoppedAt integer
Job stop time
Timestamp (epoch milliseconds) when the AWS Batch job reached a terminal state.
