Google Cloud Create

Google Cloud Create

Certified

Create a Dataproc cluster

Creates a Dataproc cluster with optional zone, image, machine shapes, disk sizes, worker count, and staging bucket. Uses project credentials; endpoint derived from region.

yaml
type: io.kestra.plugin.gcp.dataproc.clusters.Create

Creates a cluster in Google Cloud Dataproc

yaml
id: gcp_dataproc_cluster_create
namespace: company.team

tasks:
  - id: cluster_create
    type: io.kestra.plugin.gcp.dataproc.clusters.Create
    clusterName: YOUR_CLUSTER_NAME
    region: YOUR_REGION
    zone: YOUR_ZONE
    masterMachineType: n1-standard-2
    workerMachineType: n1-standard-2
    workers: 2
    bucket: YOUR_BUCKET_NAME

Creates a cluster in Google Cloud Dataproc with specific disk size

yaml
id: gcp_dataproc_cluster_create
namespace: company.team

tasks:
  - id: create_cluster_with_certain_disk_size
    type: io.kestra.plugin.gcp.dataproc.clusters.Create
    clusterName: YOUR_CLUSTER_NAME
    region: YOUR_REGION
    zone: YOUR_ZONE
    masterMachineType: n1-standard-2
    masterDiskSizeGB: 500
    workerMachineType: n1-standard-2
    workerDiskSizeGB: 200
    workers: 2
    bucket: YOUR_BUCKET_NAME
Properties
Min length1

Cluster name

Min length1

Region

Dataproc region endpoint (e.g., europe-west3)

Bucket

Optional GCS bucket for cluster staging

Dataproc image version

Compute Engine image for instances (e.g., 2.1-debian12)

The GCP service account to impersonate

Master disk size (GB)

Master machine type

Reference (ref) of the pluginDefaults to apply to this task.

The GCP project ID

SubTypestring
Default["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used

The GCP service account

Worker disk size (GB)

Worker machine type

Number of workers

Zone

Optional compute zone URI

The cluster name

Defaultfalse

Whether cluster was created successfully

More Plugins in this Category

VMware (EE)
VMware
Enterprise

Tasks that automate VMware vSphere operations within Kestra workflows.

Infrastructure

33 tasks

3 blueprints

Canonical (EE)
Canonical
Enterprise

Tasks to interact with Canonical services, enabling automation and infrastructure management through API-driven workflows.

CloudInfrastructure

5 tasks