Serialization ProtobufToIon

Serialization ProtobufToIon

Certified

Convert a Protobuf file into Amazon ION

The plugin reads one or more Protobuf messages from a binary file or stream, decodes them using a provided descriptor and message type name, and then serializes the result as ION data based on ProtoJSON Format. It requires the following information regarding the Protobuf message:

  • A descriptor file (.desc), generated with --descriptor_set_out, that contains the compiled Protobuf message definitions.
  • A type name (e.g., com.company.Product) corresponding to the root message to decode.

Here's an example of how to generate a descriptor file from a Protobuf file using protoc: 

protoc --proto_path=src/main/proto        --descriptor_set_out=products.desc        src/main/proto/products.proto
yaml
type: io.kestra.plugin.serdes.protobuf.ProtobufToIon

Convert a Protobuf file to the Amazon ION format.

yaml
id: protobuf_to_ion
namespace: company.team

tasks:
  - id: http_download
    type: io.kestra.plugin.core.http.Download
    uri: https://example.com/data/products.pb

  - id: to_ion
    type: io.kestra.plugin.serdes.protobuf.ProtobufToIon
    from: "{{ outputs.http_download.uri }}"
    descriptorFile: "kestra:///path/to/proto.desc"
    typeName: com.company.Product
Properties

Protobuf Descriptor File

Protobuf descriptor file containing message definitions.

Source file URI

The URI of the input Protobuf file to read.

Fully qualified Protobuf message type name

For example: com.company.Product.

Defaultfalse

Is the input a stream of length-delimited messages?

If true, the input file is expected to contain multiple length-delimited Protobuf messages. If false, it will be parsed as a single Protobuf message.

Defaultfalse

Whether to error on unknown fields

If true, an error will be thrown if the input contains unknown fields.

Default0

The number of records converted

Formaturi

URI of a temporary result file

Number of Protobuf messages converted

More Plugins in this Category

Core Plugins and tasks
Core Plugins and tasks

Tasks that provide Kestra's built-in orchestration, I/O, and observability capabilities.

Core

109 tasks

580 blueprints

Ruby
Ruby

Tasks that run Ruby scripts.

Core

4 tasks