Azure Reads
CertifiedRead all files from an Azure Data Lake Storage directory
Azure Reads
Read all files from an Azure Data Lake Storage directory
Read all files from an Azure Data Lake Storage directory using the Azure SDK.
type: io.kestra.plugin.azure.storage.adls.ReadsExamples
id: azure_storage_datalake_readq
namespace: company.team
tasks:
- id: read_file
type: io.kestra.plugin.azure.storage.adls.Reads
connectionString: "{{ secret('AZURE_CONNECTION_STRING') }}"
fileSystem: "tasks"
endpoint: "https://yourblob.blob.core.windows.net"
directoryPath: "path/to/my/directory/"
Properties
directoryPath *Requiredstring
Directory Name
endpoint *Requiredstring
Service endpoint URL
Base HTTPS endpoint of the target service.
fileSystem *Requiredstring
The name of the file systems. If the path name contains special characters, pass in the url encoded version of the path name
connectionString string
Connection string of the Storage Account.
failOnMissingChecksum booleanstring
Fail when the server has no Content-MD5
Only applies when validateChecksum is true. If true, the task fails when the Azure object has no stored Content-MD5. If false (default), validation is skipped with a warning.
maxFiles integerstring
25The maximum number of files to read
Limits the number of files read. If not specified, all matching files will be read.
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
sasToken string
The SAS token to use for authenticating requests.
This string should only be the query parameters (with or without a leading '?') and not a full URL.
validateChecksum booleanstring
Validate checksum against server's Content-MD5
If true, compute the MD5 of the downloaded file and compare it to the Content-MD5 stored on the Azure object. Many block blobs uploaded as streams have no Content-MD5; see failOnMissingChecksum to control that case.
Outputs
files array
The list of files
io.kestra.plugin.azure.storage.adls.models.AdlsFile
com.azure.storage.file.datalake.models.ArchiveStatus
com.azure.storage.file.datalake.models.AccessTier
date-timedate-timeINFINITEFIXEDAVAILABLELEASEDEXPIREDBREAKINGBROKENLOCKEDUNLOCKEDurioutputFiles object
The downloaded files as a map of from/to URIs
