Azure Blob

Connect to your Azure Blob containers.

Getting Started

Prerequisites to create an Azure Blob based workflow. You will need

  1. A connection to Azure Blob.

  2. A source container.

  3. A destination container. This can be the same as your source container.

Configuring an Azure Blob Connection

Azure Blob related actions require creating an azure connection. The connection must be configured with the correct permissions for each Gretel Action.

For specific permissions, please refer to the Minimum Permissions section under each corresponding action.

There are three ways to authenticate a Gretel Azure Blob Connection, each method requires different fields for connection creation:

Account Access Key

Connection Creation Parameters

name

Display name of your choosing used to identify your connection within Gretel.

account_name

Name of the Storage Account.

default_container

Default container to crawl data from. Different containers can be chosen at the azure_source and azure_destination actions.

First, create a file on your local computer containing the connection credentials. This file should also include type, name , config, and credentials. connection_target_type is optional; if omitted, the connection can be used for both source and destination action. The config and credentials fields should contain fields that are specific to the connection being created.

Below is an example Azure Blob connection using access key credentials:

{
    "type": "azure",
    "name": "my-azure-connection",
    "connection_target_type": "source"
    "config": {
        "account_name": "mystorageaccount",
        "default_container": "mycontainer",
    },
    "credentials": {
        "access_key": "..."
    }
}

Now that you've created the credentials file, use the CLI to create the connection

gretel connections create --project [project id] --from-file [credential_file.json]

Entra ID

name

Display name of your choosing used to identify your connection within Gretel.

account_name

Name of the Storage Account.

client_id

Application (client) ID.

tenant_id

Directory (tenant) ID.

username

Email of the Service Account.

entra_password

Password of the Service Account.

default_container

Default container to crawl data from. Different containers can be chosen at the azure_source and azure_destination actions.

First, create a file on your local computer containing the connection credentials. This file should also include type, name , config, and credentials. connection_target_type is optional; if omitted, the connection can be used for both source and destination action. The config and credentials fields should contain fields that are specific to the connection being created.

Below is an example Azure Blob connection using access key credentials:

{
    "type": "azure",
    "name": "my-azure-connection",
    "connection_target_type": "source"
    "config": {
        "account_name": "mystorageaccount",
        "default_container": "mycontainer",
        "entra_config": {
            "client_id": "12a345b6-1a23-1ab2-abc1-1ab234cde56f",
            "tenant_id": "78g901h2-7g89-7gh8-ghi7-7gh890ijk12l",
            "username": "[email protected]",
    },
    "credentials": {
        "entra_password": "..."
    }
}

Now that you've created the credentials file, use the CLI to create the connection

gretel connections create --project [project id] --from-file [credential_file.json]

SAS Token

name

Display name of your choosing used to identify your connection within Gretel.

account_name

Name of the Storage Account.

default_container

Default container to crawl data from. Different containers can be chosen at the azure_source and azure_destination actions.

First, create a file on your local computer containing the connection credentials. This file should also include type, name , config, and credentials. connection_target_type is optional; if omitted, the connection can be used for both source and destination action. The config and credentials fields should contain fields that are specific to the connection being created.

Below is an example Azure Blob connection file using access key credentials:

{
    "type": "azure",
    "name": "my-azure-connection",
    "connection_target_type": "source"
    "config": {
        "account_name": "mystorageaccount",
        "default_container": "mycontainer",
    },
    "credentials": {
        "sas_token": "..."
    }
}

Now that you've created the credentials file, use the CLI to create the connection

gretel connections create --project [project id] --from-file [credential_file.json]

Last updated

Was this helpful?