LogoLogo
  • Welcome to Gretel!
  • Gretel Basics
    • Getting Started
      • Quickstart
      • Blueprints
      • Use Case Examples
      • Environment Setup
        • Console
        • SDK
      • Projects
      • Inputs and Outputs
      • Gretel Connectors
        • Object Storage
          • Amazon S3
          • Google Cloud Storage
          • Azure Blob
        • Database
          • MySQL
          • PostgreSQL
          • MS SQL Server
          • Oracle Database
        • Data Warehouse
          • Snowflake
          • BigQuery
          • Databricks
        • Gretel Project
    • Release Notes
      • Platform Release Notes
        • May 2025
        • April 2025
        • March 2025
        • February 2025
        • January 2025
        • December 2024
        • November 2024
        • October 2024
        • September 2024
        • August 2024
        • July 2024
        • June 2024
      • Console Release Notes
        • January 2025
        • December 2024
        • November 2024
        • October 2024
        • September 2024
        • August 2024
      • Python SDKs
  • Create Synthetic Data
    • Gretel Safe Synthetics
      • Transform
        • Reference
        • Examples
        • Supported Entities
      • Synthetics
        • Gretel Tabular Fine-Tuning
        • Gretel Text Fine-Tuning
        • Gretel Tabular GAN
        • Benchmark Report
        • Privacy Protection
      • Evaluate
        • Synthetic Quality & Privacy Report
        • Tips to Improve Synthetic Data Quality
        • Data Privacy 101
      • SDK
    • Gretel Data Designer
      • Getting Started with Data Designer
      • Define your Data Columns
        • Column Types
        • Add Constraints to Columns
        • Custom Model Configurations
        • Upload Files as Seeds
      • Building your Dataset
        • Seeding your Dataset
        • Generating Data
      • Generate Realistic Personal Details
      • Structured Outputs
      • Code Validation
      • Data Evaluation
      • Magic Assistance
      • Using Jinja Templates
  • Gretel Playground [Legacy]
    • Getting Started
    • Prompts Tips & Best Practices
    • FAQ
    • SDK Examples
    • Tutorials
    • Videos
    • Gretel Playground [Legacy] Inference API
    • Batch Job SDK
  • Reference
    • Gretel's Python Client
    • Gretel’s Open Source Synthetic Engine
    • Gretel’s REST API
    • Homepage
    • Model Suites
Powered by GitBook
On this page
  • Getting Started
  • Configuring an Azure Blob Connection
  • Account Access Key
  • Entra ID
  • SAS Token

Was this helpful?

Export as PDF
  1. Gretel Basics
  2. Getting Started
  3. Gretel Connectors
  4. Object Storage

Azure Blob

Connect to your Azure Blob containers.

Getting Started

Prerequisites to create an Azure Blob based workflow. You will need

  1. A connection to Azure Blob.

  2. A source container.

  3. A destination container. This can be the same as your source container.

Configuring an Azure Blob Connection

Azure Blob related actions require creating an azure connection. The connection must be configured with the correct permissions for each Gretel Action.

For specific permissions, please refer to the Minimum Permissions section under each corresponding action.

There are three ways to authenticate a Gretel Azure Blob Connection, each method requires different fields for connection creation:

  1. Account Access Key

  2. #entra-id-account-active-directory

  3. SAS Token

Account Access Key

Connection Creation Parameters

name

Display name of your choosing used to identify your connection within Gretel.

account_name

Name of the Storage Account.

access_key

default_container

Default container to crawl data from. Different containers can be chosen at the azure_source and azure_destination actions.

First, create a file on your local computer containing the connection credentials. This file should also include type, name , config, and credentials. connection_target_type is optional; if omitted, the connection can be used for both source and destination action. The config and credentials fields should contain fields that are specific to the connection being created.

Below is an example Azure Blob connection using access key credentials:

{
    "type": "azure",
    "name": "my-azure-connection",
    "connection_target_type": "source"
    "config": {
        "account_name": "mystorageaccount",
        "default_container": "mycontainer",
    },
    "credentials": {
        "access_key": "..."
    }
}

Now that you've created the credentials file, use the CLI to create the connection

gretel connections create --project [project id] --from-file [credential_file.json]
  • Click the New Connection button.

  • Step 1, choose the Type for the Connection - Azure Blob.

  • Step 2, choose the Project for your Connection.

  • Step 3, fill in the credentials and select Add Connection.

from gretel_client import create_or_get_unique_project
from gretel_client.config import get_session_config
from gretel_client.rest_v1.api.connections_api import ConnectionsApi
from gretel_client.rest_v1.models import (
    CreateConnectionRequest,
    UpdateConnectionRequest,
)

session = get_session_config()
connection_api = session.get_v1_api(ConnectionsApi)

project = create_or_get_unique_project(name="azure-workflow")

connection = connection_api.create_connection(
    CreateConnectionRequest(
        name="my-azure-connection",
        project_id=project.project_guid,
        type="azure",
        #connection_target_type="",
        config={
            "account_name": "mystorageaccount",
            "default_container": "mycontainer",
        },
        # note: best practice is to read in credentials from a file
        # or secret instead of directly embedding sensitive values
        # in python code.
        credentials={
            "access_key": "...",
        },
    )
)

Entra ID

name

Display name of your choosing used to identify your connection within Gretel.

account_name

Name of the Storage Account.

client_id

Application (client) ID.

tenant_id

Directory (tenant) ID.

username

Email of the Service Account.

entra_password

Password of the Service Account.

default_container

Default container to crawl data from. Different containers can be chosen at the azure_source and azure_destination actions.

First, create a file on your local computer containing the connection credentials. This file should also include type, name , config, and credentials. connection_target_type is optional; if omitted, the connection can be used for both source and destination action. The config and credentials fields should contain fields that are specific to the connection being created.

Below is an example Azure Blob connection using access key credentials:

{
    "type": "azure",
    "name": "my-azure-connection",
    "connection_target_type": "source"
    "config": {
        "account_name": "mystorageaccount",
        "default_container": "mycontainer",
        "entra_config": {
            "client_id": "12a345b6-1a23-1ab2-abc1-1ab234cde56f",
            "tenant_id": "78g901h2-7g89-7gh8-ghi7-7gh890ijk12l",
            "username": "serviceaccountemail@domain.com",
    },
    "credentials": {
        "entra_password": "..."
    }
}

Now that you've created the credentials file, use the CLI to create the connection

gretel connections create --project [project id] --from-file [credential_file.json]

Console support for creating Azure Blob connections using Entra ID is coming soon. For now, you can create connections using Entra ID via CLI or SDK and then use those connections in Console.

from gretel_client import create_or_get_unique_project
from gretel_client.config import get_session_config
from gretel_client.rest_v1.api.connections_api import ConnectionsApi
from gretel_client.rest_v1.models import (
    CreateConnectionRequest,
    UpdateConnectionRequest,
)

session = get_session_config()
connection_api = session.get_v1_api(ConnectionsApi)

project = create_or_get_unique_project(name="azure-workflow")

connection = connection_api.create_connection(
    CreateConnectionRequest(
        name="my-azure-connection",
        project_id=project.project_guid,
        type="azure",
        #connection_target_type="",
        config={
            "account_name": "mystorageaccount",
            "default_container": "mycontainer",
            "entra_config": {
                "client_id": "12a345b6-1a23-1ab2-abc1-1ab234cde56f",
                "tenant_id": "78g901h2-7g89-7gh8-ghi7-7gh890ijk12l",
                "username": "serviceaccountemail@domain.com",
            },
        },
        # note: best practice is to read in credentials from a file
        # or secret instead of directly embedding sensitive values
        # in python code.
        credentials={
            "entra_password": "...",
        },
    )
)

SAS Token

name

Display name of your choosing used to identify your connection within Gretel.

account_name

Name of the Storage Account.

sas_token

default_container

Default container to crawl data from. Different containers can be chosen at the azure_source and azure_destination actions.

First, create a file on your local computer containing the connection credentials. This file should also include type, name , config, and credentials. connection_target_type is optional; if omitted, the connection can be used for both source and destination action. The config and credentials fields should contain fields that are specific to the connection being created.

Below is an example Azure Blob connection file using access key credentials:

{
    "type": "azure",
    "name": "my-azure-connection",
    "connection_target_type": "source"
    "config": {
        "account_name": "mystorageaccount",
        "default_container": "mycontainer",
    },
    "credentials": {
        "sas_token": "..."
    }
}

Now that you've created the credentials file, use the CLI to create the connection

gretel connections create --project [project id] --from-file [credential_file.json]

Console support for creating Azure Blob connections using SAS Tokens is coming soon. For now, you can create connections using SAS Tokens via CLI or SDK and then use those connections in Console.

from gretel_client import create_or_get_unique_project
from gretel_client.config import get_session_config
from gretel_client.rest_v1.api.connections_api import ConnectionsApi
from gretel_client.rest_v1.models import (
    CreateConnectionRequest,
    UpdateConnectionRequest,
)

session = get_session_config()
connection_api = session.get_v1_api(ConnectionsApi)

project = create_or_get_unique_project(name="azure-workflow")

connection = connection_api.create_connection(
    CreateConnectionRequest(
        name="my-azure-connection",
        project_id=project.project_guid,
        type="azure",
        #connection_target_type="",
        config={
            "account_name": "mystorageaccount",
            "default_container": "mycontainer",
        },
        # note: best practice is to read in credentials from a file
        # or secret instead of directly embedding sensitive values
        # in python code.
        credentials={
            "sas_token": "...",
        },
    )
)
PreviousGoogle Cloud StorageNextDatabase

Last updated 29 days ago

Was this helpful?

Navigate to the using the menu item in the left sidebar.

.

Connections page
Storage Account Access Key.
Shared Access Signatures