LogoLogo
  • Welcome to Gretel!
  • Gretel Basics
    • Getting Started
      • Quickstart
      • Blueprints
      • Use Case Examples
      • Environment Setup
        • Console
        • SDK
      • Projects
      • Inputs and Outputs
      • Gretel Connectors
        • Object Storage
          • Amazon S3
          • Google Cloud Storage
          • Azure Blob
        • Database
          • MySQL
          • PostgreSQL
          • MS SQL Server
          • Oracle Database
        • Data Warehouse
          • Snowflake
          • BigQuery
          • Databricks
        • Gretel Project
    • Release Notes
      • Platform Release Notes
        • May 2025
        • April 2025
        • March 2025
        • February 2025
        • January 2025
        • December 2024
        • November 2024
        • October 2024
        • September 2024
        • August 2024
        • July 2024
        • June 2024
      • Console Release Notes
        • January 2025
        • December 2024
        • November 2024
        • October 2024
        • September 2024
        • August 2024
      • Python SDKs
  • Create Synthetic Data
    • Gretel Safe Synthetics
      • Transform
        • Reference
        • Examples
        • Supported Entities
      • Synthetics
        • Gretel Tabular Fine-Tuning
        • Gretel Text Fine-Tuning
        • Gretel Tabular GAN
        • Benchmark Report
        • Privacy Protection
      • Evaluate
        • Synthetic Quality & Privacy Report
        • Tips to Improve Synthetic Data Quality
        • Data Privacy 101
      • SDK
    • Gretel Data Designer
      • Getting Started with Data Designer
      • Define your Data Columns
        • Column Types
        • Add Constraints to Columns
        • Custom Model Configurations
        • Upload Files as Seeds
      • Building your Dataset
        • Seeding your Dataset
        • Generating Data
      • Generate Realistic Personal Details
      • Structured Outputs
      • Code Validation
      • Data Evaluation
      • Magic Assistance
      • Using Jinja Templates
  • Gretel Playground [Legacy]
    • Getting Started
    • Prompts Tips & Best Practices
    • FAQ
    • SDK Examples
    • Tutorials
    • Videos
    • Gretel Playground [Legacy] Inference API
    • Batch Job SDK
  • Reference
    • Gretel's Python Client
    • Gretel’s Open Source Synthetic Engine
    • Gretel’s REST API
    • Homepage
    • Model Suites
Powered by GitBook
On this page
  • Supported Connectors
  • Object Storage
  • Relational Databases
  • Data Warehouses
  • Managing Connections
  • Permissions and Connection Sharing
  • Creating Connections
  • Updating Connections
  • Deleting Connections

Was this helpful?

Export as PDF
  1. Gretel Basics
  2. Getting Started

Gretel Connectors

Integrate Gretel with your existing data services using Workflows.

PreviousInputs and OutputsNextObject Storage

Last updated 27 days ago

Was this helpful?

With Gretel Workflows, you can train and run one or more Gretel Models by connecting directly to your data sources and destinations. We support the following integrations for data inputs and outputs.

Supported Connectors

Object Storage

Relational Databases

Data Warehouses

Managing Connections

Connections are used to authenticate Workflow Actions. Each action is compatible with a specific type of connection. For example, the s3_source action requires a s3 connection.

Permissions and Connection Sharing

When creating a connection, you must select a project for the connection to reside in. The connection will inherit all the project permissions and user memberships.

Project Permission
Connection Permissions

Read

  • Users can list and view connection metadata.

Write

  • Users can access connections in a Workflow.

Administrator

  • Users can create, edit and delete connections.

Owner

  • Full control.

Connections can be reused across Workflows, actions, and projects.

Creating Connections

  • Navigate to the Connections page using the menu item in the left sidebar.

  • Click the New Connection button in the top right corner.

  • Select the project where the connection will be stored.

  • Next, fill in your credentials and select Add Connection. The example below shows the flow for creating an Amazon S3 connection. All connections follow the same pattern.

  • The connection can now be used in a workflow.

  • Select the option to connect to an external data source, and choose the connection you created above, or create a new one.

from gretel_client import create_or_get_unique_project
from gretel_client.config import get_session_config
from gretel_client.rest_v1.api.connections_api import ConnectionsApi
from gretel_client.rest_v1.models import (
    CreateConnectionRequest,
    UpdateConnectionRequest,
)

session = get_session_config()
connection_api = session.get_v1_api(ConnectionsApi)

project = create_or_get_unique_project(name="workflow-testing")

connection = connection_api.create_connection(
    CreateConnectionRequest(
        name="my-s3-conn",
        project_id=project.project_guid,
        type="s3",
        # note: best practice is to read in credentials from a file
        # or secret instead of directly embedding sensitive values
        # in python code.
        config={
            "access_key_id": "...",
        },
        credentials={
            "secret_access_key": "...",
        },
    )
)

Updating Connections

Connection types cannot be changed once a connection is created.

  • Navigate to the Connections page using the menu item in the left sidebar.

  • Go to the Connection you'd like to update and click the three vertical dots (aka overflow actions).

  • Select Update Connection.

  • Modify the name and/or credentials and select Save. All workflows that use this connection will automatically use the new information.

If you only want to edit the connection name, leave the credentials fields as they are.

Expanding on the example from Creating Connections

connection = connection_api.update_connection(
    connection_id=connection.id,
    update_connection_request=UpdateConnectionRequest(
        name="my-s3-conn",
        config={
            "access_key_id": "...",
        },
        credentials={
            "secret_access_key": "...",
        },
    )
)

Deleting Connections

  • On the Connections list page, select the three dots (aka overflow actions) to the right of the connection you want to delete.

  • Select Delete

Expanding on the example from Creating Connections

connection_api.delete_connection(connection_id=connection.id)

Select the .

Data sources also can be configured during a blueprint flow. Go to the or page and select a use case, for example "Generate synthetic data". This will start a guided flow to help you create a workflow.

Amazon S3
Google Cloud Storage
Azure Blob Storage
MySQL
PostgreSQL
Microsoft SQL Server
Oracle Database
Snowflake
Google BigQuery
Databricks
Dashboard
Blueprints
Connector Type
Create a connection in the Gretel Console
Create a new connection during a workflow
Update a connection