Database

Connect Gretel to database management systems.

Gretel workflows support connecting to the following databases:

• MySQL

• PostgreSQL

• MS SQL Server

• Oracle Database

Gretel database connectors can be used in Gretel Workflows with the gretel_tabular action to operationalize synthetic data into your data pipeline.

Overview

For the source database connection, we recommend using a backup or clone with read-only permissions, instead of connecting directly to your production database.

Do not use your input database connection as an output connector. This action can result in the unintended overwriting of existing data.

When reading from a database connector, the source action can extract:

• an entire database, OR

• selected tables from a database, OR

• the results of SQL query/queries against a database.

Each time the workflow is run the source action will extract the most recent data from the source database.

When combined in a workflow, the data extracted from the {database}_source action is used to train models and generate synthetic data with the gretel_tabular action, and can be written to an output database a {database}_destination action.

Limitations

• Gretel database connectors can be used in Workflows with the gretel_tabular action type. They are not compatible with gretel_model.

• Destination database must exist and contain placeholder tables/schema

• While referential integrity can be maintained up to a certain extent, this functionality works best on single tables and we recommend processing either individual tables or views.