Benchmark Report
Check out this Benchmark report, running Gretel models on popular ML datasets, indexed by industry
You can use a Benchmark report like the one shown here to evaluate which Gretel model is best for your synthetic data goals.
For example, Gretel LSTM consistently generates synthetic data with high Synthetic Data Quality Score (SQS) on multiple types of tabular or complex data. As seen in the results below, Gretel ACTGAN is great for particularly long or wide datasets and has generally a faster runtime. If you’re looking to quickly generate lots of data, Gretel Amplify produces results in 1/10 of the time (check out the fast train and generate times!). Gretel GPT generates high-quality synthetic data for natural language datasets.
Depending on your specific goals with synthetic data or constraints, you may find particular Gretel models to be best suited for your use case. You can reference the Benchmark report below to guide how you evaluate Gretel models, or of course, try Benchmark on your datasets.
The publicly available datasets used in this results leaderboard were sourced from the following ML dataset repositories: UCI, Kaggle, and HuggingFace.
Input Data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
bank_marketing_large/data.csv | GretelAmplify | tabular_mixed | 41188 | 21 | 73 | 36.07 | 29.75 | 65.82 |
bank_marketing_large/data.csv | GretelACTGAN | tabular_mixed | 41188 | 21 | 85 | 1300.75 | 33.24 | 1333.99 |
bank_marketing_large/data.csv | GretelLSTM | tabular_mixed | 41188 | 21 | 84 | 317.79 | 401.04 | 718.83 |
bank_marketing_small/data.csv | GretelAmplify | tabular_mixed | 4521 | 17 | 80 | 24.41 | 23.81 | 48.22 |
bank_marketing_small/data.csv | GretelACTGAN | tabular_mixed | 4521 | 17 | 84 | 169.32 | 175.63 | 344.95 |
bank_marketing_small/data.csv | GretelLSTM | tabular_mixed | 4521 | 17 | 84 | 326.26 | 96.73 | 422.99 |
dow_jones_index/data.csv | GretelAmplify | time_series | 750 | 16 | 76 | 81.5 | 23.32 | 104.82 |
dow_jones_index/data.csv | GretelACTGAN | time_series | 750 | 16 | 70 | 221.58 | 129.15 | 350.73 |
dow_jones_index/data.csv | GretelLSTM | time_series | 750 | 16 | 83 | 424.2 | 64.66 | 488.86 |
Input data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
bike_sales/data.csv | GretelAmplify | tabular_numeric | 16519 | 24 | 79 | 119.94 | 30 | 149.94 |
bike_sales/data.csv | GretelLSTM | tabular_numeric | 16519 | 24 | 88 | 911.59 | 249.68 | 1161.27 |
car_evaluation/data.csv | GretelAmplify | tabular_numeric | 1728 | 7 | 85 | 24.06 | 23.7 | 47.76 |
car_evaluation/data.csv | GretelACTGAN | tabular_numeric | 1728 | 7 | 77 | 201.19 | 44.5 | 245.69 |
car_evaluation/data.csv | GretelLSTM | tabular_numeric | 1728 | 7 | 87 | 357.66 | 54.02 | 411.68 |
olist_order_payments/data.csv | GretelAmplify | tabular_numeric | 103886 | 5 | 69 | 529.06 | 40.41 | 569.47 |
olist_order_payments/data.csv | GretelLSTM | tabular_numeric | 103886 | 5 | 93 | 4201.89 | 897.22 | 5099.11 |
Input data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
data_science_job_candidates/data.csv | GretelAmplify | tabular_mixed | 19158 | 14 | 88 | 107.66 | 23.26 | 130.92 |
data_science_job_candidates/data.csv | GretelACTGAN | tabular_mixed | 19158 | 14 | 90 | 609.02 | 128.11 | 737.13 |
data_science_job_candidates/data.csv | GretelLSTM | tabular_mixed | 19158 | 14 | 93 | 358.21 | 276.29 | 634.5 |
ibm_employee_attrition/data.csv | GretelAmplify | tabular_mixed | 1470 | 37 | 88 | 24.09 | 20.5 | 44.59 |
ibm_employee_attrition/data.csv | GretelACTGAN | tabular_mixed | 1470 | 37 | 80 | 368.13 | 33.93 | 402.06 |
ibm_employee_attrition/data.csv | GretelLSTM | tabular_mixed | 1470 | 37 | 93 | 365.18 | 127.79 | 492.97 |
Input data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
energydata_complete/data.csv | GretelAmplify | time_series | 19735 | 29 | 74 | 103.68 | 33.09 | 136.77 |
energydata_complete/data.csv | GretelLSTM | time_series | 19735 | 29 | 93 | 1531.88 | 400.29 | 1932.17 |
telco_customer_churn/data.csv | GretelAmplify | tabular_mixed | 7043 | 33 | 82 | 40.84 | 30.17 | 71.01 |
telco_customer_churn/data.csv | GretelACTGAN | tabular_mixed | 7043 | 33 | 79 | 5911.34 | 55.36 | 5966.7 |
telco_customer_churn/data.csv | GretelLSTM | tabular_mixed | 7043 | 33 | 76 | 787.7 | 155.55 | 943.25 |
Input data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
air_quality_uci/data.csv | GretelAmplify | time_series | 9357 | 15 | 65 | 96.15 | 52.47 | 148.62 |
air_quality_uci/data.csv | GretelACTGAN | time_series | 9357 | 15 | 62 | 6656.12 | 54.49 | 6710.61 |
air_quality_uci/data.csv | GretelLSTM | time_series | 9357 | 15 | 89 | 398.42 | 211.66 | 610.08 |
iris/data.csv | GretelAmplify | tabular_numeric | 150 | 5 | 66 | 87.56 | 23.3 | 110.86 |
iris/data.csv | GretelACTGAN | tabular_numeric | 150 | 5 | 51 | 140.46 | 170.43 | 310.89 |
iris/data.csv | GretelLSTM | tabular_numeric | 150 | 5 | 79 | 148.21 | 160.02 | 308.23 |
winequality_red/data.csv | GretelAmplify | tabular_numeric | 1599 | 12 | 82 | 81.94 | 23.68 | 105.62 |
winequality_red/data.csv | GretelACTGAN | tabular_numeric | 1599 | 12 | 61 | 76.14 | 43.87 | 120.01 |
winequality_red/data.csv | GretelLSTM | tabular_numeric | 1599 | 12 | 89 | 221.92 | 54.69 | 276.61 |
winequality_white/data.csv | GretelAmplify | tabular_numeric | 4898 | 12 | 88 | 24.65 | 23.27 | 47.92 |
winequality_white/data.csv | GretelACTGAN | tabular_numeric | 4898 | 12 | 81 | 139.03 | 33.15 | 172.18 |
winequality_white/data.csv | GretelLSTM | tabular_numeric | 4898 | 12 | 91 | 287.84 | 76.4 | 364.24 |
Input data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
portuguese_election_data/data.csv | GretelAmplify | tabular_numeric | 21643 | 28 | 52 | 31.33 | 107.04 | 138.37 |
portuguese_election_data/data.csv | GretelACTGAN | tabular_numeric | 21643 | 28 | 72 | 928.15 | 128.73 | 1056.88 |
portuguese_election_data/data.csv | GretelLSTM | tabular_numeric | 21643 | 28 | 81 | 455.56 | 327.19 | 782.75 |
adult/data.csv | GretelAmplify | tabular_mixed | 32561 | 15 | 85 | 213.54 | 58.46 | 272 |
adult/data.csv | GretelACTGAN | tabular_mixed | 32561 | 15 | 87 | 965.31 | 128.03 | 1093.34 |
adult/data.csv | GretelLSTM | tabular_mixed | 32561 | 15 | 94 | 667.21 | 615.08 | 1282.29 |
Input data | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
|---|---|---|---|---|---|---|---|---|
processed_cleveland_heart_disease_uci/data.csv | GretelAmplify | tabular_numeric | 303 | 14 | 83 | 35.87 | 23.01 | 58.88 |
processed_cleveland_heart_disease_uci/data.csv | GretelACTGAN | tabular_numeric | 303 | 14 | 70 | 66.97 | 33.48 | 100.45 |
processed_cleveland_heart_disease_uci/data.csv | GretelLSTM | tabular_numeric | 303 | 14 | 91 | 221.56 | 54.26 | 275.82 |
breast_cancer_wisconsin/data.csv | GretelAmplify | tabular_numeric | 699 | 11 | 55 | 23.89 | 23.55 | 47.44 |
breast_cancer_wisconsin/data.csv | GretelACTGAN | tabular_numeric | 699 | 11 | 56 | 67.4 | 203.11 | 270.51 |
breast_cancer_wisconsin/data.csv | GretelLSTM | tabular_numeric | 699 | 11 | 83 | 206.88 | 64.73 | 271.61 |
Last modified 2mo ago