Comment on page
Benchmark Report
Check out this Benchmark report, running Gretel models on popular ML datasets, indexed by industry
You can use a Benchmark report like the one shown here to evaluate which Gretel model is best for your synthetic data goals.
For example, Gretel LSTM consistently generates synthetic data with high Synthetic Data Quality Score (SQS) on multiple types of tabular or complex data. As seen in the results below, Gretel ACTGAN is great for particularly long or wide datasets and has generally a faster runtime. If you’re looking to quickly generate lots of data, Gretel Amplify produces results in 1/10 of the time (check out the fast train and generate times!). Gretel GPT generates high-quality synthetic data for natural language datasets.
Depending on your specific goals with synthetic data or constraints, you may find particular Gretel models to be best suited for your use case. You can reference the Benchmark report below to guide how you evaluate Gretel models, or of course, try Benchmark on your datasets.
The publicly available datasets used in this results leaderboard were sourced from the following ML dataset repositories: UCI, Kaggle, and HuggingFace.
Input Data | Size | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
bank_marketing_large/data.csv | 4.9 MB | GretelAmplify | tabular_mixed | 41188 | 21 | 73 | 36.07 | 29.75 | 65.82 |
bank_marketing_large/data.csv | 4.9 MB | GretelACTGAN | tabular_mixed | 41188 | 21 | 85 | 1300.75 | 33.24 | 1333.99 |
bank_marketing_large/data.csv | 4.9 MB | GretelLSTM | tabular_mixed | 41188 | 21 | 84 | 317.79 | 401.04 | 718.83 |
bank_marketing_small/data.csv | 371 KB | GretelAmplify | tabular_mixed | 4521 | 17 | 80 | 24.41 | 23.81 | 48.22 |
bank_marketing_small/data.csv | 371 KB | GretelACTGAN | tabular_mixed | 4521 | 17 | 84 | 169.32 | 175.63 | 344.95 |
bank_marketing_small/data.csv | 371 KB | GretelLSTM | tabular_mixed | 4521 | 17 | 84 | 326.26 | 96.73 | 422.99 |
dow_jones_index/data.csv | 89 KB | GretelAmplify | time_series | 750 | 16 | 76 | 81.5 | 23.32 | 104.82 |
dow_jones_index/data.csv | 89 KB | GretelACTGAN | time_series | 750 | 16 | 70 | 221.58 | 129.15 | 350.73 |
dow_jones_index/data.csv | 89 KB | GretelLSTM | time_series | 750 | 16 | 83 | 424.2 | 64.66 | 488.86 |
Input data | Size | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
bike_sales/data.csv | 2.4 MB | GretelAmplify | tabular_numeric | 16519 | 24 | 79 | 119.94 | 30 | 149.94 |
bike_sales/data.csv | 2.4 MB | GretelLSTM | tabular_numeric | 16519 | 24 | 88 | 911.59 | 249.68 | 1161.27 |
car_evaluation/data.csv | 52 KB | GretelAmplify | tabular_numeric | 1728 | 7 | 85 | 24.06 | 23.7 | 47.76 |
car_evaluation/data.csv | 52 KB | GretelACTGAN | tabular_numeric | 1728 | 7 | 77 | 201.19 | 44.5 | 245.69 |
car_evaluation/data.csv | 52 KB | GretelLSTM | tabular_numeric | 1728 | 7 | 87 | 357.66 | 54.02 | 411.68 |
olist_order_payments/data.csv | 5.6 MB | GretelAmplify | tabular_numeric | 103886 | 5 | 69 | 529.06 | 40.41 | 569.47 |
olist_order_payments/data.csv | 5.6 MB | GretelLSTM | tabular_numeric | 103886 | 5 | 93 | 4201.89 | 897.22 | 5099.11 |
Input data | Size | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
data_science_job_candidates/data.csv | 1.9 MB | GretelAmplify | tabular_mixed | 19158 | 14 | 88 | 107.66 | 23.26 | 130.92 |
data_science_job_candidates/data.csv | 1.9 MB | GretelACTGAN | tabular_mixed | 19158 | 14 | 90 | 609.02 | 128.11 | 737.13 |
data_science_job_candidates/data.csv | 1.9 MB | GretelLSTM | tabular_mixed | 19158 | 14 | 93 | 358.21 | 276.29 | 634.5 |
ibm_employee_attrition/data.csv | 274 KB | GretelAmplify | tabular_mixed | 1470 | 37 | 88 | 24.09 | 20.5 | 44.59 |
ibm_employee_attrition/data.csv | 274 KB | GretelACTGAN | tabular_mixed | 1470 | 37 | 80 | 368.13 | 33.93 | 402.06 |
ibm_employee_attrition/data.csv | 274 KB | GretelLSTM | tabular_mixed | 1470 | 37 | 93 | 365.18 | 127.79 | 492.97 |
Input data | Size | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
energydata_complete/data.csv | 11.4 MB | GretelAmplify | time_series | 19735 | 29 | 74 | 103.68 | 33.09 | 136.77 |
energydata_complete/data.csv | 11.4 MB | GretelLSTM | time_series | 19735 | 29 | 93 | 1531.88 | 400.29 | 1932.17 |
telco_customer_churn/data.csv | 1.7 MB | GretelAmplify | tabular_mixed | 7043 | 33 | 82 | 40.84 | 30.17 | 71.01 |
telco_customer_churn/data.csv | 1.7 MB | GretelACTGAN | tabular_mixed | 7043 | 33 | 79 | 5911.34 | 55.36 | 5966.7 |
telco_customer_churn/data.csv | 1.7 MB | GretelLSTM | tabular_mixed | 7043 | 33 | 76 | 787.7 | 155.55 | 943.25 |
Input data | Text | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
air_quality_uci/data.csv | 822 KB | GretelAmplify | time_series | 9357 | 15 | 65 | 96.15 | 52.47 | 148.62 |
air_quality_uci/data.csv | 822 KB | GretelACTGAN | time_series | 9357 | 15 | 62 | 6656.12 | 54.49 | 6710.61 |
air_quality_uci/data.csv | 822 KB | GretelLSTM | time_series | 9357 | 15 | 89 | 398.42 | 211.66 | 610.08 |
iris/data.csv | 4 KB | GretelAmplify | tabular_numeric | 150 | 5 | 66 | 87.56 | 23.3 | 110.86 |
iris/data.csv | 4 KB | GretelACTGAN | tabular_numeric | 150 | 5 | 51 | 140.46 | 170.43 | 310.89 |
iris/data.csv | 4 KB | GretelLSTM | tabular_numeric | 150 | 5 | 79 | 148.21 | 160.02 | 308.23 |
winequality_red/data.csv | 90 KB | GretelAmplify | tabular_numeric | 1599 | 12 | 82 | 81.94 | 23.68 | 105.62 |
winequality_red/data.csv | 90 KB | GretelACTGAN | tabular_numeric | 1599 | 12 | 61 | 76.14 | 43.87 | 120.01 |
winequality_red/data.csv | 90 KB | GretelLSTM | tabular_numeric | 1599 | 12 | 89 | 221.92 | 54.69 | 276.61 |
winequality_white/data.csv | 281 KB | GretelAmplify | tabular_numeric | 4898 | 12 | 88 | 24.65 | 23.27 | 47.92 |
winequality_white/data.csv | 281 KB | GretelACTGAN | tabular_numeric | 4898 | 12 | 81 | 139.03 | 33.15 | 172.18 |
winequality_white/data.csv | 281 KB | GretelLSTM | tabular_numeric | 4898 | 12 | 91 | 287.84 | 76.4 | 364.24 |
Input data | Size | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
portuguese_election_data/data.csv | 3 MB | GretelAmplify | tabular_numeric | 21643 | 28 | 52 | 31.33 | 107.04 | 138.37 |
portuguese_election_data/data.csv | 3 MB | GretelACTGAN | tabular_numeric | 21643 | 28 | 72 | 928.15 | 128.73 | 1056.88 |
portuguese_election_data/data.csv | 3 MB | GretelLSTM | tabular_numeric | 21643 | 28 | 81 | 455.56 | 327.19 | 782.75 |
adult/data.csv | 3.6 MB | GretelAmplify | tabular_mixed | 32561 | 15 | 85 | 213.54 | 58.46 | 272 |
adult/data.csv | 3.6 MB | GretelACTGAN | tabular_mixed | 32561 | 15 | 87 | 965.31 | 128.03 | 1093.34 |
adult/data.csv | 3.6 MB | GretelLSTM | tabular_mixed | 32561 | 15 | 94 | 667.21 | 615.08 | 1282.29 |
Input data | Size | Model | Datatype | Rows | Cols | SQS | Train time (s) | Generate time (s) | Total time (s) |
---|---|---|---|---|---|---|---|---|---|
processed_cleveland_heart_disease_uci/data.csv | 18 KB | GretelAmplify | tabular_numeric | 303 | 14 | 83 | 35.87 | 23.01 | 58.88 |
processed_cleveland_heart_disease_uci/data.csv | 18 KB | GretelACTGAN | tabular_numeric | 303 | 14 | 70 | 66.97 | 33.48 | 100.45 |
processed_cleveland_heart_disease_uci/data.csv | 18 KB | GretelLSTM | tabular_numeric | 303 | 14 | 91 | 221.56 | 54.26 | 275.82 |
breast_cancer_wisconsin/data.csv | 19 KB | GretelAmplify | tabular_numeric | 699 | 11 | 55 | 23.89 | 23.55 | 47.44 |
breast_cancer_wisconsin/data.csv | 19 KB | GretelACTGAN | tabular_numeric | 699 | 11 | 56 | 67.4 | 203.11 | 270.51 |
breast_cancer_wisconsin/data.csv | 19 KB | GretelLSTM | tabular_numeric | 699 | 11 | 83 | 206.88 | 64.73 | 271.61 |
Input data | Size | Model | DataType | Rows | Columns | SQS | Train time (sec) | Generate time (sec) | Total time (sec) |
---|---|---|---|---|---|---|---|---|---|
covertype | 75 MB | GretelAmplify | tabular_numeric | 581012 | 55 | 91 | 238.24 | 60.64 | 298.88 |
covertype | 75 MB | GretelACTGAN | tabular_numeric | 581012 | 55 | 92 | 5870.86 | 1004.67 | 6875.53 |
covertype | 75 MB | GretelLSTM | tabular_numeric | 581012 | 55 | 83 | 7547.14 | 2317.74 | 9864.88 |
duck_duck_geese | 311 MB | GretelAmplify | tabular_numeric | 27000 | 1349 | 53 | 1962.59 | 593.78 | 2556.37 |
duck_duck_geese | 311 MB | GretelACTGAN | tabular_numeric | 27000 | 1349 | 75 | 55879.28 | 425.86 | 56305.14 |
kdd_cup_1999 | 743 MB | GretelACTGAN | tabular_mixed | 4898430 | 42 | 87 | 17408.25 | 3986.04 | 21394.29 |
pems_sf | 421 MB | GretelAmplify | tabular_numeric | 63360 | 967 | 68 | 3103.19 | 1247.72 | 4350.91 |
phoneme_spectra | 154 MB | GretelAmplify | tabular_numeric | 1446956 | 15 | 78 | 402.33 | 256.85 | 659.18 |
phoneme_spectra | 154 MB | GretelACTGAN | tabular_numeric | 1446956 | 15 | 74 | 5263.59 | 1741.28 | 7004.87 |
phoneme_spectra | 154 MB | GretelLSTM | tabular_numeric | 1446956 | 15 | 86 | 5099.49 | 1777.7 | 6877.19 |
poker_hand_test | 24 MB | GretelAmplify | tabular_numeric | 1000000 | 11 | 92 | 138.71 | 76.8 | 215.51 |
poker_hand_test | 24 MB | GretelACTGAN | tabular_numeric | 1000000 | 11 | 93 | 2207.99 | 1047.02 | 3255.01 |
poker_hand_test | 24 MB | GretelLSTM | tabular_numeric | 1000000 | 11 | 98 | 1646.91 | 1018.37 | 2665.28 |
poker_hand_train | 614 KB | GretelAmplify | tabular_numeric | 25010 | 11 | 94 | 28.55 | 23.7 | 52.25 |
poker_hand_train | 614 KB | GretelACTGAN | tabular_numeric | 25010 | 11 | 91 | 852.35 | 33.23 | 885.58 |
poker_hand_train |