Search…
⌃K

Benchmark Report

Check out this Benchmark report, running Gretel models on popular ML datasets, indexed by industry
You can use a Benchmark report like the one shown here to evaluate which Gretel model is best for your synthetic data goals.
For example, Gretel LSTM consistently generates synthetic data with high Synthetic Data Quality Score (SQS) on multiple types of tabular or complex data. As seen in the results below, Gretel ACTGAN is great for particularly long or wide datasets and has generally a faster runtime. If you’re looking to quickly generate lots of data, Gretel Amplify produces results in 1/10 of the time (check out the fast train and generate times!). Gretel GPT generates high-quality synthetic data for natural language datasets.
Depending on your specific goals with synthetic data or constraints, you may find particular Gretel models to be best suited for your use case. You can reference the Benchmark report below to guide how you evaluate Gretel models, or of course, try Benchmark on your datasets.
The publicly available datasets used in this results leaderboard were sourced from the following ML dataset repositories: UCI, Kaggle, and HuggingFace.

Ads, Finance, Marketing

Input Data
Model
Datatype
Rows
Cols
SQS
Train time (sec)
Generate time (sec)
Total time (sec)
bank_marketing_large/data.csv
GretelAmplify
tabular_mixed
41188
21
73
36.07
29.75
65.82
bank_marketing_large/data.csv
GretelACTGAN
tabular_mixed
41188
21
85
1300.75
33.24
1333.99
bank_marketing_large/data.csv
GretelLSTM
tabular_mixed
41188
21
84
317.79
401.04
718.83
bank_marketing_small/data.csv
GretelAmplify
tabular_mixed
4521
17
80
24.41
23.81
48.22
bank_marketing_small/data.csv
GretelACTGAN
tabular_mixed
4521
17
84
169.32
175.63
344.95
bank_marketing_small/data.csv
GretelLSTM
tabular_mixed
4521
17
84
326.26
96.73
422.99
dow_jones_index/data.csv
GretelAmplify
time_series
750
16
76
81.5
23.32
104.82
dow_jones_index/data.csv
GretelACTGAN
time_series
750
16
70
221.58
129.15
350.73
dow_jones_index/data.csv
GretelLSTM
time_series
750
16
83
424.2
64.66
488.86

E-commerce

Input data
Model
Datatype
Rows
Cols
SQS
Train time
Generate time
Total time
bike_sales/data.csv
GretelAmplify
tabular_numeric
16519
24
79
119.94
30
149.94
bike_sales/data.csv
GretelLSTM
tabular_numeric
16519
24
88
911.59
249.68
1161.27
car_evaluation/data.csv
GretelAmplify
tabular_numeric
1728
7
85
24.06
23.7
47.76
car_evaluation/data.csv
GretelACTGAN
tabular_numeric
1728
7
77
201.19
44.5
245.69
car_evaluation/data.csv
GretelLSTM
tabular_numeric
1728
7
87
357.66
54.02
411.68
olist_order_payments/data.csv
GretelAmplify
tabular_numeric
103886
5
69
529.06
40.41
569.47
olist_order_payments/data.csv
GretelLSTM
tabular_numeric
103886
5
93
4201.89
897.22
5099.11

Employment

Input data
Model
Datatype
Rows
Cols
SQS
Train time
Generate time
Total time
data_science_job_candidates/data.csv
GretelAmplify
tabular_mixed
19158
14
88
107.66
23.26
130.92
data_science_job_candidates/data.csv
GretelACTGAN
tabular_mixed
19158
14
90
609.02
128.11
737.13
data_science_job_candidates/data.csv
GretelLSTM
tabular_mixed
19158
14
93
358.21
276.29
634.5
ibm_employee_attrition/data.csv
GretelAmplify
tabular_mixed
1470
37
88
24.09
20.5
44.59
ibm_employee_attrition/data.csv
GretelACTGAN
tabular_mixed
1470
37
80
368.13
33.93
402.06
ibm_employee_attrition/data.csv
GretelLSTM
tabular_mixed
1470
37
93
365.18
127.79
492.97

Energy, Telecom

Input data
Model
Datatype
Rows
Cols
SQS
Train time
Generate time
Total time
energydata_complete/data.csv
GretelAmplify
time_series
19735
29
74
103.68
33.09
136.77
energydata_complete/data.csv
GretelLSTM
time_series
19735
29
93
1531.88
400.29
1932.17
telco_customer_churn/data.csv
GretelAmplify
tabular_mixed
7043
33
82
40.84
30.17
71.01
telco_customer_churn/data.csv
GretelACTGAN
tabular_mixed
7043
33
79
5911.34
55.36
5966.7
telco_customer_churn/data.csv
GretelLSTM
tabular_mixed
7043
33
76
787.7
155.55
943.25

Environment, Food

Input data
Model
Datatype
Rows
Cols
SQS
Train time
Generate time
Total time
air_quality_uci/data.csv
GretelAmplify
time_series
9357
15
65
96.15
52.47
148.62
air_quality_uci/data.csv
GretelACTGAN
time_series
9357
15
62
6656.12
54.49
6710.61
air_quality_uci/data.csv
GretelLSTM
time_series
9357
15
89
398.42
211.66
610.08
iris/data.csv
GretelAmplify
tabular_numeric
150
5
66
87.56
23.3
110.86
iris/data.csv
GretelACTGAN
tabular_numeric
150
5
51
140.46
170.43
310.89
iris/data.csv
GretelLSTM
tabular_numeric
150
5
79
148.21
160.02
308.23
winequality_red/data.csv
GretelAmplify
tabular_numeric
1599
12
82
81.94
23.68
105.62
winequality_red/data.csv
GretelACTGAN
tabular_numeric
1599
12
61
76.14
43.87
120.01
winequality_red/data.csv
GretelLSTM
tabular_numeric
1599
12
89
221.92
54.69
276.61
winequality_white/data.csv
GretelAmplify
tabular_numeric
4898
12
88
24.65
23.27
47.92
winequality_white/data.csv
GretelACTGAN
tabular_numeric
4898
12
81
139.03
33.15
172.18
winequality_white/data.csv
GretelLSTM
tabular_numeric
4898
12
91
287.84
76.4
364.24

Government

Input data
Model
Datatype
Rows
Cols
SQS
Train time (sec)
Generate time (sec)
Total time (sec)
portuguese_election_data/data.csv
GretelAmplify
tabular_numeric
21643
28
52
31.33
107.04
138.37
portuguese_election_data/data.csv
GretelACTGAN
tabular_numeric
21643
28
72
928.15
128.73
1056.88
portuguese_election_data/data.csv
GretelLSTM
tabular_numeric
21643
28
81
455.56
327.19
782.75
adult/data.csv
GretelAmplify
tabular_mixed
32561
15
85
213.54
58.46
272
adult/data.csv
GretelACTGAN
tabular_mixed
32561
15
87
965.31
128.03
1093.34
adult/data.csv
GretelLSTM
tabular_mixed
32561
15
94
667.21
615.08
1282.29

Healthcare

Input data
Model
Datatype
Rows
Cols
SQS
Train time
Generate time
Total time
processed_cleveland_heart_disease_uci/data.csv
GretelAmplify
tabular_numeric
303
14
83
35.87
23.01
58.88
processed_cleveland_heart_disease_uci/data.csv
GretelACTGAN
tabular_numeric
303
14
70
66.97
33.48
100.45
processed_cleveland_heart_disease_uci/data.csv
GretelLSTM
tabular_numeric
303
14
91
221.56
54.26
275.82
breast_cancer_wisconsin/data.csv
GretelAmplify
tabular_numeric
699
11
55
23.89
23.55
47.44
breast_cancer_wisconsin/data.csv
GretelACTGAN
tabular_numeric
699
11
56
67.4
203.11
270.51
breast_cancer_wisconsin/data.csv
GretelLSTM
tabular_numeric
699
11
83
206.88
64.73
271.61