Ask or search…
⌃K
Comment on page

Benchmark Report

Check out this Benchmark report, running Gretel models on popular ML datasets, indexed by industry
You can use a Benchmark report like the one shown here to evaluate which Gretel model is best for your synthetic data goals.
For example, Gretel LSTM consistently generates synthetic data with high Synthetic Data Quality Score (SQS) on multiple types of tabular or complex data. As seen in the results below, Gretel ACTGAN is great for particularly long or wide datasets and has generally a faster runtime. If you’re looking to quickly generate lots of data, Gretel Amplify produces results in 1/10 of the time (check out the fast train and generate times!). Gretel GPT generates high-quality synthetic data for natural language datasets.
Depending on your specific goals with synthetic data or constraints, you may find particular Gretel models to be best suited for your use case. You can reference the Benchmark report below to guide how you evaluate Gretel models, or of course, try Benchmark on your datasets.
The publicly available datasets used in this results leaderboard were sourced from the following ML dataset repositories: UCI, Kaggle, and HuggingFace.

Ads, Finance, Marketing

Input Data
Size
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
bank_marketing_large/data.csv
4.9 MB
GretelAmplify
tabular_mixed
41188
21
73
36.07
29.75
65.82
bank_marketing_large/data.csv
4.9 MB
GretelACTGAN
tabular_mixed
41188
21
85
1300.75
33.24
1333.99
bank_marketing_large/data.csv
4.9 MB
GretelLSTM
tabular_mixed
41188
21
84
317.79
401.04
718.83
bank_marketing_small/data.csv
371 KB
GretelAmplify
tabular_mixed
4521
17
80
24.41
23.81
48.22
bank_marketing_small/data.csv
371 KB
GretelACTGAN
tabular_mixed
4521
17
84
169.32
175.63
344.95
bank_marketing_small/data.csv
371 KB
GretelLSTM
tabular_mixed
4521
17
84
326.26
96.73
422.99
dow_jones_index/data.csv
89 KB
GretelAmplify
time_series
750
16
76
81.5
23.32
104.82
dow_jones_index/data.csv
89 KB
GretelACTGAN
time_series
750
16
70
221.58
129.15
350.73
dow_jones_index/data.csv
89 KB
GretelLSTM
time_series
750
16
83
424.2
64.66
488.86

E-commerce

Input data
Size
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
bike_sales/data.csv
2.4 MB
GretelAmplify
tabular_numeric
16519
24
79
119.94
30
149.94
bike_sales/data.csv
2.4 MB
GretelLSTM
tabular_numeric
16519
24
88
911.59
249.68
1161.27
car_evaluation/data.csv
52 KB
GretelAmplify
tabular_numeric
1728
7
85
24.06
23.7
47.76
car_evaluation/data.csv
52 KB
GretelACTGAN
tabular_numeric
1728
7
77
201.19
44.5
245.69
car_evaluation/data.csv
52 KB
GretelLSTM
tabular_numeric
1728
7
87
357.66
54.02
411.68
olist_order_payments/data.csv
5.6 MB
GretelAmplify
tabular_numeric
103886
5
69
529.06
40.41
569.47
olist_order_payments/data.csv
5.6 MB
GretelLSTM
tabular_numeric
103886
5
93
4201.89
897.22
5099.11

Employment

Input data
Size
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
data_science_job_candidates/data.csv
1.9 MB
GretelAmplify
tabular_mixed
19158
14
88
107.66
23.26
130.92
data_science_job_candidates/data.csv
1.9 MB
GretelACTGAN
tabular_mixed
19158
14
90
609.02
128.11
737.13
data_science_job_candidates/data.csv
1.9 MB
GretelLSTM
tabular_mixed
19158
14
93
358.21
276.29
634.5
ibm_employee_attrition/data.csv
274 KB
GretelAmplify
tabular_mixed
1470
37
88
24.09
20.5
44.59
ibm_employee_attrition/data.csv
274 KB
GretelACTGAN
tabular_mixed
1470
37
80
368.13
33.93
402.06
ibm_employee_attrition/data.csv
274 KB
GretelLSTM
tabular_mixed
1470
37
93
365.18
127.79
492.97

Energy, Telecom

Input data
Size
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
energydata_complete/data.csv
11.4 MB
GretelAmplify
time_series
19735
29
74
103.68
33.09
136.77
energydata_complete/data.csv
11.4 MB
GretelLSTM
time_series
19735
29
93
1531.88
400.29
1932.17
telco_customer_churn/data.csv
1.7 MB
GretelAmplify
tabular_mixed
7043
33
82
40.84
30.17
71.01
telco_customer_churn/data.csv
1.7 MB
GretelACTGAN
tabular_mixed
7043
33
79
5911.34
55.36
5966.7
telco_customer_churn/data.csv
1.7 MB
GretelLSTM
tabular_mixed
7043
33
76
787.7
155.55
943.25

Environment, Food

Input data
Text
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
air_quality_uci/data.csv
822 KB
GretelAmplify
time_series
9357
15
65
96.15
52.47
148.62
air_quality_uci/data.csv
822 KB
GretelACTGAN
time_series
9357
15
62
6656.12
54.49
6710.61
air_quality_uci/data.csv
822 KB
GretelLSTM
time_series
9357
15
89
398.42
211.66
610.08
iris/data.csv
4 KB
GretelAmplify
tabular_numeric
150
5
66
87.56
23.3
110.86
iris/data.csv
4 KB
GretelACTGAN
tabular_numeric
150
5
51
140.46
170.43
310.89
iris/data.csv
4 KB
GretelLSTM
tabular_numeric
150
5
79
148.21
160.02
308.23
winequality_red/data.csv
90 KB
GretelAmplify
tabular_numeric
1599
12
82
81.94
23.68
105.62
winequality_red/data.csv
90 KB
GretelACTGAN
tabular_numeric
1599
12
61
76.14
43.87
120.01
winequality_red/data.csv
90 KB
GretelLSTM
tabular_numeric
1599
12
89
221.92
54.69
276.61
winequality_white/data.csv
281 KB
GretelAmplify
tabular_numeric
4898
12
88
24.65
23.27
47.92
winequality_white/data.csv
281 KB
GretelACTGAN
tabular_numeric
4898
12
81
139.03
33.15
172.18
winequality_white/data.csv
281 KB
GretelLSTM
tabular_numeric
4898
12
91
287.84
76.4
364.24

Government

Input data
Size
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
portuguese_election_data/data.csv
3 MB
GretelAmplify
tabular_numeric
21643
28
52
31.33
107.04
138.37
portuguese_election_data/data.csv
3 MB
GretelACTGAN
tabular_numeric
21643
28
72
928.15
128.73
1056.88
portuguese_election_data/data.csv
3 MB
GretelLSTM
tabular_numeric
21643
28
81
455.56
327.19
782.75
adult/data.csv
3.6 MB
GretelAmplify
tabular_mixed
32561
15
85
213.54
58.46
272
adult/data.csv
3.6 MB
GretelACTGAN
tabular_mixed
32561
15
87
965.31
128.03
1093.34
adult/data.csv
3.6 MB
GretelLSTM
tabular_mixed
32561
15
94
667.21
615.08
1282.29

Healthcare

Input data
Size
Model
Datatype
Rows
Cols
SQS
Train time (s)
Generate time (s)
Total time (s)
processed_cleveland_heart_disease_uci/data.csv
18 KB
GretelAmplify
tabular_numeric
303
14
83
35.87
23.01
58.88
processed_cleveland_heart_disease_uci/data.csv
18 KB
GretelACTGAN
tabular_numeric
303
14
70
66.97
33.48
100.45
processed_cleveland_heart_disease_uci/data.csv
18 KB
GretelLSTM
tabular_numeric
303
14
91
221.56
54.26
275.82
breast_cancer_wisconsin/data.csv
19 KB
GretelAmplify
tabular_numeric
699
11
55
23.89
23.55
47.44
breast_cancer_wisconsin/data.csv
19 KB
GretelACTGAN
tabular_numeric
699
11
56
67.4
203.11
270.51
breast_cancer_wisconsin/data.csv
19 KB
GretelLSTM
tabular_numeric
699
11
83
206.88
64.73
271.61

Large Datasets

Input data
Size
Model
DataType
Rows
Columns
SQS
Train time (sec)
Generate time (sec)
Total time (sec)
covertype
75 MB
GretelAmplify
tabular_numeric
581012
55
91
238.24
60.64
298.88
covertype
75 MB
GretelACTGAN
tabular_numeric
581012
55
92
5870.86
1004.67
6875.53
covertype
75 MB
GretelLSTM
tabular_numeric
581012
55
83
7547.14
2317.74
9864.88
duck_duck_geese
311 MB
GretelAmplify
tabular_numeric
27000
1349
53
1962.59
593.78
2556.37
duck_duck_geese
311 MB
GretelACTGAN
tabular_numeric
27000
1349
75
55879.28
425.86
56305.14
kdd_cup_1999
743 MB
GretelACTGAN
tabular_mixed
4898430
42
87
17408.25
3986.04
21394.29
pems_sf
421 MB
GretelAmplify
tabular_numeric
63360
967
68
3103.19
1247.72
4350.91
phoneme_spectra
154 MB
GretelAmplify
tabular_numeric
1446956
15
78
402.33
256.85
659.18
phoneme_spectra
154 MB
GretelACTGAN
tabular_numeric
1446956
15
74
5263.59
1741.28
7004.87
phoneme_spectra
154 MB
GretelLSTM
tabular_numeric
1446956
15
86
5099.49
1777.7
6877.19
poker_hand_test
24 MB
GretelAmplify
tabular_numeric
1000000
11
92
138.71
76.8
215.51
poker_hand_test
24 MB
GretelACTGAN
tabular_numeric
1000000
11
93
2207.99
1047.02
3255.01
poker_hand_test
24 MB
GretelLSTM
tabular_numeric
1000000
11
98
1646.91
1018.37
2665.28
poker_hand_train
614 KB
GretelAmplify
tabular_numeric
25010
11
94
28.55
23.7
52.25
poker_hand_train
614 KB
GretelACTGAN
tabular_numeric
25010
11
91
852.35
33.23
885.58
poker_hand_train