Benchmark Report

Check out this Benchmark report, running Gretel models on popular ML datasets, indexed by industry

Gretel Tabular Fine-Tuning consistently generates synthetic data with high Synthetic Data Quality Score (SQS) on multiple types of tabular data, and Gretel Tabular GAN is great for particularly long or wide datasets.

Depending on your specific goals with synthetic data or constraints, you may find particular Gretel models to be best suited for your use case. You can reference the Benchmark report below to guide how you evaluate Gretel models, or of course, try Benchmark on your datasets.

The publicly available datasets used in this results leaderboard were sourced from the following ML dataset repositories: UCI, Kaggle, and HuggingFace.

Ads, Finance, Marketing

Input Data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
bank_marketing_large.csv	Tabular Fine-Tuning	84	83	1034.299	4.9 MB	tabular_mixed	21	41188
bank_marketing_large.csv	Tabular GAN	89	86	1080.329	4.9 MB	tabular_mixed	21	41188
bank_marketing_large.csv	Tabular Fine-Tuning	95	85	669.117	371 KB	tabular_mixed	17	4521
bank_marketing_large.csv	Tabular GAN	87	87	148.713	371 KB	tabular_mixed	17	4521
dow_jones_index.csv	Tabular Fine-Tuning	93	53	2126.556	89 KB	time_series	16	750
dow_jones_index.csv	Tabular GAN	60	97	62.725	89 KB	time_series	16	750

E-commerce

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
bike_sales.csv	Tabular Fine-Tuning	87	94	1334.324	2.4 MB	tabular_numeric	24	16519
bike_sales.csv	Tabular GAN	78	95	444.495	2.4 MB	tabular_numeric	24	16519
car_evaluation.csv	Tabular Fine-Tuning	95	75	368.91	52 KB	tabular_numeric	7	1728
car_evaluation.csv	Tabular GAN	86	74	67.322	52 KB	tabular_numeric	7	1728
olist_order_payments.csv	Tabular Fine-Tuning	83	97	507.772	5.6 MB	tabular_numeric	5	103886
olist_order_payments.csv	Tabular GAN	87	77	868.921	5.6 MB	tabular_numeric	5	103886

Employment

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
data_science_job_candidates.csv	Tabular Fine-Tuning	93	91	874.715	1.9 MB	tabular_mixed	14	19158
data_science_job_candidates.csv	Tabular GAN	91	91	417.622	1.9 MB	tabular_mixed	14	19158
ibm_employee_attrition.csv	Tabular Fine-Tuning	91	73	2008.613	274 KB	tabular_mixed	37	1470
ibm_employee_attrition.csv	Tabular GAN	75	89	98.267	274 KB	tabular_mixed	37	1470

Energy, Telecom

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
energydata_complete.csv	Tabular Fine-Tuning	89	95	2277.711	11.4 MB	time_series	29	19735
energydata_complete.csv	Tabular GAN	75	84	653.85	11.4 MB	time_series	29	19735
telco_customer_churn.csv	Tabular Fine-Tuning	87	90	1805.024	1.7 MB	tabular_mixed	33	7043
telco_customer_churn.csv	Tabular GAN	80	91	265.678	1.7 MB	tabular_mixed	33	7043

Environment, Food

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
air_quality_uci.csv	Tabular Fine-Tuning	90	81	1144.394	822 KB	time_series	15	9357
air_quality_uci.csv	Tabular GAN	69	86	214.324	822 KB	time_series	15	9357
iris.csv	Tabular Fine-Tuning	82	78	376.864	4 KB	tabular_numeric	5	150
iris.csv	Tabular GAN	78	58	56.388	4 KB	tabular_numeric	5	150
winequality_red.csv	Tabular Fine-Tuning	92	90	775.856	90 KB	tabular_numeric	12	1599
winequality_red.csv	Tabular GAN	66	92	60.729	90 KB	tabular_numeric	12	1599
winequality_white.csv	Tabular Fine-Tuning	94	88	742.663	281 KB	tabular_numeric	12	4898
winequality_white.csv	Tabular GAN	82	89	120.683	281 KB	tabular_numeric	12	4898

Government

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
portuguese_election_data.csv	Tabular Fine-Tuning	89	65	2170.874	3 MB	tabular_numeric	28	21643
portuguese_election_data.csv	Tabular GAN	87	88	710.705	3 MB	tabular_numeric	28	21643
adult.csv	Tabular Fine-Tuning	90	79	815.833	3.6 MB	tabular_mixed	15	32561
adult.csv	Tabular GAN	92	80	683.149	3.6 MB	tabular_mixed	15	32561

Healthcare

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
processed_cleveland_heart_disease_uci.csv	Tabular Fine-Tuning	92	53	723.425	18 KB	tabular_numeric	14	303
processed_cleveland_heart_disease_uci.csv	Tabular GAN	73	73	47.015	18 KB	tabular_numeric	14	303
breast_cancer_wisconsin.csv	Tabular Fine-Tuning	83	76	649.074	19 KB	tabular_numeric	11	699
breast_cancer_wisconsin.csv	Tabular GAN	75	78	47.237	19 KB	tabular_numeric	11	699

Large Datasets

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	DataType	Columns	Rows
covtype	Tabular Fine-Tuning	85	84	12805.043	75 MB	tabular_numeric	55	581012
covtype	Tabular GAN	92	81	3135.347	75 MB	tabular_numeric	55	581012
duck_duck_geese	Tabular GAN	86	50	94297.403	311 MB	tabular_numeric	1349	27000
kdd_cup_1999	Tabular GAN	83	77	22133.246	743 MB	tabular_mixed	42	4898430
pems_sf	Tabular GAN	83	50	154719.489	421 MB	tabular_numeric	967	63360
phoneme_spectra	Tabular Fine-Tuning	93	98	1353.606	154 MB	tabular_numeric	15	1446956
phoneme_spectra	Tabular GAN	92	99	10106.598	154 MB	tabular_numeric	15	1446956
poker_hand_large	Tabular Fine-Tuning	99	87	511.645	24 MB	tabular_numeric	11	1000000
poker_hand_large	Tabular GAN	95	85	5350.631	24 MB	tabular_numeric	11	1000000
poker_hand_small	Tabular Fine-Tuning	99	89	445.614	614 KB	tabular_numeric	11	25010
poker_hand_small	Tabular GAN	91	90	419.063	614 KB	tabular_numeric	11	25010
record_linkage	Tabular Fine-Tuning	67	94	1608.793	262 MB	tabular_numeric	12	5749132
record_linkage	Tabular GAN	85	92	33233.547	262 MB	tabular_numeric	12	5749132
rossman_store_sales	Tabular Fine-Tuning	92	92	570.957	38 MB	tabular_mixed	9	1017209
rossman_store_sales	Tabular GAN	89	89	5040.424	38 MB	tabular_mixed	9	1017209