Benchmark Report

Check out this Benchmark report, running Gretel models on popular ML datasets, indexed by industry

You can use a Benchmark report like the one shown here to evaluate which Gretel model is best for your synthetic data goals.

For example, Gretel Navigator Fine Tuning consistently generates synthetic data with high Synthetic Data Quality Score (SQS) on multiple types of tabular data, and Gretel ACTGAN is great for particularly long or wide datasets.

Depending on your specific goals with synthetic data or constraints, you may find particular Gretel models to be best suited for your use case. You can reference the Benchmark report below to guide how you evaluate Gretel models, or of course, try Benchmark on your datasets.

The publicly available datasets used in this results leaderboard were sourced from the following ML dataset repositories: UCI, Kaggle, and HuggingFace.

Ads, Finance, Marketing

Input Data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
bank_marketing_large/data.csv	GretelNavigatorFineTuning	84	83	1034.299	4.9 MB	tabular_mixed	21	41188
bank_marketing_large/data.csv	GretelACTGAN	89	86	1080.329	4.9 MB	tabular_mixed	21	41188
bank_marketing_small/data.csv	GretelNavigatorFineTuning	95	85	669.117	371 KB	tabular_mixed	17	4521
bank_marketing_small/data.csv	GretelACTGAN	87	87	148.713	371 KB	tabular_mixed	17	4521
dow_jones_index/data.csv	GretelNavigatorFineTuning	93	53	2126.556	89 KB	time_series	16	750
dow_jones_index/data.csv	GretelACTGAN	60	97	62.725	89 KB	time_series	16	750

E-commerce

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
bike_sales/data.csv	GretelNavigatorFineTuning	87	94	1334.324	2.4 MB	tabular_numeric	24	16519
bike_sales/data.csv	GretelACTGAN	78	95	444.495	2.4 MB	tabular_numeric	24	16519
car_evaluation/data.csv	GretelNavigatorFineTuning	95	75	368.91	52 KB	tabular_numeric	7	1728
car_evaluation/data.csv	GretelACTGAN	86	74	67.322	52 KB	tabular_numeric	7	1728
olist_order_payments/data.csv	GretelNavigatorFineTuning	83	97	507.772	5.6 MB	tabular_numeric	5	103886
olist_order_payments/data.csv	GretelACTGAN	87	77	868.921	5.6 MB	tabular_numeric	5	103886

Employment

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
data_science_job_candidates/data.csv	GretelNavigatorFineTuning	93	91	874.715	1.9 MB	tabular_mixed	14	19158
data_science_job_candidates/data.csv	GretelACTGAN	91	91	417.622	1.9 MB	tabular_mixed	14	19158
ibm_employee_attrition/data.csv	GretelNavigatorFineTuning	91	73	2008.613	274 KB	tabular_mixed	37	1470
ibm_employee_attrition/data.csv	GretelACTGAN	75	89	98.267	274 KB	tabular_mixed	37	1470

Energy, Telecom

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
energydata_complete/data.csv	GretelNavigatorFineTuning	89	95	2277.711	11.4 MB	time_series	29	19735
energydata_complete/data.csv	GretelACTGAN	75	84	653.85	11.4 MB	time_series	29	19735
telco_customer_churn/data.csv	GretelNavigatorFineTuning	87	90	1805.024	1.7 MB	tabular_mixed	33	7043
telco_customer_churn/data.csv	GretelACTGAN	80	91	265.678	1.7 MB	tabular_mixed	33	7043

Environment, Food

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
air_quality_uci/data.csv	GretelNavigatorFineTuning	90	81	1144.394	822 KB	time_series	15	9357
air_quality_uci/data.csv	GretelACTGAN	69	86	214.324	822 KB	time_series	15	9357
iris/data.csv	GretelNavigatorFineTuning	82	78	376.864	4 KB	tabular_numeric	5	150
iris/data.csv	GretelACTGAN	78	58	56.388	4 KB	tabular_numeric	5	150
winequality_red/data.csv	GretelNavigatorFineTuning	92	90	775.856	90 KB	tabular_numeric	12	1599
winequality_red/data.csv	GretelACTGAN	66	92	60.729	90 KB	tabular_numeric	12	1599
winequality_white/data.csv	GretelNavigatorFineTuning	94	88	742.663	281 KB	tabular_numeric	12	4898
winequality_white/data.csv	GretelACTGAN	82	89	120.683	281 KB	tabular_numeric	12	4898

Government

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
portuguese_election_data/data.csv	GretelNavigatorFineTuning	89	65	2170.874	3 MB	tabular_numeric	28	21643
portuguese_election_data/data.csv	GretelACTGAN	87	88	710.705	3 MB	tabular_numeric	28	21643
adult/data.csv	GretelNavigatorFineTuning	90	79	815.833	3.6 MB	tabular_mixed	15	32561
adult/data.csv	GretelACTGAN	92	80	683.149	3.6 MB	tabular_mixed	15	32561

Healthcare

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	Datatype	Cols	Rows
processed_cleveland_heart_disease_uci/data.csv	GretelNavigatorFineTuning	92	53	723.425	18 KB	tabular_numeric	14	303
processed_cleveland_heart_disease_uci/data.csv	GretelACTGAN	73	73	47.015	18 KB	tabular_numeric	14	303
breast_cancer_wisconsin/data.csv	GretelNavigatorFineTuning	83	76	649.074	19 KB	tabular_numeric	11	699
breast_cancer_wisconsin/data.csv	GretelACTGAN	75	78	47.237	19 KB	tabular_numeric	11	699

Large Datasets

Input data	Model	SQS	Data Privacy Score	Time (sec)	Size	DataType	Columns	Rows
covtype	GretelNavigatorFineTuning	85	84	12805.043	75 MB	tabular_numeric	55	581012
covtype	GretelACTGAN	92	81	3135.347	75 MB	tabular_numeric	55	581012
duck_duck_geese	GretelACTGAN	86	50	94297.403	311 MB	tabular_numeric	1349	27000
kdd_cup_1999	GretelACTGAN	83	77	22133.246	743 MB	tabular_mixed	42	4898430
pems_sf	GretelACTGAN	83	50	154719.489	421 MB	tabular_numeric	967	63360
phoneme_spectra	GretelNavigatorFineTuning	93	98	1353.606	154 MB	tabular_numeric	15	1446956
phoneme_spectra	GretelACTGAN	92	99	10106.598	154 MB	tabular_numeric	15	1446956
poker_hand_test	GretelNavigatorFineTuning	99	87	511.645	24 MB	tabular_numeric	11	1000000
poker_hand_test	GretelACTGAN	95	85	5350.631	24 MB	tabular_numeric	11	1000000
poker_hand_train	GretelNavigatorFineTuning	99	89	445.614	614 KB	tabular_numeric	11	25010
poker_hand_train	GretelACTGAN	91	90	419.063	614 KB	tabular_numeric	11	25010
record_linkage	GretelNavigatorFineTuning	67	94	1608.793	262 MB	tabular_numeric	12	5749132
record_linkage	GretelACTGAN	85	92	33233.547	262 MB	tabular_numeric	12	5749132
rossman_store_sale	GretelNavigatorFineTuning	92	92	570.957	38 MB	tabular_mixed	9	1017209
rossman_store_sale	GretelACTGAN	89	89	5040.424	38 MB	tabular_mixed	9	1017209