Artificial Inductive Bias for Synthetic Tabular Data Generation in Data-Scarce Scenarios
arXiv:2407.03080 · doi:10.1016/j.neucom.2025.131122
Abstract
While synthetic tabular data generation using Deep Generative Models (DGMs) offers a compelling solution to data scarcity and privacy concerns, their effectiveness relies on the availability of substantial training data, often lacking in real-world scenarios. To overcome this limitation, we propose a novel methodology that explicitly integrates artificial inductive biases into the generative process to improve data quality in low-data regimes. Our framework leverages transfer learning and meta-learning techniques to construct and inject informative inductive biases into DGMs. We evaluate four approaches (pre-training, model averaging, Model-Agnostic Meta-Learning (MAML), and Domain Randomized Search (DRS)) and analyze their impact on the quality of the generated text. Experimental results show that incorporating inductive bias substantially improves performance, with transfer learning methods outperforming meta-learning, achieving up to 60\% gains in Jensen-Shannon divergence. The methodology is model-agnostic and especially relevant in domains such as healthcare and finance, where high-quality synthetic data are essential, and data availability is often limited.
arXiv comment19 pages, 6 Figures
References in corpus (13)
- SMOTE: Synthetic Minority Over-sampling Technique
- Auto-Encoding Variational Bayes
- Denoising Diffusion Probabilistic Models
- Deep Neural Networks and Tabular Data: A Survey
- Data Synthesis based on Generative Adversarial Networks
- Video Transformers: A Survey
- TransTab: Learning Transferable Tabular Transformers Across Tables
- Transformers Can Do Bayesian Inference
- Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space
- The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning
- STaSy: Score-based Tabular data Synthesis
- Modeling and Optimization Trade-off in Meta-learning
- XTab: Cross-table Pretraining for Tabular Transformers