9 papers
GRAFT: Growing Agglomerative Foundation Models via Continual Teacher Distillation
Zhenghao Zhao, Chi Zhang, Qingshuang Chen +1
Vision foundation models such as DINOv2, SigLIP2, and MASt3R develop complementary capabilities from different pretraining objectives, yet their knowledge remains distributed acros…
Fisher-Guided Submodular Data Selection for Continual Pre-Training of Large Language Models
Zhenghao Zhao, Gaowen Liu, Zhiling Lan +1
Data selection is already a central bottleneck in large-language-model training, where web-scale corpora are noisy and token budgets are finite. In continual pre-training (CPT), it…
Distill Video Datasets into Images
Zhenghao Zhao, Haoxuan Wang, Kai Wang +3
Dataset distillation aims to synthesize compact yet informative datasets that allow models trained on them to achieve performance comparable to training on the full dataset. While…
Consistent Instance Field for Dynamic Scene Understanding
Junyi Wu, Van Nguyen Nguyen, Benjamin Planche +11
We introduce Consistent Instance Field, a continuous and probabilistic spatio-temporal representation for dynamic scene understanding. Unlike prior methods that rely on discrete tr…
From Particles to Fields: Reframing Photon Mapping with Continuous Gaussian Photon Fields
Jiachen Tao, Benjamin Planche, Van Nguyen Nguyen +13
Accurately modeling light transport is essential for realistic image synthesis. Photon mapping provides physically grounded estimates of complex global illumination effects such as…
Efficient Multimodal Dataset Distillation via Generative Models
Zhenghao Zhao, Haoxuan Wang, Junyi Wu +3
Dataset distillation aims to synthesize a small dataset from a large dataset, enabling the model trained on it to perform well on the original dataset. With the blooming of large l…
CaO: Rectifying Inconsistencies in Diffusion-Based Dataset Distillation
Haoxuan Wang, Zhenghao Zhao, Junyi Wu +3
The recent introduction of diffusion models in dataset distillation has shown promising potential in creating compact surrogate datasets for large, high-resolution target datasets,…
Distilling Long-tailed Datasets
Zhenghao Zhao, Haoxuan Wang, Yuzhang Shang +2
Dataset distillation aims to synthesize a small, information-rich dataset from a large one for efficient model training. However, existing dataset distillation methods struggle wit…
Dataset Quantization with Active Learning based Adaptive Sampling
Zhenghao Zhao, Yuzhang Shang, Junyi Wu +1
Deep learning has made remarkable progress recently, largely due to the availability of large, well-labeled datasets. However, the training on such datasets elevates costs and comp…