6 papers
Generalization error of min-norm interpolators in transfer learning
Yanke Song, Kenneth Gu, Sohom Bhattacharya +1
This paper establishes the generalization error of pooled min--norm interpolation in transfer learning, where data from diverse distributions are available. Min-norm interp…
Hardware Co-Design Scaling Laws via Roofline Modelling for On-Device LLMs
Luoyang Sun, Jiwen Jiang, Yifeng Ding +9
Vision-Language-Action Models (VLAs) have emerged as a key paradigm of Physical AI and are increasingly deployed in autonomous vehicles, robots, and smart spaces. In these resource…
From to : Two-Sided Low-Rank Communication for Adam in Distributed Training with Memory Efficiency
Sizhe Dang, Jiaqi Shao, Xiaodong Zheng +3
As foundation models continue to scale, pretraining increasingly relies on data-parallel distributed optimization, making bandwidth-limited gradient synchronization a key bottlenec…
Multi-Environment GLAMP: Approximate Message Passing for Transfer Learning with Applications to Lasso-based Estimators
Longlin Wang, Yanke Song, Kuanhao Jiang +1
Approximate Message Passing (AMP) algorithms enable precise characterization of certain classes of random objects in the high-dimensional limit, and have found widespread applicati…
A Poisson Process AutoDecoder for X-ray Sources
Yanke Song, Victoria Ashley Villar, Juan Rafael Martinez-Galarza +1
X-ray observing facilities, such as the Chandra X-ray Observatory and the eROSITA, have detected millions of astronomical sources associated with high-energy phenomena. The arrival…
Multi-student Diffusion Distillation for Better One-step Generators
Yanke Song, Jonathan Lorraine, Weili Nie +2
Diffusion models achieve high-quality sample generation at the cost of a lengthy multistep inference procedure. To overcome this, diffusion distillation techniques produce student…