6 papers
Sharper Analysis of Single-Loop Methods for Bilevel Optimization
Yubo Zhou, Jun Shu, Luo Luo +4
Bilevel optimization underpins many machine learning applications, including hyperparameter optimization, meta-learning, neural architecture search, and reinforcement learning. Whi…
A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws
Jun Shu, Junxiong Jia, Deyu Meng +1
Emergent intelligence have played a major role in the modern AI development. While existing studies primarily rely on empirical observations to characterize this phenomenon, a rigo…
Understanding the Generalization of Bilevel Programming in Hyperparameter Optimization: A Tale of Bias-Variance Decomposition
Yubo Zhou, Jun Shu, Junmin Liu +1
Gradient-based hyperparameter optimization (HPO) have emerged recently, leveraging bilevel programming techniques to optimize hyperparameter by estimating hypergradient w.r.t. vali…
KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra
Liangyu Su, Jun Shu, Rui Liu +2
Representing and predicting high-dimensional and spatiotemporally chaotic dynamical systems remains a fundamental challenge in dynamical systems and machine learning. Although data…
Are Dense Labels Always Necessary for 3D Object Detection from Point Cloud?
Chenqiang Gao, Chuandong Liu, Jun Shu +5
Current state-of-the-art (SOTA) 3D object detection methods often require a large amount of 3D bounding box annotations for training. However, collecting such large-scale densely-s…
Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learning
Zhida Jiang, Zhaolong Xing, Jiawei Lu +13
Despite algorithm-level innovations for multi-agent reinforcement learning (MARL), the underlying networked infrastructure for large-scale MARL training remains underexplored. Exis…