6 citations · 6 across the 15 of their papers we have counts for
8 papers · 1 filter
ORCA: The Annealed Spectral Conditioning Optimizer for Faster, Better LLM Training
Yuanshi Liu, Boyuan Jiang, Liang Hou +4
Modern LLM optimizers such as Muon often produce weight matrices with higher effective rank than Adam, yet further spectral control has delivered only modest gains. We identify a t…
Beyond Overparameterization: Provable Learning of Input-Convex Multi-Layer Polynomial Networks with Active Queries
Jinqi Tang, Qian Chen, Shihong Ding +1
The theoretical understanding of multi-layer neural networks is largely confined to overparameterized settings, which obscure parameter identifiability and incur high sample comple…
What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression
Wendao Wu, Fangqing Zhang, Haihan Zhang +1
Teacher-Student Knowledge Transfer (KT) is ubiquitous in modern machine learning, ranging from classical model compression via Knowledge Distillation (KD) to the emergent phenomeno…
SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
Lirui Luo, Guoxi Zhang, Hongming Xu +2
In deep reinforcement learning (DRL), an agent is trained from a stream of experience. In a continual learning setting, such agents can suffer from plasticity loss: their ability t…
Near-optimal and Efficient First-Order Algorithm for Multi-Task Learning with Shared Linear Representation
Shihong Ding, Fangyu Du, Cong Fang
Multi-task learning (MTL) has emerged as a pivotal paradigm in machine learning by leveraging shared structures across multiple related tasks. Despite its empirical success, the de…
Mild Over-Parameterization Benefits Asymmetric Tensor PCA
Shihong Ding, Weicheng Lin, Cong Fang
Asymmetric Tensor PCA (ATPCA) is a prototypical model for studying the trade-offs between sample complexity, computation, and memory. Existing algorithms for this problem typically…
Accelerating Single-Pass SGD for Generalized Linear Prediction
Qian Chen, Shihong Ding, Cong Fang
We study generalized linear prediction under a streaming setting, where each iteration uses only one fresh data point for a gradient-level update. While momentum is well-establishe…
MVR: Multi-view Video Reward Shaping for Reinforcement Learning
Lirui Luo, Guoxi Zhang, Hongming Xu +3
Reward design is of great importance for solving complex tasks with reinforcement learning. Recent studies have explored using image-text similarity produced by vision-language mod…