89 citations · 94 across the 8 of their papers we have counts for
8 papers
Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoning
Peihao Wang, Shan Yang, Xijun Wang +8
Associative memory has long underpinned the design of sequential models. Beyond recall, humans reason by projecting future states and selecting goal-directed actions, a capability…
Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions
Xuxing Chen, Tesi Xiao, Krishnakumar Balasubramanian
Stochastic Bilevel optimization usually involves minimizing an upper-level (UL) function that is dependent on the arg-min of a strongly-convex lower-level (LL) function. Several al…
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
Tesi Xiao, Xuxing Chen, Krishnakumar Balasubramanian +1
We focus on decentralized stochastic non-convex optimization, where agents work together to optimize a composite objective function which is a sum of a smooth term and a non-sm…
Field-wise Embedding Size Search via Structural Hard Auxiliary Mask Pruning for Click-Through Rate Prediction
Tesi Xiao, Xia Xiao, Ming Chen +1
Feature embeddings are one of the most essential steps when training deep learning based Click-Through Rate prediction models, which map high-dimensional sparse features to dense e…
A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimization
Tesi Xiao, Krishnakumar Balasubramanian, Saeed Ghadimi
We propose a projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization, where the objective function is a nested composition of…
Statistical Inference for Polyak-Ruppert Averaged Zeroth-order Stochastic Gradient Algorithm
Yanhao Jin, Tesi Xiao, Krishnakumar Balasubramanian
Statistical machine learning models trained with stochastic gradient algorithms are increasingly being deployed in critical scientific applications. However, computing the stochast…
Improved Complexities for Stochastic Conditional Gradient Methods under Interpolation-like Conditions
Tesi Xiao, Krishnakumar Balasubramanian, Saeed Ghadimi
We analyze stochastic conditional gradient methods for constrained optimization problems arising in over-parametrized machine learning. We show that one could leverage the interpol…
Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise
Xuanqing Liu, Tesi Xiao, Si Si +3
Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discr…