8 papers
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Da Chang, Qiankun Shi, Lvgang Zhang +4
Muon-style optimizers apply a polar map to matrix momentum, but their updates also depend on the representation of each parameter block before orthogonalization. We study this repr…
A Fletcher's Augmented Lagrangian-Based Stochastic First-Order Method for Nonconvex Equality-Constrained Optimization
Yawen Cui, Qiankun Shi, Xiao Wang +1
In this paper, we study nonconvex equality-constrained optimization problems in which only stochastic first-order approximations of the objective and constraint functions are avail…
A Note on Stability for Orthogonalized Matrix Momentum with Client Sampling
Da Chang, Qiankun Shi, Lvgang Zhang +2
We study finite-sample generalization for a client-sampled distributed optimization scheme with matrix-valued parameters and orthogonalized momentum updates. The central quantity i…
MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration
Da Chang, Qiankun Shi, Lvgang Zhang +5
Orthogonalized-update optimizers such as Muon improve training of matrix-valued parameters, but existing extensions typically either rescale updates after orthogonalization or use…
The Effective Number of Nonzeros: Theory and Regularization for Sparse Recovery
Haoyu He, Hao Wang, Jiashan Wang +1
Classical sparse recovery treats all nonzero entries equally, though numerical noise often creates long tails of negligible coefficients. This paper develops an entropy-based notio…
Adaptive directional decomposition methods for nonconvex constrained optimization
Qiankun Shi, Xiao Wang
In this paper, we study nonconvex constrained optimization problems with both equality and inequality constraints, covering deterministic and stochastic settings. We propose a nove…
Self-Healing Hybrid Control as a Proxy for Detection and Mitigation of Sensor Attacks in Cooperative Driving
Mischa Huisman, Carlos Murguia, Erjen Lefeber +1
We propose a real-time hybrid controller scheme to detect and mitigate False-Data Injection (FDI) attacks on Cooperative Adaptive Cruise Control (CACC). Our method uses sensor redu…
Optimal Complexity in Byzantine-Robust Distributed Stochastic Optimization with Data Heterogeneity
Qiankun Shi, Jie Peng, Kun Yuan +2
In this paper, we establish tight lower bounds for Byzantine-robust distributed first-order stochastic optimization methods in both strongly convex and non-convex stochastic optimi…