5 papers
On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers
Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5
We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems. In this perspective, tokens are modeled as particles…
Optimal drift optimizer for non-convex optimization
Qin Li, Sixu Li, Eitan Tadmor +1
We study a finite-horizon stochastic control criterion for non-convex optimization in which Brownian exploration is balanced against a quadratic control cost. Rather than emphasizi…
Learning Higher-Order Structure from Incomplete Spatiotemporal Data: Multi-Scale Hypergraph Laplacians with Neural Refinement
Keshu Wu, Sixu Li, Zihao Li +3
Sensor networks increasingly govern modern infrastructure, yet the data they lose are rarely missing in the uniform-random patterns assumed by standard imputation benchmarks. Loop…
Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
Nicolás GarcÃa Trillos, Aditya Kumar Akash, Sixu Li +2
Adversarial attacks pose significant challenges in many machine learning applications, particularly in the setting of distributed training and federated learning, where malicious a…
CBO: Consensus-Based Bi-Level Optimization
Nicolás GarcÃa Trillos, Sixu Li, Konstantin Riedl +1
Bi-level optimization problems, where one wishes to find the global minimizer of an upper-level objective function over the globally optimal solution set of a lower-level objective…