2 papers
cs.LG2024
AttentionX: Exploiting Consensus Discrepancy In Attention from A Distributed Optimization Perspective
Guoqiang Zhang, Richard Heusdens
In this paper, we extend the standard Attention in transformer by exploiting the consensus discrepancy from a distributed optimization perspective, referred to as AttentionX. It is…
cs.DC2024
Distributed Nonlinear Conic Optimisation with partially separable Structure
Richard Heusdens, Guoqiang Zhang
In this paper we consider the problem of distributed nonlinear optimisation of a separable convex cost function over a graph subject to cone constraints. We show how to generalise,…