4 papers
Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones
Ayoub Ghriss, Sourav Chakraborty
Linear attention promises constant-time recurrent inference but degrades sharply on associative recall. We formulate attention recall as a spherical-packing problem and introduce K…
S: Structured Sparsity Specification
Ayoub Ghriss
We introduce the Structured Sparsity Specification (S), an algebraic framework for defining, composing, and implementing structured sparse patterns. S specifies sparsity th…
Beyond Spectral Clustering: Probabilistic Cuts for Differentiable Graph Partitioning
Ayoub Ghriss
Probabilistic relaxations of graph cuts offer a differentiable alternative to spectral clustering, enabling end-to-end and online learning without eigendecompositions, yet prior wo…
Deep Clustering via Probabilistic Ratio-Cut Optimization
Ayoub Ghriss, Claire Monteleoni
We propose a novel approach for optimizing the graph ratio-cut by modeling the binary assignments as random variables. We provide an upper bound on the expected ratio-cut, as well…