Sparse Transformer Architectures via Regularized Wasserstein Proximal Operator with $L_1$ Prior

Fuqun Han, Stanley Osher, Wuchen Li

In this work, we propose a sparse transformer architecture that incorporates prior information about the underlying data distribution directly into the transformer structure of the…