Sparse and Continuous Attention Mechanisms

André F. T. Martins, António Farinhas, Marcos Treviso +3

Exponential families are widely used in machine learning; they include many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical di…