13 citations · 18 across the 6 of their papers we have counts for
6 papers
DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
Yuxiang Huang, Nuno M. T. Gonçalves, Federico Alvetreti +5
Current hierarchical attention methods, such as NSA and InfLLMv2, select the top-k relevant key-value (KV) blocks based on coarse attention scores and subsequently apply fine-grain…
AdaSplash-2: Faster Differentiable Sparse Attention
Nuno Gonçalves, Hugo Pitorro, Vlad Niculae +4
Sparse attention has been proposed as a way to alleviate the quadratic cost of transformers, a central bottleneck in long-context training. A promising line of work is -entmax a…
Sparse Attention as Compact Kernel Regression
Saul Santos, Nuno Gonçalves, Daniel C. McNamee +2
Recent work has revealed a link between self-attention mechanisms in transformers and test-time kernel regression via the Nadaraya-Watson estimator, with standard softmax attention…
Neural Implicit Morphing of Face Images
Guilherme Schardong, Tiago Novello, Hallison Paz +4
Face morphing is a problem in computer graphics with numerous artistic and forensic applications. It is challenging due to variations in pose, lighting, gender, and ethnicity. This…
Reducing Overconfidence Predictions for Autonomous Driving Perception
Gledson Melotti, Cristiano Premebida, Jordan J. Bird +2
In state-of-the-art deep learning for object recognition, SoftMax and Sigmoid functions are most commonly employed as the predictor outputs. Such layers often produce overconfident…
Bio-Inspired Modality Fusion for Active Speaker Detection
Gustavo Assunção, Nuno Gonçalves, Paulo Menezes
Human beings have developed fantastic abilities to integrate information from various sensory sources exploring their inherent complementarity. Perceptual capabilities are therefor…