7 citations · 30 across the 16 of their papers we have counts for
3 papers · 1 filter
Tree Cross Attention
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Cross Attention is a popular method for retrieving information from a set of context tokens for making predictions. At inference time, for each prediction, Cross Attention scans th…
Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Modern foundation model architectures rely on attention mechanisms to effectively capture context. However, these methods require linear or quadratic memory in terms of the number…
Memory Efficient Neural Processes via Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi +2
Neural Processes (NPs) are popular meta-learning methods for efficiently modelling predictive uncertainty. Recent state-of-the-art methods, however, leverage expensive attention me…