2 papers
cs.CV2024
A Manifold Representation of the Key in Vision Transformers
Li Meng, Morten Goodwin, Anis Yazidi +1
Vision Transformers implement multi-head self-attention via stacking multiple attention blocks. The query, key, and value are often intertwined and generated within those blocks vi…
cs.LG2024
Maximum Manifold Capacity Representations in State Representation Learning
Li Meng, Morten Goodwin, Anis Yazidi +1
The expanding research on manifold-based self-supervised learning (SSL) builds on the manifold hypothesis, which suggests that the inherent complexity of high-dimensional data can…