Natively Trainable Sparse Attention for Hierarchical Point Cloud Datasets

Nicolas Lapautre, Maria Marchenko, Carlos Miguel Patiño +1

Unlocking the potential of transformers on datasets of large physical systems depends on overcoming the quadratic scaling of the attention mechanism. This work explores combining t…