FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning

Waleed Razzaq, Yun-Bo Zhao

Continuous-time (CT) Transformers improve irregular and long-range modeling over CT-RNNs by exploiting inputs or outputs embeddings with continuous dynamics. However, the core scal…