TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba

Xiuwei Chen, Wentao Hu, Xiao Dong +7

Transformer-based architectures have become the backbone of both uni-modal and multi-modal foundation models, largely due to their scalability via attention mechanisms, resulting i…