MossNet: Mixture of State-Space Experts is a Multi-Head Attention

Shikhar Tuli, James Seale Smith, Haris Jeelani +5

Large language models (LLMs) have significantly advanced generative applications in natural language processing (NLP). Recent trends in model architectures revolve around efficient…