1 citations · 1 across the 8 of their papers we have counts for
Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
Jingze Shi, Zhangyang Peng, Yizhang Zhu +3
Mixture-of-Experts (MoE) architectures are evolving towards finer granularity to improve parameter efficiency. However, existing MoE designs face an inherent trade-off between the…
cs.CL2025★ 1 cited
TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
Bingheng Wu, Jingze Shi, Yifan Wu +2
Transformers exhibit proficiency in capturing long-range dependencies, whereas State Space Models (SSMs) facilitate linear-time sequence modeling. Notwithstanding their synergistic…
cs.CL2024
OTCE: Hybrid SSM and Attention with Cross Domain Mixture of Experts to construct Observer-Thinker-Conceiver-Expresser
Jingze Shi, Ting Xie, Bingheng Wu +2
Recent research has shown that combining Mamba with Transformer architecture, which has selective state space and quadratic self-attention mechanism, outperforms using Mamba or Tra…