291 citations · 788 across the 42 of their papers we have counts for
8 papers · 1 filter
Benchmarking the Thinking Mode of Multimodal Large Language Models in Clinical Tasks
Jindong Hong, Tianjie Chen, Lingjie Luo +10
A recent advancement in Multimodal Large Language Models (MLLMs) research is the emergence of "reasoning MLLMs" that offer explicit control over their internal thinking processes (…
Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
Chuanyang Zheng, Jiankai Sun, Yihang Gao +13
Mixture-of-Experts (MoE) has become a cornerstone in recent state-of-the-art large language models (LLMs). Traditionally, MoE relies on as the router score funct…
SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Model
Wencheng Zhang, Shiqin Qiao, Lingjie Luo +9
With the widespread adoption of large language models (LLMs) in practical applications, selecting an appropriate model requires balancing not only performance but also operational…
SAS: Simulated Attention Score
Chuanyang Zheng, Jiankai Sun, Yihang Gao +12
The attention mechanism is a core component of the Transformer architecture. Various methods have been developed to compute attention scores, including multi-head attention (MHA),…
Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
Liliang Ren, Congcong Chen, Haoran Xu +11
Recent advances in language modeling have demonstrated the effectiveness of State Space Models (SSMs) for efficient sequence modeling. While hybrid architectures such as Samba and…
SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation
Ting Xu, Zhichao Huang, Jiankai Sun +2
We present Sequential Policy Optimization for Simultaneous Machine Translation (SeqPO-SiMT), a new policy optimization framework that defines the simultaneous machine translation (…
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation
Chuanyang Zheng, Yihang Gao, Han Shi +9
The attention mechanism is a fundamental component of the Transformer model, contributing to interactions among distinct tokens, in contrast to earlier feed-forward neural networks…
Lyra: Orchestrating Dual Correction in Automated Theorem Proving
Chuanyang Zheng, Haiming Wang, Enze Xie +6
Large Language Models (LLMs) present an intriguing avenue for exploration in the field of formal theorem proving. Nevertheless, their full potential, particularly concerning the mi…