6 papers
What Gets Activated: Uncovering Domain and Driver Experts in MoE Language Models
Guimin Hu, Meng Li, Qiwei Peng +3
Most interpretability work focuses on layer- or neuron-level mechanisms in Transformers, leaving expert-level behavior in MoE LLMs underexplored. Motivated by functional specializa…
ENTRA: Entropy-Based Redundancy Avoidance in Large Language Model Reasoning
Ruichu Cai, Haopeng Du, Qingwen Lin +3
Large Reasoning Models (LRMs) often suffer from overthinking, generating unnecessarily long reasoning chains even for simple tasks. This leads to substantial computational overhead…
Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation
Zhifeng Hao, Qibin Song, Ruichu Cai +1
Recent divide-and-conquer reasoning approaches, particularly those based on Chain-of-Thought (CoT), have substantially improved the Text-to-SQL capabilities of Large Language Model…
CAMA: Enhancing Mathematical Reasoning in Large Language Models with Causal Knowledge
Lei Zan, Keli Zhang, Ruichu Cai +1
Large Language Models (LLMs) have demonstrated strong performance across a wide range of tasks, yet they still struggle with complex mathematical reasoning, a challenge fundamental…
Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting
Wei Chen, Jiahao Zhang, Haipeng Zhu +5
Large language models (LLMs) have shown great potential in decision-making due to the vast amount of knowledge stored within the models. However, these pre-trained models are prone…
TP-UNet: Temporal Prompt Guided UNet for Medical Image Segmentation
Ranmin Wang, Limin Zhuang, Hongkun Chen +2
The advancement of medical image segmentation techniques has been propelled by the adoption of deep learning techniques, particularly UNet-based approaches, which exploit semantic…