1 citations · 1 across the 17 of their papers we have counts for
8 papers · 1 filter
You Only Index Once: Cross-Layer Sparse Attention with Shared Routing
Yutao Sun, Yanqi Zhang, Li Dong +2
Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially in reasoning-heavy settings where models generate long intermediate chains of t…
Universal YOCO for Efficient Depth Scaling
Yutao Sun, Li Dong, Tianzhu Ye +3
The rise of test-time scaling has remarkably boosted the reasoning and agentic proficiency of Large Language Models (LLMs). Yet, standard Transformers struggle to scale inference-t…
Talking to Yourself: Defying Forgetting in Large Language Models
Yutao Sun, Mingshuai Chen, Tiancheng Zhao +5
Catastrophic forgetting remains a major challenge when fine-tuning large language models (LLMs) on narrow, task-specific data, often degrading their general knowledge and reasoning…
Novel Parasitic Dual-Scale Modeling for Efficient and Accurate Multilingual Speech Translation
Chenyang Le, Yinfeng Xia, Huiyan Li +4
Recent advancements in speech-to-text translation have led to the development of multilingual models capable of handling multiple language pairs simultaneously. However, these unif…
Efficient Attention Mechanisms for Large Language Models: A Survey
Yutao Sun, Zhenyu Li, Yike Zhang +4
Transformer-based architectures have become the prevailing backbone of large language models. However, the quadratic time and memory complexity of self-attention remains a fundamen…
MFLA: Monotonic Finite Look-ahead Attention for Streaming Speech Recognition
Yinfeng Xia, Huiyan Li, Chenyang Le +4
Applying large pre-trained speech models like Whisper has shown promise in reducing training costs for various speech tasks. However, integrating these models into streaming system…
The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding?
Yutao Sun, Mingshuai Chen, Tiancheng Zhao +3
Self-improving large language models (LLMs) -- i.e., to improve the performance of an LLM by fine-tuning it with synthetic data generated by itself -- is a promising way to advance…
Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression
Zilun Zhang, Yutao Sun, Tiancheng Zhao +4
Humans can retain old knowledge while learning new information, but Large Language Models (LLMs) often suffer from catastrophic forgetting when post-pretrained or supervised fine-t…