2 citations · 3 across the 9 of their papers we have counts for
9 papers
Distilling LLM Reasoning into Graph of Concept Predictors
Ziyang Yu, Liang Zhao
Deploying Large Language Models (LLMs) for discriminative workloads is often limited by inference latency, compute, and API costs at scale. Active distillation reduces these costs…
HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
Yizhao Gao, Jianyu Wei, Qihao Zhang +11
This work introduces Hybrid Sparse Attention (HySparse), a new architecture that interleaves each full attention layer with several sparse attention layers. While conceptually simp…
MiMo-Audio: Audio Language Models are Few-Shot Learners
Core Team, Dong Zhang, Gang Wang +97
Existing audio language models typically rely on task-specific fine-tuning to accomplish particular audio tasks. In contrast, humans are able to generalize to new audio tasks with…
ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems
Hassan Hamad, Yingru Xu, Liang Zhao +2
Tool-augmented large language models (LLMs) are increasingly employed in real-world applications, but tool usage errors still hinder their reliability. We introduce ToolCritic, a d…
CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysis
Xinzhe Xu, Liang Zhao, Hongshen Xu +1
Large Language Models (LLMs) are increasingly tasked with analyzing legal texts and citing relevant statutes, yet their reliability is often compromised by general pre-training tha…
MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining
LLM-Core Xiaomi, :, Bingquan Xia +62
We present MiMo-7B, a large language model born for reasoning tasks, with optimization across both pre-training and post-training stages. During pre-training, we enhance the data p…
MiMo-VL Technical Report
Core Team, Zihao Yue, Zhenru Lin +71
We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal rea…
The Wisdom of Agent Crowds: A Human-AI Interaction Innovation Ignition Framework
Senhao Yang, Qiwen Cheng, Ruiqi Ma +3
With the widespread application of large AI models in various fields, the automation level of multi-agent systems has been continuously improved. However, in high-risk decision-mak…
Unveiling and Controlling Anomalous Attention Distribution in Transformers
Ruiqing Yan, Xingbo Du, Haoyu Deng +7
With the advent of large models based on the Transformer architecture, researchers have observed an anomalous phenomenon in the Attention mechanism--there is a very high attention…