9 papers
Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters
Ailin Huang, Ang Li, Aobo Kong +213
We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency. We focus on what matters most wh…
DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Training
Xinwei Qiang, Hongmin Chen, Shixuan Sun +3
Determinism is indispensable for reproducibility in large language model (LLM) training, yet it often exacts a steep performance cost. In widely used attention implementations such…
Focusing on Language: Revealing and Exploiting Language Attention Heads in Multilingual Large Language Models
Xin Liu, Qiyang Song, Qihang Zhou +5
Large language models (LLMs) increasingly support multilingual understanding and generation. Meanwhile, efforts to interpret their internal mechanisms have emerged, offering insigh…
WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidance
Genglin Liu, Shijie Geng, Sha Li +4
Multimodal LLM-powered agents have recently demonstrated impressive capabilities in web navigation, enabling agents to complete complex browsing tasks across diverse domains. Howev…
Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models
Xin Liu, Qiyang Song, Shaowen Xu +6
Large Language Models (LLMs) often retain inaccurate or outdated information from pre-training, leading to incorrect predictions or biased outputs during inference. While existing…
Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap
Yifan Sun, Yushan Liang, Zhen Zhang +2
Self-improvement is a significant techniques within the realm of large language model (LLM), aiming to enhance the LLM performance without relying on external data. Despite its sig…
MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science
Ran Xu, Yuchen Zhuang, Yishan Zhong +13
We introduce MedAgentGym, a scalable and interactive training environment designed to enhance coding-based biomedical reasoning capabilities in large language model (LLM) agents. M…
Matryoshka Model Learning for Improved Elastic Student Models
Chetan Verma, Aditya Srinivas Timmaraju, Cho-Jui Hsieh +7
Industry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development. In this paper, we propose…
BIG-Bench Extra Hard
Mehran Kazemi, Bahare Fatemi, Hritik Bansal +17
Large language models (LLMs) are increasingly deployed in everyday applications, demanding robust general reasoning capabilities and diverse reasoning skillset. However, current LL…