6 papers
TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding
Jiaming Fan, Daming Cao, Canchen Huang +6
Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing prop…
SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration
Yuyao Zhang, Junjie Gao, Zhengxian Wu +11
Recent advances in Tool-Integrated Large Language Models have made web search a core capability of information-seeking agents. However, as interaction histories grow, agents increa…
RedVLA: Physical Red Teaming for Vision-Language-Action Models
Yuhao Zhang, Borong Zhang, Jiaming Fan +4
The real-world deployment of Vision-Language-Action (VLA) models remains limited by the risk of unpredictable and irreversible physical harm. However, we currently lack effective m…
Flatter Tokens are More Valuable for Speculative Draft Model Training
Jiaming Fan, Daming Cao, Xiangzhong Luo +3
Speculative Decoding (SD) is a key technique for accelerating Large Language Model (LLM) inference, but it typically requires training a draft model on a large dataset. We approach…
Fast Large Language Model Collaborative Decoding via Speculation
Jiale Fu, Yuchu Jiang, Junkai Chen +3
Large Language Model (LLM) collaborative decoding techniques improve output quality by combining the outputs of multiple models at each generation step, but they incur high computa…
GraphIC: A Graph-Based In-Context Example Retrieval Model for Multi-Step Reasoning
Jiale Fu, Yaqing Wang, Simeng Han +2
In-context learning (ICL) enhances large language models (LLMs) by incorporating demonstration examples, yet its effectiveness heavily depends on the quality of selected examples.…