1 citations · 2 across the 24 of their papers we have counts for
8 papers · 1 filter
Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
Xin Gao, Xingming Xu
Transformer architectures form the foundation of modern natural language processing, yet the Key-Value (KV) cache introduces substantial memory and bandwidth overhead during long-c…
Closing the Data Loop: Using OpenDataArena to Engineer Superior Training Datasets
Xin Gao, Xiaoyang Wang, Yun Zhu +3
The construction of Supervised Fine-Tuning (SFT) datasets is a critical yet under-theorized stage in the post-training of Large Language Models (LLMs), as prevalent practices often…
Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs
Xin Gao, Ruiyi Zhang, Daniel Du +3
Large Language Models (LLMs) are widely used for temporal prediction, but their reliance on pretraining data raises contamination concerns, as accurate predictions on pre-cutoff te…
Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning
Honglin Lin, Qizhi Pei, Xin Gao +5
Reasoning capability is pivotal for Large Language Models (LLMs) to solve complex tasks, yet achieving reliable and scalable reasoning remains challenging. While Chain-of-Thought (…
TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligence
Guiyang Hou, Xing Gao, Yuchuan Wu +8
Recently, Large Language Models (LLMs) have made significant progress in IQ-related domains that require careful thinking, such as mathematics and coding. However, enhancing LLMs'…
A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis
Xin Gao, Qizhi Pei, Zinan Tang +5
While data synthesis and distillation are promising strategies to enhance small language models, current approaches heavily rely on Large Language Models (LLMs), which suffer from…