From the 1 of 13 linked papers with an AI index.
10 papers
FARS: A Fully Automated Research System Deployed at Scale
Qiong Tang, Tianxiang Sun, Xiangkun Hu +54
FARS is an AI-driven system that autonomously creates research projects, runs experiments, and writes full AI/ML papers across many topics, and its output was evaluated through str…
Position Bias Correction is Insufficient for One-Pass Attention Sorting
Qiong Tang, Xiangkun Hu, Xiangyang Liu +2
Long-context language models suffer from position bias, where information in middle positions is underutilized. Attention Sorting addresses this by iteratively reordering documents…
NLL-Guided Full-Attention Layer Selection for Training-Free Sliding-Window Adaptation
Qiong Tang, Xiangkun Hu, Xiangyang Liu +2
Hybrid attention models that mix full and sliding-window attention across layers offer a promising approach to efficient long-context inference, but the critical question of \emph{…
Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study
Qiong Tang, Xiangkun Hu, Xiangyang Liu +2
Training-free compression methods for large language models (LLMs) often use calibration data to guide compression decisions. ROCKET, a recent method combining sparse-dictionary fa…
FastMCTS: A Simple Sampling Strategy for Data Synthesis
Peiji Li, Kai Lv, Yunfan Shao +5
Synthetic high-quality multi-step reasoning data can significantly enhance the performance of large language models on various tasks. However, most existing methods rely on rejecti…
Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
Zhaoye Fei, Yunfan Shao, Linyang Li +6
Large language models (LLMs) have demonstrated remarkable potential in various tasks, however, there remains a significant lack of open-source models and data for specific domains.…