6 papers
ReMiT: RL-Guided Mid-Training for Iterative LLM Evolution
Junjie Huang, Jiarui Qin, Di Yin +4
Standard training pipelines for large language models (LLMs) are typically unidirectional, progressing from pre-training to post-training. However, the potential for a bidirectiona…
Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
Jiayi Kuang, Yinghui Li, Xin Zhang +5
Large language model-based agents show promise for software engineering, but environment configuration remains a bottleneck due to heavy manual effort and scarce large-scale, high-…
APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training
Jiarui Qin, Yunjia Xi, Junjie Huang +6
With the rapid development of LLM-based agents, there is a growing trend to incorporate agent-specific data into the pre-training stage of LLMs, aiming to better align LLMs with re…
CoDiEmb: A Collaborative yet Distinct Framework for Unified Representation Learning in Information Retrieval and Semantic Textual Similarity
Bowen Zhang, Zixin Song, Chunquan Chen +3
Learning unified text embeddings that excel across diverse downstream tasks is a central goal in representation learning, yet negative transfer remains a persistent obstacle. This…
Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning
Junnan Dong, Siyu An, Yifei Yu +6
Graph retrieval-augmented generation (GraphRAG) has effectively enhanced large language models in complex reasoning by organizing fragmented knowledge into explicitly structured gr…
TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference
Xiaojuan Tang, Fanxu Meng, Pingzhi Tang +4
Multi-Head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory. In tensor parall…