11 papers
MoG: Mixture of Experts for Graph-based Retrieval-Augmented Generation
Zheng Yuan, Chuang Zhou, Linhao Luo +4
Retrieval-augmented generation is intensively studied to ground large language models on external evidence. However, retrieving from a unified knowledge base could inevitably intro…
Toward Native Multimodal Modeling: A Roadmap
Siyu An, Junru Lu, Junnan Dong +18
Multimodal modeling represents a vital step from modality-agnostic reasoning toward world modeling. While early approaches predominantly rely on late-fusion that assembles encoders…
HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
Yufei Xu, Fanxu Meng, Fan Jiang +11
Token-level sparse attention mechanisms, exemplified by DeepSeek Sparse Attention (DSA), achieve fine-grained key selection by scoring every historical key for each query through a…
Deep Tabular Research via Continual Experience-Driven Execution
Junnan Dong, Chuang Zhou, Zheng Yuan +7
Large language models often struggle with complex long-horizon analytical tasks over unstructured tables, which typically feature hierarchical and bidirectional headers and non-can…
PoLi-RL: A Point-to-List Reinforcement Learning Framework for Conditional Semantic Textual Similarity
Zixin Song, Bowen Zhang, Qian-Wen Zhang +3
Conditional Semantic Textual Similarity (C-STS) measures the semantic proximity between text segments under a specific condition, thereby overcoming the ambiguity inherent in tradi…
ReMiT: RL-Guided Mid-Training for Iterative LLM Evolution
Junjie Huang, Jiarui Qin, Di Yin +4
Standard training pipelines for large language models (LLMs) are typically unidirectional, progressing from pre-training to post-training. However, the potential for a bidirectiona…