7 papers
BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training
Zili Zhang, Chengxu Yang, Shenglong Zhang +8
Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. Existing systems redesign the training pipeline to address these challenges, b…
ConflictRAG: Detecting and Resolving Knowledge Conflicts in Retrieval Augmented Generation
Chenyu Wang, Yueyuan Li, Yingmin Liu +1
Retrieval-Augmented Generation (RAG) systems implicitly assume mutual consistency among retrieved documents -- an assumption that frequently fails in practice. We present ConflictR…
HRM-Text: Efficient Pretraining Beyond Scaling
Guan Wang, Changling Liu, Chenyu Wang +6
The current pretraining paradigm for large language models relies on massive compute and internet-scale raw text, creating a significant barrier to foundational research. In contra…
MetaCogAgent: A Metacognitive Multi-Agent LLM Framework with Self-Aware Task Delegation
Chenyu Wang, Yang Shu
Multi-agent large language model (LLM) systems have shown promise for solving complex tasks through agent collaboration. However, existing frameworks assign tasks based on predefin…
Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
Cai Zhou, Chenxiao Yang, Yi Hu +7
Diffusion language models, especially masked discrete diffusion models, have achieved great success recently. While there are some theoretical and primary empirical results showing…
Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning
Cai Zhou, Zekai Wang, Menghua Wu +6
While test-time scaling has enabled large language models to solve highly difficult tasks, state-of-the-art results come at exorbitant compute costs. These inefficiencies can be at…