2 citations · 3 across the 18 of their papers we have counts for
18 papers
Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
Qiyuan Zhu, Dezhi Li, Pengyu Cheng +8
Large Reasoning Models (LRMs) excel on complex tasks through long chain-of-thought (CoT) reasoning, but their lengthy intermediate steps cause severe overthinking that inflates inf…
LLMSurgeon: Diagnosing Data Mixture of Large Language Models
Yaxin Luo, Jiacheng Cui, Xiaohan Zhao +5
The pretraining data mixture of Large Language Models (LLMs) constitutes their "digital DNA", shaping model behaviors, capabilities, and failure modes. Yet this composition is rare…
From Fact Overwriting to Knowledge Evolution: Causal Editing via On-Policy Self-Distillation
Shuaike Li, Kai Zhang, Xianquan Wang +2
While Knowledge Editing (KE) enables efficient updates, its dominant Static Fact Overwriting paradigm treats LLMs as discrete databases, forcibly injecting isolated facts. Fracturi…
Dynamic Video Generation: Shaping Video Generation Across Time and Space
Shikang Zheng, Jingkai Huang, Jiacheng Liu +5
Diffusion models have achieved impressive performance in video generation, but their iterative denoising process remains computationally expensive due to the large number of tokens…
Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion
Peiliang Cai, Evelyn Zhang, Jiacheng Liu +8
Recent advances in autoregressive video diffusion have enabled sequential and streaming video generation. However, long-horizon generation requires increasingly large KV caches, ma…
IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
Xintong Yang, Hao Gu, Binxing Xu +6
Large Language Models (LLMs) are increasingly expected to operate over long contexts, yet standard softmax attention incurs a KV cache that grows linearly with sequence length, qui…
AI for Auto-Research: Roadmap & User Guide
Lingdong Kong, Xian Sun, Wei Chow +17
AI-assisted research is crossing a threshold: fully automated systems can now generate research papers for as little as $15, while long-horizon agents can execute experiments, draf…
Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
Binxing Xu, Hao Gu, Lujun Li +8
Training LLMs at ultra-low precision remains a formidable challenge. Direct low-bit QAT often suffers from convergence instability and substantial training costs, exacerbated by qu…
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
Hao Gu, Hao Wang, Jiacheng Liu +9
Large language model (LLM) reinforcement learning (RL) pipelines are often bottlenecked by rollout generation, making end-to-end training slow. Recent work mitigates this by runnin…
Just-in-Time Catching Test Generation at Meta
Matthew Becker, Yifei Chen, Nicholas Cochran +12
We report on Just-in-Time catching test generation at Meta, designed to prevent bugs in large scale backend systems of hundreds of millions of line of code. Unlike traditional hard…
OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
Liujianfu Wang, Yuyang Du, Yuchen Pan +3
Mixture-of-Experts (MoE), while offering significant advantages as a Large Language Model (LLM) architecture, faces substantial challenges when deployed on low-cost edge devices wi…
Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Data
Jiacheng Liu, Mayi Xu, Qiankun Pi +5
Large Language Models (LLMs) are increasingly employed in applications that require processing information from heterogeneous formats, including texts, tables, infoboxes, and knowl…
DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
Tianteng Gu, Bei Liu, Bo Xiao +3
Pruning is a widely used technique to compress large language models (LLMs) by removing unimportant weights, but it often suffers from significant performance degradation - especia…
ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
Tong Chen, Faeze Brahman, Jiacheng Liu +5
Language models (LMs) can memorize and reproduce segments from their pretraining data verbatim even in non-adversarial settings, raising concerns about copyright, plagiarism, priva…
DataDecide: How to Predict Best Pretraining Data with Small Experiments
Ian Magnusson, Nguyen Tai, Ben Bogin +10
Because large language models are expensive to pretrain on different datasets, using smaller-scale experiments to decide on data is crucial for reducing costs. Which benchmarks and…
Efficient Test-Time Scaling via Self-Calibration
Chengsong Huang, Langlin Huang, Jixuan Leng +2
Increasing test-time computation is a straightforward approach to enhancing the quality of responses in Large Language Models (LLMs). While Best-of-N sampling and Self-Consistency…
AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text
Ximing Lu, Melanie Sclar, Skyler Hallinan +8
Creativity has long been considered one of the most difficult aspect of human intelligence for AI to mimic. However, the rise of Large Language Models (LLMs), like ChatGPT, has rai…
Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning
Filippos Christianos, Georgios Papoudakis, Matthieu Zimmer +13
A key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL). However, constructing a standalone RL policy that maps perception to action directly e…