1 citations · 1 across the 19 of their papers we have counts for
23 papers
A matching lower bound for the sextic large sieve and structured prime moments
Yuanda Xu, Hejian Sang
We study the contrast between the large-sieve norm and the second moment of structured prime coefficients for sextic residue characters over the Eisenstein integers. For squarefree…
Co-DSL: Human-LLM Co-design of Reusable Spatial Concepts for Layout Generation
R. Kenny Jones, Jiajun Wu, Maneesh Agrawala
We present Co-DSL, a framework in which users and LLMs co-design reusable spatial concepts for layout generation. In the first stage, our system allows users to create a library of…
Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy Learning
Hejian Sang, Yuanda Xu, Zhengze Zhou +4
Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollout from one prompt domain and their sign…
AvoKV-E: Payload-Aware KV Cache Eviction for Long Reasoning
Han Yu, Wenhui Zhu, Xiwen Chen +9
Long-output reasoning shifts the KV-cache bottleneck from the fixed prompt to the generated trace. Existing reasoning-cache eviction methods largely treat cached entries as routing…
LatentPress: Context Compression Beyond Text and Vision
Zhengze Zhou, Yuanda Xu, Hejian Sang +3
Compressed context is usually carried as human-readable text or as rendered images that must be decoded, even when its consumer is a language model. We introduce LatentPress, which…
TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning
Yuanda Xu, Zhengze Zhou, Hejian Sang +6
Agentic reinforcement learning requires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions. Standard GRPO…
TREK: Distill to Explore, Reinforce to Refine
Yuanda Xu, Zhengze Zhou, Kayhan Behdin +10
Group Relative Policy Optimization (GRPO) is effective when the current policy already samples useful reasoning trajectories, but it stalls on hard prompts whose correct solution m…
TIP: Token Importance in On-Policy Distillation
Yuanda Xu, Hejian Sang, Zhengze Zhou +3
On-policy knowledge distillation (OPD) trains a student on its own rollouts under token-level supervision from a teacher. Not all token positions matter equally, but existing views…
Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
Yuanda Xu, Hejian Sang, Zhengze Zhou +3
In settings where labeled verifiable training data is the binding constraint, each checked example should be allocated to the model and reward density where it is most informative.…
PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
Yuanda Xu, Hejian Sang, Zhengze Zhou +2
Standard LLM distillation treats all training problems equally -- wasting compute on problems the student has already mastered or cannot yet solve. We empirically show that this in…
SODA: Semi On-Policy Black-Box Distillation for Large Language Models
Xiwen Chen, Jingjing Wang, Wenhui Zhu +7
Black-box knowledge distillation for large language models presents a strict trade-off. Simple off-policy methods (e.g., sequence-level knowledge distillation) struggle to correct…
CRISP: Compressed Reasoning via Iterative Self-Policy Distillation
Hejian Sang, Yuanda Xu, Zhengze Zhou +3
Reasoning models often generate far more tokens than a task requires, which raises inference cost and can compound errors. We introduce CRISP (Compressed Reasoning via Iterative Se…
Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning
Yuanda Xu, Hejian Sang, Zhengze Zhou +2
Reinforcement Learning with Verifiable Rewards (RLVR) has become the leading paradigm for enhancing reasoning in Large Language Models (LLMs). However, standard RLVR algorithms suf…
Not all tokens are needed(NAT): token efficient reinforcement learning
Hejian Sang, Yuanda Xu, Zhengze Zhou +2
Reinforcement learning (RL) has become a key driver of progress in large language models, but scaling RL to long chain-of-thought (CoT) trajectories is increasingly constrained by…
AriadneMem: Threading the Maze of Lifelong Memory for LLM Agents
Wenhui Zhu, Xiwen Chen, Zhipeng Wang +11
Long-horizon LLM agents require memory systems that remain accurate under fixed context budgets. However, existing systems struggle with two persistent challenges in long-term dial…
Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
Xiaofeng Lin, Sirou Zhu, Yilei Chen +6
Large language models (LLMs) achieve strong performance when all task-relevant information is available upfront, as in static prediction and instruction-following problems. However…
Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5
Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data. However, knowledge disti…
Aligning Diffusion Language Models via Unpaired Preference Optimization
Vaibhav Jindal, Hejian Sang, Chun-Mao Lai +2
Diffusion language models (dLLMs) are an emerging alternative to autoregressive (AR) generators, but aligning them to human preferences is challenging because sequence log-likeliho…
Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search
Kayhan Behdin, Qingquan Song, Sriram Vasudevan +17
Large Language Models (LLMs) have demonstrated impressive quality when applied to predictive tasks such as relevance ranking and semantic search. However, deployment of such LLMs r…
Planner-R1: Reward Shaping Enables Efficient Agentic RL with Smaller LLMs
Siyu Zhu, Yanbin Jiang, Hejian Sang +6
We investigated Agentic RL with large language models on the \textsc{TravelPlanner} benchmark. Our approach, \textsc{Planner-R1}, achieved a \textbf{56.9\%} final-pass rate with on…
Debunk the Myth of SFT Generalization
Xiaofeng Lin, Hejian Sang, Zhipeng Wang +1
A prevailing view holds that supervised fine-tuning (SFT) memorizes training data and fails to generalize, whereas reinforcement learning (RL) attains broader robustness. We revisi…
PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
Kuntai Du, Bowen Wang, Chen Zhang +9
Besides typical generative applications, like ChatGPT, GitHub Copilot, and Cursor, we observe an emerging trend that LLMs are increasingly used in traditional discriminative tasks,…
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song +17
Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Al…