14 papers
Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting
Zijie Liu, Zhen Tan, Charles Fleming +1
Adapting language models to new domains during post-training risks degrading previously acquired capabilities, a phenomenon commonly known as catastrophic forgetting. Existing miti…
"very likely" Means "uncertain"? How LLMs Diverge from Humans in Linguistic Uncertainty Quantification
Jinhao Duan, Zicheng Liu, Zijie Liu +2
Humans express uncertainty verbally via markers (e.g., "possible," "likely"), yet most LLM uncertainty quantification (UQ) relies on costing likelihood- or consistency-based signal…
When Retain Constraints Conflict: Mitigating Forget-Retain Interference in Tabular Data
Zijie Liu, Jinhao Duan, Bingqi Shang +3
Machine unlearning aims to remove the influence of designated training data while preserving model utility, but its behavior on tabular data remains underexplored. This gap is impo…
DrugReason: Dynamic Multi-View Reasoning over Knowledge Graph and Language Evidence for Drug Repurposing
Zijie Liu, Hongxuan Li, Zhen Tan +5
Drug repurposing aims to identify new therapeutic uses for existing compounds and, compared with de novo drug discovery, offers a faster and more cost-effective path to clinical tr…
Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning
Zijie Liu, Jinhao Duan, Gaowen Liu +2
Machine unlearning for vision-language models (VLMs) remains underexplored. Unlike language models, VLMs combine a language backbone with visual components, which makes unlearning…
RAVEN: Agentic RAG for Automated Vulnerability Repair
Varun Gadey, Zijie Liu, Alexandra Dmitrienko
Automated vulnerability repair has emerged as a promising direction to mitigate the growing number of software vulnerabilities. Recent advances in Large Language Models (LLMs) have…
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
Jianing Deng, Song Wang, Dongwei Wang +4
Mixture-of-Experts Large Language Models (MoE-LLMs) achieve strong performance but incur substantial memory overhead due to massive expert parameters. Mixed-precision quantization…
SWE-Next: Scalable Real-World Software Engineering Tasks for Agents
Jiarong Liang, Zhiheng Lyu, Zijie Liu +4
Executable software engineering data is valuable for training SWE agents, but scaling it remains difficult for two reasons: only a small fraction of real repository changes yield v…
TMS: Trajectory-Mixed Supervision for Reward-Free, On-Policy SFT
Rana Muhammad Shahroz Khan, Zijie Liu, Zhen Tan +2
Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT) are the two dominant paradigms for enhancing Large Language Model (LLM) performance on downstream tasks. While RL gener…
A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs
Zijie Liu, Jie Peng, Jinhao Duan +7
Sparse Mixture-of-Experts (SMoE) architectures are increasingly used to scale large language models efficiently, delivering strong accuracy under fixed compute budgets. However, SM…
Can GRPO Help LLMs Transcend Their Pretraining Origin?
Kangqi Ni, Zhen Tan, Zijie Liu +2
Reinforcement Learning with Verifiable Rewards (RLVR), primarily driven by the Group Relative Policy Optimization (GRPO) algorithm, is a leading approach for enhancing the reasonin…
FIER: Fine-Grained and Efficient KV Cache Retrieval for Long-context LLM Inference
Dongwei Wang, Zijie Liu, Song Wang +5
The Key-Value (KV) cache reading latency increases significantly with context lengths, hindering the efficiency of long-context LLM inference. To address this, previous works propo…
Model Editing as a Double-Edged Sword: Steering Agent Ethical Behavior Toward Beneficence or Harm
Baixiang Huang, Zhen Tan, Haoran Wang +6
Agents based on Large Language Models (LLMs) have demonstrated strong capabilities across a wide range of tasks. However, deploying LLM-based agents in high-stakes domains comes wi…
Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations
Zijie Liu, Xinyu Zhao, Jie Peng +5
Current medical AI systems often fail to replicate real-world clinical reasoning, as they are predominantly trained and evaluated on static text and question-answer tasks. These tu…