1 citations · 1 across the 11 of their papers we have counts for
13 papers
Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting
Zijie Liu, Zhen Tan, Charles Fleming +1
Adapting language models to new domains during post-training risks degrading previously acquired capabilities, a phenomenon commonly known as catastrophic forgetting. Existing miti…
Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness
Songyuan Sui, Zhen Tan, Mohan Zhang +3
Missing modality remains a longstanding challenge in multimodal learning. Existing methods typically address this issue through modality recovery or adaptive strategies. However, t…
MASkills: Continual Skills Optimization for Multi-Agent LLM Systems
Huaiyuan Yao, Xiaoou Liu, Charles Fleming +2
LLM-based multi-agent systems have shown strong performance on complex tasks, yet continual improvement from interaction experience remains challenging. Existing self-reflection me…
The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability
Abigail Woodring, Adrian Chan, Rana Muhammad Shahroz Khan +3
Fine-tuning has been widely used to adapt large language models (LLMs) for domain-specific tasks. Parameter efficient fine-tuning (PEFT) methods such as low-rank adaptation (LoRA)…
LangMARL: Natural Language Multi-Agent Reinforcement Learning
Huaiyuan Yao, Longchao Da, Xiaoou Liu +3
Large language model (LLM) agents struggle to autonomously evolve coordination strategies in dynamic environments, largely because coarse global outcomes obscure the causal signals…
Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs
Kangqi Ni, Wenyue Hua, Xiaoxiang Shi +3
Multi-agent applications often execute complex tasks as multi-stage workflows, where each stage is an LLM call whose output becomes part of context for subsequent steps. Existing L…
Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
Mingyu Jin, Yutong Yin, Jingcheng Niu +7
In this work, we investigate how Large Language Models (LLMs) adapt their internal representations when encountering inputs of increasing difficulty, quantified as the degree of ou…
TMS: Trajectory-Mixed Supervision for Reward-Free, On-Policy SFT
Rana Muhammad Shahroz Khan, Zijie Liu, Zhen Tan +2
Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT) are the two dominant paradigms for enhancing Large Language Model (LLM) performance on downstream tasks. While RL gener…
DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation
Pingzhi Li, Zhen Tan, Mohan Zhang +3
Large Language Models (LLMs) represent substantial intellectual and economic investments, yet their effectiveness can inadvertently facilitate model imitation via knowledge distill…
The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
Ruichen Zhang, Rana Muhammad Shahroz Khan, Zhen Tan +3
Data-centric distillation, including data augmentation, selection, and mixing, offers a promising path to creating smaller, more efficient student Large Language Models (LLMs) that…
ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
Rana Muhammad Shahroz Khan, Dongwen Tang, Pingzhi Li +2
Parameter generation has emerged as a novel paradigm for neural network development, offering an alternative to traditional neural network training by synthesizing high-quality mod…
: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks
Rana Muhammad Shahroz Khan, Zhen Tan, Sukwon Yun +2
Most discussions about Large Language Model (LLM) safety have focused on single-agent settings but multi-agent LLM systems now create novel adversarial risks because their behavior…
BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment
Sizhe Wang, Yongqi Tong, Hengyuan Zhang +3
Reinforcement Learning with Human Feedback (RLHF) is the key to the success of large language models (LLMs) in recent years. In this work, we first introduce the concepts of knowle…