10 papers
The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability
Abigail Woodring, Adrian Chan, Rana Muhammad Shahroz Khan +3
Fine-tuning has been widely used to adapt large language models (LLMs) for domain-specific tasks. Parameter efficient fine-tuning (PEFT) methods such as low-rank adaptation (LoRA)…
LangMARL: Natural Language Multi-Agent Reinforcement Learning
Huaiyuan Yao, Longchao Da, Xiaoou Liu +3
Large language model (LLM) agents struggle to autonomously evolve coordination strategies in dynamic environments, largely because coarse global outcomes obscure the causal signals…
Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs
Kangqi Ni, Wenyue Hua, Xiaoxiang Shi +3
Multi-agent applications often execute complex tasks as multi-stage workflows, where each stage is an LLM call whose output becomes part of context for subsequent steps. Existing L…
Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
Mingyu Jin, Yutong Yin, Jingcheng Niu +7
In this work, we investigate how Large Language Models (LLMs) adapt their internal representations when encountering inputs of increasing difficulty, quantified as the degree of ou…
The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
Ruichen Zhang, Rana Muhammad Shahroz Khan, Zhen Tan +3
Data-centric distillation, including data augmentation, selection, and mixing, offers a promising path to creating smaller, more efficient student Large Language Models (LLMs) that…
TMS: Trajectory-Mixed Supervision for Reward-Free, On-Policy SFT
Rana Muhammad Shahroz Khan, Zijie Liu, Zhen Tan +2
Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT) are the two dominant paradigms for enhancing Large Language Model (LLM) performance on downstream tasks. While RL gener…