4 citations · 14 across the 32 of their papers we have counts for
7 papers · 2 filters
Drift No More? Context Equilibria in Multi-Turn LLM Interactions
Vardhan Dongre, Ryan A. Rossi, Viet Dac Lai +3
Large Language Models (LLMs) excel at single-turn tasks such as instruction following and summarization, yet real-world deployments require sustained multi-turn interactions where…
Steering MoE LLMs via Expert (De)Activation
Mohsen Fayyaz, Ali Modarressi, Hanieh Deilamsalehy +5
Mixture-of-Experts (MoE) in Large Language Models (LLMs) routes each token through a subset of specialized Feed-Forward Networks (FFN), known as experts. We present SteerMoE, a fra…
Lizard: An Efficient Linearization Framework for Large Language Models
Chien Van Nguyen, Huy Nguyen, Ruiyi Zhang +10
We propose Lizard, a linearization framework that transforms pretrained Transformer-based Large Language Models (LLMs) into subquadratic architectures. Transformers faces severe co…
Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact
Hyunji Lee, Seunghyun Yoon, Yunjae Won +7
Instruction tuning is a widely used approach to improve the instruction-following ability of large language models (LLMs). Instruction-tuning datasets typically include a mixture o…
Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization
Subhojyoti Mukherjee, Viet Dac Lai, Raghavendra Addanki +6
Offline reinforcement learning (RL) is a variant of RL where the policy is learned from a previously collected dataset of trajectories and rewards. In our work, we propose a practi…
CORG: Generating Answers from Complex, Interrelated Contexts
Hyunji Lee, Franck Dernoncourt, Trung Bui +1
In a real-world corpus, knowledge frequently recurs across documents but often contains inconsistencies due to ambiguous naming, outdated information, or errors, leading to complex…
NoLiMa: Long-Context Evaluation Beyond Literal Matching
Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt +4
Recent large language models (LLMs) support long contexts ranging from 128K to 1M tokens. A popular method for evaluating these capabilities is the needle-in-a-haystack (NIAH) test…