5 papers
BODHI: Do LLMs Branch Out and Discover Heterogeneous Inferences?
Soumadeep Saha, Krish Sharma, Akshay Chaturvedi +1
Although reinforcement learning with verifiable rewards (RLVR) has improved the performance of large language models (LLMs) across a variety of reasoning tasks, there is significan…
TAPIOCA: Why Task- Aware Pruning Improves OOD model Capability
Krish Sharma, Omar Naim, Soumadeep Saha +3
Recent work has promoted task-aware layer pruning as a way to improve model performance on particular tasks, as shown by TALE. In this paper, we investigate when such improvements…
TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination
Omar Naim, Krish Sharma, Niyar R Barman +1
Large Language Models (LLMs) typically come with a fixed architecture, despite growing evidence that not all layers contribute equally to every downstream task. We introduce TALE (…
Analyzing limits for in-context learning
Omar Naim, Jerome Bolte, Nicholas Asher
Our paper challenges claims from prior research that transformer-based models, when learning in context, implicitly implement standard learning algorithms. We present empirical evi…
Re-examining learning linear functions in context
Omar Naim, Guilhem Fouilhé, Nicholas Asher
In-context learning (ICL) has emerged as a powerful paradigm for easily adapting Large Language Models (LLMs) to various tasks. However, our understanding of how ICL works remains…