11 papers
Structured Thoughts For Improved Reasoning And Context Pruning
Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu +5
Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured T…
Know Thy Reasoner: Not All Language Models Explore Alike
Moulik Choraria, Argyrios Gerogiannis, Anirban Das +4
Compute scaling for LLM reasoning trades off exploring solution approaches (\emph{breadth}) against refining promising ones (\emph{depth}), yet why a given trade-off works, and why…
T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains
Genta Indra Winata, Amartya Chakraborty, Yuzhen Lin +12
Recent advances in reasoning and tool-calling capabilities of large language models (LLMs) have enabled increasingly capable agentic systems. However, existing benchmarks remain li…
AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
Duy Nguyen, Hanqi Xiao, Archiki Prasad +7
Self-distillation enables language models to learn on-policy from their own trajectories by using the same model as both student and teacher, with the teacher being conditioned on…
Can Large Language Models Understand, Reason About, and Generate Code-Switched Text?
Genta Indra Winata, David Anugraha, Patrick Amadeus Irawan +15
Code-switching is a pervasive phenomenon in multilingual communication, yet the robustness of large language models (LLMs) in mixed-language settings remains insufficiently underst…
Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien, Charles-Ãtienne Joseph, Zain Sarwar +7
Sparsely-activated Mixture of Experts (MoE) transformers are promising architectures for foundation models. Compared to dense transformers that require the same amount of floating-…