8 papers
Self-Adapting Group of Experts for Multi-Agent Reasoning
Mohammad Atif Quamar, Nurbek Tastan, Karthik Nandakumar +1
Multi-agent systems bring together language model agents with different roles to propose, review, and refine solutions. Each agent's response depends on its model's capabilities, t…
Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models
Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar +1
Diffusion Language Models (DLMs) have recently become increasingly competitive with autoregressive (AR) models, and even outperform them on certain tasks. Unlike AR models, DLMs pr…
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar +1
On-Policy Self-Distillation (OPSD) is commonly interpreted as the transfer of privileged information: a teacher observes the verified solution to the target problem and supervises…
Reliable Chain-of-Thought via Prefix Consistency
Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar +1
Large Language Models often improve accuracy on reasoning tasks by sampling multiple Chain-of-Thought (CoT) traces and aggregating them with majority voting (MV), a test-time techn…
Learning Modal-Mixed Chain-of-Thought Reasoning with Latent Embeddings
Yifei Shao, Kun Zhou, Ziming Xu +5
We study how to extend chain-of-thought (CoT) beyond language to better handle multimodal reasoning. While CoT helps LLMs and VLMs articulate intermediate steps, its text-only form…
Logit-Entropy Adaptive Stopping Heuristic for Efficient Chain-of-Thought Reasoning
Mohammad Atif Quamar, Mohammad Areeb
Chain-of-Thought (CoT) prompting is a key technique for enabling complex reasoning in large language models. However, generating full, fixed-length rationales is computationally wa…
STARS: Synchronous Token Alignment for Robust Supervision in Large Language Models
Mohammad Atif Quamar, Mohammad Areeb, Mikhail Kuznetsov +2
Aligning large language models (LLMs) with human values is crucial for safe deployment. Inference-time techniques offer granular control over generation; however, they rely on mode…
Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models
Mohammad Atif Quamar, Mohammad Areeb, Nishant Sharma +5
LLM alignment remains a critical challenge. Inference-time methods provide a flexible alternative to fine-tuning, but their uniform computational effort often yields suboptimal ali…