6 papers
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar +1
On-Policy Self-Distillation (OPSD) is commonly interpreted as the transfer of privileged information: a teacher observes the verified solution to the target problem and supervises…
Reliable Chain-of-Thought via Prefix Consistency
Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar +1
Large Language Models often improve accuracy on reasoning tasks by sampling multiple Chain-of-Thought (CoT) traces and aggregating them with majority voting (MV), a test-time techn…
STARS: Synchronous Token Alignment for Robust Supervision in Large Language Models
Mohammad Atif Quamar, Mohammad Areeb, Mikhail Kuznetsov +2
Aligning large language models (LLMs) with human values is crucial for safe deployment. Inference-time techniques offer granular control over generation; however, they rely on mode…
Learning Modal-Mixed Chain-of-Thought Reasoning with Latent Embeddings
Yifei Shao, Kun Zhou, Ziming Xu +5
We study how to extend chain-of-thought (CoT) beyond language to better handle multimodal reasoning. While CoT helps LLMs and VLMs articulate intermediate steps, its text-only form…
Logit-Entropy Adaptive Stopping Heuristic for Efficient Chain-of-Thought Reasoning
Mohammad Atif Quamar, Mohammad Areeb
Chain-of-Thought (CoT) prompting is a key technique for enabling complex reasoning in large language models. However, generating full, fixed-length rationales is computationally wa…
Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models
Mohammad Atif Quamar, Mohammad Areeb, Nishant Sharma +5
LLM alignment remains a critical challenge. Inference-time methods provide a flexible alternative to fine-tuning, but their uniform computational effort often yields suboptimal ali…