10 papers
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Siyang Lyu, Zhijing Sun, Xinghao Chen +3
Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students, but verbose teacher traces inflate both training and inference c…
What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translation
Shaomu Tan, Dawei Zhu, Ke Tran +5
Iterative self-refinement is a simple inference-time strategy for machine translation: an LLM revises its own translation over multiple inference-time passes. Yet document-scale re…
Language models can learn implicit multi-hop reasoning, but only if they have lots of training data
Yuekun Yao, Yupei Du, Dawei Zhu +2
Implicit reasoning is the ability of a language model to solve multi-hop reasoning tasks in a single forward pass, without chain of thought. We investigate this capability using GP…
SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Models
Yirong Sun, Yanjun Chen, Xin Qiu +8
Large Audio Language Models (LALMs) excel at semantic and paralinguistic tasks, yet their ability to perceive the fundamental physical attributes of audio such as pitch, loudness,…
PLD: A Choice-Theoretic List-Wise Knowledge Distillation
Ejafa Bassam, Dawei Zhu, Kaigui Bian
Knowledge distillation is a model compression technique in which a compact "student" network is trained to replicate the predictive behavior of a larger "teacher" network. In logit…
Same evaluation, more tokens: On the effect of input length for machine translation evaluation using Large Language Models
Tobias Domhan, Dawei Zhu
Accurately evaluating machine-translated text remains a long-standing challenge, particularly for long documents. Recent work has shown that large language models (LLMs) can serve…