3 papers
cs.LG2025
Layer Specialization Underlying Compositional Reasoning in Transformers
Jing Liu
Transformers exhibit compositional reasoning on sequences not observed during training, a capability often attributed to in-context learning (ICL) and skill composition. We investi…
cs.CL2025
A Survey on Parallel Reasoning
Ziqi Wang, Boye Niu, Zipeng Gao +10
With the increasing capabilities of Large Language Models (LLMs), parallel reasoning has emerged as a new inference paradigm that enhances reasoning robustness by concurrently expl…
cs.CV2024
Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher
Yong Guo, Shulian Zhang, Haolin Pan +3
Knowledge distillation aims to transfer knowledge from a large teacher model to a compact student counterpart, often coming with a significant performance gap between them. We find…