Distilling Reasoning Capabilities into Smaller Language Models

Kumar Shridhar, Alessandro Stolfo, Mrinmaya Sachan

Step-by-step reasoning approaches like chain of thought (CoT) have proved to be very effective in inducing reasoning capabilities in large language models. However, the success of…