37 citations · 39 across the 7 of their papers we have counts for
7 papers
The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models
Liuyuan Wen, Xun Zhu, Lihao Huang +2
Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual str…
Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation
Xunyu Zhu, Jian Li, Can Ma +1
Large Language Models (LLMs) demonstrate exceptional reasoning capabilities, often achieving state-of-the-art performance in various tasks. However, their substantial computational…
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Xunyu Zhu, Jian Li, Can Ma +1
Large Language Models (LLMs) have demonstrated exceptional proficiency in mathematical reasoning tasks due to their extensive parameter counts and training on vast datasets. Despit…
Distilling Mathematical Reasoning Capabilities into Small Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
This work addresses the challenge of democratizing advanced Large Language Models (LLMs) by compressing their mathematical reasoning capabilities into sub-billion parameter Small L…
A Survey on Model Compression for Large Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
Large Language Models (LLMs) have transformed natural language processing tasks successfully. Yet, their large size and high computational needs pose challenges for practical use,…
Robust Neural Architecture Search
Xunyu Zhu, Jian Li, Yong Liu +1
Neural Architectures Search (NAS) becomes more and more popular over these years. However, NAS-generated models tends to suffer greater vulnerability to various malicious attacks.…
Improving Differentiable Architecture Search via Self-Distillation
Xunyu Zhu, Jian Li, Yong Liu +1
Differentiable Architecture Search (DARTS) is a simple yet efficient Neural Architecture Search (NAS) method. During the search stage, DARTS trains a supernet by jointly optimizing…