6 papers
Compressing LLMs with MoP: Mixture of Pruners
Bruno Lopes Yamamoto, Lucas Lauton de Alcantara, Victor Zacarias +7
The high computational demands of Large Language Models (LLMs) motivate methods that reduce parameter count and accelerate inference. In response, model pruning emerges as an effec…
Layer-wise LoRA fine-tuning: a similarity metric approach
Keith Ando Ogawa, Bruno Lopes Yamamoto, Lucas Lauton de Alcantara +5
Pre-training Large Language Models (LLMs) on web-scale datasets becomes fundamental for advancing general-purpose AI. In contrast, enhancing their predictive performance on downstr…
Technical Report on Text Dataset Distillation
Keith Ando Ogawa, Bruno Lopes Yamamoto, Lucas Lauton de Alcantara +6
In the vision domain, dataset distillation arises as a technique to condense a large dataset into a smaller synthetic one that exhibits a similar result in the training process. Wh…
The Virtues of Brevity: Avoid Overthinking in Parallel Test-Time Reasoning
Raul Cavalcante Dinardi, Bruno Yamamoto, Anna Helena Reali Costa +1
Reasoning models represent a significant advance in LLM capabilities, particularly for complex reasoning tasks such as mathematics and coding. Previous studies confirm that paralle…
Efficient LLMs with AMP: Attention Heads and MLP Pruning
Leandro Giusti Mugnaini, Bruno Lopes Yamamoto, Lucas Lauton de Alcantara +5
Deep learning drives a new wave in computing systems and triggers the automation of increasingly complex problems. In particular, Large Language Models (LLMs) have significantly ad…
Effective Layer Pruning Through Similarity Metric Perspective
Ian Pons, Bruno Yamamoto, Anna H. Reali Costa +1
Deep neural networks have been the predominant paradigm in machine learning for solving cognitive tasks. Such models, however, are restricted by a high computational overhead, limi…