5 papers · 1 filter
CopySpec: Accelerating LLMs with Speculative Copy-and-Paste Without Compromising Quality
Razvan-Gabriel Dumitru, Minglai Yang, Vikas Yadav +1
We introduce CopySpec, a simple yet effective technique to tackle the inefficiencies LLMs face when generating responses that closely resemble previous outputs or responses that ca…
ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models
Razvan-Gabriel Dumitru, Darius Peteleaza, Vikas Yadav +1
Large language models excel at complex tasks by breaking down problems into structured reasoning steps. However, reasoning traces often extend beyond reaching a correct answer, cau…
Change Is the Only Constant: Dynamic LLM Slicing based on Layer Redundancy
Razvan-Gabriel Dumitru, Paul-Ioan Clotan, Vikas Yadav +2
This paper introduces a novel model compression approach through dynamic layer-specific pruning in Large Language Models (LLMs), enhancing the traditional methodology established b…
Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
Razvan-Gabriel Dumitru, Vikas Yadav, Rishabh Maheshwary +3
We present a simple meta quantization approach that quantizes different layers of a large language model (LLM) at different bit levels, and is independent of the underlying quantiz…
ELLEN: Extremely Lightly Supervised Learning For Efficient Named Entity Recognition
Haris Riaz, Razvan-Gabriel Dumitru, Mihai Surdeanu
In this work, we revisit the problem of semi-supervised named entity recognition (NER) focusing on extremely light supervision, consisting of a lexicon containing only 10 examples…