4 papers · 1 filter
Efficient Compositional Multi-tasking for On-device Large Language Models
Ondrej Bohdal, Mete Ozay, Jijoong Moon +3
Adapter parameters provide a mechanism to modify the behavior of machine learning models and have gained significant popularity in the context of large language models (LLMs) and g…
Clustering-driven Memory Compression for On-device Large Language Models
Ondrej Bohdal, Pramit Saha, Umberto Michieli +2
Large language models (LLMs) often rely on user-specific memories distilled from past interactions to enable personalized generation. A common practice is to concatenate these memo…
On-device System of Compositional Multi-tasking in Large Language Models
Ondrej Bohdal, Konstantinos Theodosiadis, Asterios Mpatziakas +10
Large language models (LLMs) are commonly adapted for diverse downstream tasks via parameter-efficient fine-tuning techniques such as Low-Rank Adapters (LoRA). While adapters can b…
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
Hasan Abed Al Kader Hammoud, Umberto Michieli, Fabio Pizzati +4
Merging Large Language Models (LLMs) is a cost-effective technique for combining multiple expert LLMs into a single versatile model, retaining the expertise of the original ones. H…