6 papers
Efficient Compositional Multi-tasking for On-device Large Language Models
Ondrej Bohdal, Mete Ozay, Jijoong Moon +3
Adapter parameters provide a mechanism to modify the behavior of machine learning models and have gained significant popularity in the context of large language models (LLMs) and g…
Data-driven Clustering and Merging of Adapters for On-device Large Language Models
Ondrej Bohdal, Taha Ceritli, Mete Ozay +4
On-device large language models commonly employ task-specific adapters (e.g., LoRAs) to deliver strong performance on downstream tasks. While storing all available adapters is impr…
On-device System of Compositional Multi-tasking in Large Language Models
Ondrej Bohdal, Konstantinos Theodosiadis, Asterios Mpatziakas +10
Large language models (LLMs) are commonly adapted for diverse downstream tasks via parameter-efficient fine-tuning techniques such as Low-Rank Adapters (LoRA). While adapters can b…
FFT-based Selection and Optimization of Statistics for Robust Recognition of Severely Corrupted Images
Elena Camuffo, Umberto Michieli, Jijoong Moon +2
Improving model robustness in case of corrupted images is among the key challenges to enable robust vision systems on smart devices, such as robotic agents. Particularly, robust te…
HydraOpt: Navigating the Efficiency-Performance Trade-off of Adapter Merging
Taha Ceritli, Ondrej Bohdal, Mete Ozay +4
Large language models (LLMs) often leverage adapters, such as low-rank-based adapters, to achieve strong performance on downstream tasks. However, storing a separate adapter for ea…
Controllable Forgetting Mechanism for Few-Shot Class-Incremental Learning
Kirill Paramonov, Mete Ozay, Eunju Yang +2
Class-incremental learning in the context of limited personal labeled samples (few-shot) is critical for numerous real-world applications, such as smart home devices. A key challen…