50 citations · 240 across the 54 of their papers we have counts for
6 papers · 1 filter
FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization
Hang Zou, Chao Zhang, Yuzhi Yang +3
Federated Learning (FL) enables privacy-preserving fine-tuning of Large Language Models (LLMs), yet the massive communication overhead remains a critical bottleneck. Furthermore, a…
ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression
Wenya Yu, Chao Zhang, Li Wang +2
Post-Training Quantization (PTQ) and Low-Rank Adaptation (LoRA) constitute the standard pipeline for efficient Large Language Model (LLM) deployment. However, applying them sequent…
Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models?
Christophe El Zeinaty, Wassim Hamidouche, Glenn Herrou +2
This paper introduces a novel framework for designing efficient neural network architectures specifically tailored to tiny machine learning (TinyML) platforms. By leveraging large…
Reasoning Beyond Limits: Advances and Open Problems for LLMs
Mohamed Amine Ferrag, Norbert Tihanyi, Merouane Debbah
Recent breakthroughs in generative reasoning have fundamentally reshaped how large language models (LLMs) address complex tasks, enabling them to dynamically retrieve, refine, and…
SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead
Minsu Kim, Walid Saad, Merouane Debbah +1
The large communication and computation overhead of federated learning (FL) is one of the main challenges facing its practical deployment over resource-constrained clients and syst…
How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou +2
The phenomenon of model collapse, introduced in (Shumailov et al., 2023), refers to the deterioration in performance that occurs when new models are trained on synthetic data gener…