6 papers
Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devices
Mohamed Aboelenien Ahmed, Kilian Pfeiffer, Ramin Khalili +2
Federated Learning (FL) is a promising paradigm for finetuning Large Language Models (LLMs) across distributed data sources while preserving data privacy. However, finetuning such…
Co-Design of CNN Accelerators for TinyML using Approximate Matrix Decomposition
José Juan Hernández Morales, Georgios Mentzos, Frank Hannig +4
The paradigm shift towards local and on-device inference under stringent resource constraints is represented by the tiny machine learning (TinyML) domain. The primary goal of TinyM…
TransAxx: Efficient Transformers with Approximate Computing
Dimitrios Danopoulos, Georgios Zervakis, Dimitrios Soudris +1
Vision Transformer (ViT) models which were recently introduced by the transformer architecture have shown to be very competitive and often become a popular alternative to Convoluti…
Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devices
Kilian Pfeiffer, Mohamed Aboelenien Ahmed, Ramin Khalili +1
In recent years, Large Language Models (LLMs) through Transformer structures have dominated many machine learning tasks, especially text processing. However, these models require m…
Accelerated Training on Low-Power Edge Devices
Mohamed Aboelenien Ahmed, Kilian Pfeiffer, Heba Khdr +3
Training on edge devices poses several challenges as these devices are generally resource-constrained, especially in terms of power. State-of-the-art techniques at the device level…
Energy-Aware Heterogeneous Federated Learning via Approximate DNN Accelerators
Kilian Pfeiffer, Konstantinos Balaskas, Kostas Siozios +1
In Federated Learning (FL), devices that participate in the training usually have heterogeneous resources, i.e., energy availability. In current deployments of FL, devices that do…