296 citations · 657 across the 21 of their papers we have counts for
22 papers
MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models
Nurbek Tastan, Stefanos Laskaridis, Karthik Nandakumar +1
Mixture-of-Experts (MoE) models scale large language models efficiently by sparsely activating experts, but once an expert is selected, it is executed fully. Hence, the trade-off b…
FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment
Riccardo Zaccone, Stefanos Laskaridis, Marco Ciccone +1
The growing scale of deep neural networks, encompassing large language models (LLMs) and vision transformers (ViTs), has made training from scratch prohibitively expensive and depl…
LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning
Nurbek Tastan, Stefanos Laskaridis, Martin Takac +2
Large pre-trained models are commonly adapted to downstream tasks using parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA), which injects small trainable lo…
NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptation
Rob Romijnders, Stefanos Laskaridis, Ali Shahin Shamsabadi +1
Large Language Models (LLM) are typically trained on vast amounts of data from various sources. Even when designed modularly (e.g., Mixture-of-Experts), LLMs can leak privacy on th…
Recurrent Early Exits for Federated Learning with Heterogeneous Clients
Royson Lee, Javier Fernandez-Marques, Shell Xu Hu +6
Federated learning (FL) has enabled distributed learning of a model across multiple clients in a privacy-preserving manner. One of the main challenges of FL is to accommodate clien…
A collection of the accepted papers for the Human-Centric Representation Learning workshop at AAAI 2024
Dimitris Spathis, Aaqib Saeed, Ali Etemad +6
This non-archival index is not complete, as some accepted papers chose to opt-out of inclusion. The list of all accepted papers is available on the workshop website.
MELTing point: Mobile Evaluation of Language Transformers
Stefanos Laskaridis, Kleomenis Katevas, Lorenzo Minto +1
Transformers have revolutionized the machine learning landscape, gradually making their way into everyday tasks and equipping our computers with "sparks of intelligence". However,…
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition
Samuel Horvath, Stefanos Laskaridis, Shashank Rajput +1
Deep Neural Networks (DNNs) have been a large driver for AI breakthroughs in recent years. However, these models have been getting increasingly large as they become more accurate a…
Federated Learning for Inference at Anytime and Anywhere
Zicheng Liu, Da Li, Javier Fernandez-Marques +6
Federated learning has been predominantly concerned with collaborative training of deep networks from scratch, and especially the many challenges that arise, such as communication…
The Future of Consumer Edge-AI Computing
Stefanos Laskaridis, Stylianos I. Venieris, Alexandros Kouris +2
In the last decade, Deep Learning has rapidly infiltrated the consumer end, mainly thanks to hardware acceleration across devices. However, as we look towards the future, it is evi…
Fluid Batching: Exit-Aware Preemptive Serving of Early-Exit Neural Networks on Edge NPUs
Alexandros Kouris, Stylianos I. Venieris, Stefanos Laskaridis +1
With deep neural networks (DNNs) emerging as the backbone in a multitude of computer vision tasks, their adoption in real-world applications broadens continuously. Given the abunda…
FedorAS: Federated Architecture Search under system heterogeneity
Lukasz Dudziak, Stefanos Laskaridis, Javier Fernandez-Marques
Federated learning (FL) has recently gained considerable attention due to its ability to learn on decentralised data while preserving client privacy. However, it also poses additio…
Smart at what cost? Characterising Mobile Deep Neural Networks in the wild
Mario Almeida, Stefanos Laskaridis, Abhinav Mehrotra +3
With smartphones' omnipresence in people's pockets, Machine Learning (ML) on mobile is gaining traction as devices become more powerful. With applications ranging from visual filte…
Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
Stefanos Laskaridis, Alexandros Kouris, Nicholas D. Lane
DNNs are becoming less and less over-parametrised due to recent advances in efficient model design, through careful hand-crafted or NAS-based methods. Relying on the fact that not…
Multi-Exit Semantic Segmentation Networks
Alexandros Kouris, Stylianos I. Venieris, Stefanos Laskaridis +1
Semantic segmentation arises as the backbone of many vision systems, spanning from self-driving cars and robot navigation to augmented reality and teleconferencing. Frequently oper…
DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device
Mario Almeida, Stefanos Laskaridis, Stylianos I. Venieris +2
Recently, there has been an explosive growth of mobile and embedded applications using convolutional neural networks(CNNs). To alleviate their excessive computational demands, deve…
It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation
Ilias Leontiadis, Stefanos Laskaridis, Stylianos I. Venieris +1
On-device machine learning is becoming a reality thanks to the availability of powerful hardware and model compression techniques. Typically, these models are pretrained on large G…
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
Samuel Horvath, Stefanos Laskaridis, Mario Almeida +3
Federated Learning (FL) has been gaining significant traction across different ML tasks, ranging from vision to keyboard predictions. In large-scale deployments, client heterogenei…
SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
Stefanos Laskaridis, Stylianos I. Venieris, Mario Almeida +2
Despite the soaring use of convolutional neural networks (CNNs) in mobile applications, uniformly sustaining high-performance inference on mobile has been elusive due to the excess…
HAPI: Hardware-Aware Progressive Inference
Stefanos Laskaridis, Stylianos I. Venieris, Hyeji Kim +1
Convolutional neural networks (CNNs) have recently become the state-of-the-art in a diversity of AI tasks. Despite their popularity, CNN inference still comes at a high computation…
ShrinkML: End-to-End ASR Model Compression Using Reinforcement Learning
Łukasz Dudziak, Mohamed S. Abdelfattah, Ravichander Vipperla +2
End-to-end automatic speech recognition (ASR) models are increasingly large and complex to achieve the best possible accuracy. In this paper, we build an AutoML system that uses re…
EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices
Mario Almeida, Stefanos Laskaridis, Ilias Leontiadis +2
In recent years, advances in deep learning have resulted in unprecedented leaps in diverse tasks spanning from speech and object recognition to context awareness and health monitor…