61 citations · 150 across the 17 of their papers we have counts for
10 papers · 1 filter
Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Training
Jonathan Lys, Vincent Gripon, Bastien Pasdeloup +4
Deep Learning architectures, and in particular Transformers, are conventionally viewed as a composition of layers. These layers are actually often obtained as the sum of two contri…
GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameters
Anand Choudhary, Yasser Sulaıman, Lukas Mauch +3
Sparse fine-tuning techniques adapt LLMs to downstream tasks by only tuning a sparse subset of model parameters. However, the effectiveness of sparse adaptation depends on optimall…
DBsurf: A Discrepancy Based Method for Discrete Stochastic Gradient Estimation
Pau Mulet Arabi, Alec Flowers, Lukas Mauch +1
Computing gradients of an expectation with respect to the distributional parameters of a discrete distribution is a problem arising in many fields of science and engineering. Typic…
Efficient Training of Deep Equilibrium Models
Bac Nguyen, Lukas Mauch
Deep equilibrium models (DEQs) have proven to be very powerful for learning data representations. The idea is to replace traditional (explicit) feedforward neural networks with an…
A Statistical Model for Predicting Generalization in Few-Shot Classification
Yassir Bendou, Vincent Gripon, Bastien Pasdeloup +5
The estimation of the generalization error of classifiers often relies on a validation set. Such a set is hardly available in few-shot learning scenarios, a highly disregarded shor…
Efficient Sampling for Predictor-Based Neural Architecture Search
Lukas Mauch, Stephen Tiedemann, Javier Alonso Garcia +4
Recently, predictor-based algorithms emerged as a promising approach for neural architecture search (NAS). For NAS, we typically have to calculate the validation accuracy of a larg…
Iteratively Training Look-Up Tables for Network Quantization
Fabien Cardinaux, Stefan Uhlich, Kazuki Yoshiyama +5
Operating deep neural networks (DNNs) on devices with limited resources requires the reduction of their memory as well as computational footprint. Popular reduction methods are net…
Mixed Precision DNNs: All you need is a good parametrization
Stefan Uhlich, Lukas Mauch, Fabien Cardinaux +5
Efficient deep neural network (DNN) inference on mobile or embedded devices typically involves quantization of the network parameters and activations. In particular, mixed precisio…
Deep Neural Network inference with reduced word length
Lukas Mauch, Bin Yang
Deep neural networks (DNN) are powerful models for many pattern recognition tasks, yet their high computational complexity and memory requirement limit them to applications on high…
Neural Network Ensembles to Real-time Identification of Plug-level Appliance Measurements
Karim Said Barsim, Lukas Mauch, Bin Yang
The problem of identifying end-use electrical appliances from their individual consumption profiles, known as the appliance identification problem, is a primary stage in both Non-I…