115 citations · 342 across the 14 of their papers we have counts for
15 papers
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Zechun Liu, Barlas Oguz, Changsheng Zhao +6
Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits. We find that these methods break d…
Evaluating Privacy Leakage in Split Learning
Xinchi Qiu, Ilias Leontiadis, Luca Melis +2
Privacy-Preserving machine learning (PPML) can help us train and deploy models that utilize private information. In particular, on-device machine learning allows us to avoid sharin…
Green Federated Learning
Ashkan Yousefpour, Shen Guo, Ashish Shenoy +7
The rapid progress of AI is fueled by increasingly large and computationally intensive machine learning models and datasets. As a consequence, the amount of compute used in trainin…
Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design
Chuan Guo, Kamalika Chaudhuri, Pierre Stock +1
In private federated learning (FL), a server aggregates differentially private updates from a large number of clients in order to train a machine learning model. The main challenge…
TAN Without a Burn: Scaling Laws of DP-SGD
Tom Sander, Pierre Stock, Alexandre Sablayrolles
Differentially Private methods for training Deep Neural Networks (DNNs) have progressed recently, in particular with the use of massive batches and aggregated data augmentations fo…
CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning
Samuel Maddock, Alexandre Sablayrolles, Pierre Stock
Federated Learning (FL) is a setting for training machine learning models in distributed environments where the clients do not share their raw data but instead send model updates t…
Reconciling Security and Communication Efficiency in Federated Learning
Karthik Prasad, Sayan Ghosh, Graham Cormode +3
Cross-device Federated Learning is an increasingly popular machine learning setting to train a model by leveraging a large population of client devices with high privacy and securi…
Defending against Reconstruction Attacks with Rényi Differential Privacy
Pierre Stock, Igor Shilov, Ilya Mironov +1
Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. It has been recently shown that simple heuristics can…
An Embedding of ReLU Networks and an Analysis of their Identifiability
Pierre Stock, Rémi Gribonval
Neural networks with the Rectified Linear Unit (ReLU) nonlinearity are described by a vector of parameters , and realized as a piecewise linear continuous function $R_θ: x \in \…
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
Ben Graham, Alaaeldin El-Nouby, Hugo Touvron +4
We design a family of image classification architectures that optimize the trade-off between accuracy and efficiency in a high-speed regime. Our work exploits recent findings in at…
Low Bandwidth Video-Chat Compression using Deep Generative Models
Maxime Oquab, Pierre Stock, Oran Gafni +9
To unlock video chat for hundreds of millions of people hindered by poor connectivity or unaffordable data costs, we propose to authentically reconstruct faces on the receiver's de…
Training with Quantization Noise for Extreme Model Compression
Angela Fan, Pierre Stock, Benjamin Graham +4
We tackle the problem of producing compact models, maximizing their accuracy for a given model size. A standard solution is to train networks with Quantization Aware Training, wher…
And the Bit Goes Down: Revisiting the Quantization of Neural Networks
Pierre Stock, Armand Joulin, Rémi Gribonval +2
In this paper, we address the problem of reducing the memory footprint of convolutional network architectures. We introduce a vector quantization method that aims at preserving the…
Equi-normalization of Neural Networks
Pierre Stock, Benjamin Graham, Rémi Gribonval +1
Modern neural networks are over-parametrized. In particular, each rectified linear hidden unit can be modified by a multiplicative factor by adjusting input and output weights, wit…
ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
Pierre Stock, Moustapha Cisse
ConvNets and Imagenet have driven the recent success of deep learning for image classification. However, the marked slowdown in performance improvement combined with the lack of ro…