23 citations · 81 across the 41 of their papers we have counts for
16 papers · 1 filter
Robustness Tokens: Towards Adversarial Robustness of Transformers
Brian Pulfer, Yury Belousov, Slava Voloshynovskiy
Recently, large pre-trained foundation models have become widely adopted by machine learning practitioners for a multitude of tasks. Given that such models are publicly available,…
TwinTURBO: Semi-Supervised Fine-Tuning of Foundation Models via Mutual Information Decompositions for Downstream Task and Latent Spaces
Guillaume Quétant, Pavlo Molchanov, Slava Voloshynovskiy
We present a semi-supervised fine-tuning framework for foundation models that utilises mutual information decomposition to address the challenges of training for a limited amount o…
TURBO: The Swiss Knife of Auto-Encoders
Guillaume Quétant, Yury Belousov, Vitaliy Kinakh +1
We present a novel information-theoretic framework, termed as TURBO, designed to systematically analyse and generalise auto-encoding methods. We start by examining the principles o…
Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility
Behrooz Razeghi, Flavio P. Calmon, Deniz Gunduz +1
Bottleneck problems are an important class of optimization problems that have recently gained increasing attention in the domain of machine learning and information theory. They ar…
Turbo-Sim: a generalised generative model with a physical latent space
Guillaume Quétant, Mariia Drozdova, Vitaliy Kinakh +2
We present Turbo-Sim, a generalised autoencoder framework derived from principles of information theory that can be used as a generative model. By maximising the mutual information…
Generation of data on discontinuous manifolds via continuous stochastic non-invertible networks
Mariia Drozdova, Vitaliy Kinakh, Guillaume Quétant +2
The generation of discontinuous distributions is a difficult task for most known frameworks such as generative autoencoders and generative adversarial networks. Generative non-inve…
Funnels: Exact maximum likelihood with dimensionality reduction
Samuel Klein, John A. Raine, Sebastian Pina-Otey +2
Normalizing flows are diffeomorphic, typically dimension-preserving, models trained using the likelihood of the model. We use the SurVAE framework to construct dimension reducing s…
Variational Leakage: The Role of Information Complexity in Privacy Leakage
Amir Ahooye Atashin, Behrooz Razeghi, Deniz Gündüz +1
We study the role of information complexity in privacy leakage about an attribute of an adversary's interest, which is not known a priori to the system designer. Considering the su…
-VAE: Autoregressive parametrization of the VAE encoder
Sohrab Ferdowsi, Maurits Diephuis, Shideh Rezaeifar +1
We make a minimal, but very effective alteration to the VAE model. This is about a drop-in replacement for the (sample-dependent) approximate posterior to change it from the standa…
Robustification of deep net classifiers by key based diversified aggregation with pre-filtering
Olga Taran, Shideh Rezaeifar, Taras Holotyak +1
In this paper, we address a problem of machine learning system vulnerability to adversarial attacks. We propose and investigate a Key based Diversified Aggregation (KDA) mechanism…
Reconstruction of Privacy-Sensitive Data from Protected Templates
Shideh Rezaeifar, Behrooz Razeghi, Olga Taran +2
In this paper, we address the problem of data reconstruction from privacy-protected templates, based on recent concept of sparse ternary coding with ambiguization (STCA). The STCA…
Defending against adversarial attacks by randomized diversification
Olga Taran, Shideh Rezaeifar, Taras Holotyak +1
The vulnerability of machine learning systems to adversarial attacks questions their usage in many applications. In this paper, we propose a randomized diversification as a defense…
Network Parameter Learning Using Nonlinear Transforms, Local Representation Goals and Local Propagation Constraints
Dimche Kostadinov, Behrooz Razdehi, Slava Voloshynovskiy
In this paper, we introduce a novel concept for learning of the parameters in a neural network. Our idea is grounded on modeling a learning problem that addresses a trade-off betwe…
Clustering with Jointly Learned Nonlinear Transforms Over Discriminating Min-Max Similarity/Dissimilarity Assignment
Dimche Kostadinov, Behrooz Razeghi, Taras Holotyak +1
This paper presents a novel clustering concept that is based on jointly learned nonlinear transforms (NTs) with priors on the information loss and the discrimination. We introduce…
Network Learning with Local Propagation
Dimche Kostadinov, Behrooz Razeghi, Sohrab Ferdowsi +1
This paper presents a locally decoupled network parameter learning with local propagation. Three elements are taken into account: (i) sets of nonlinear transforms that describe the…
Regularized Residual Quantization: a multi-layer sparse dictionary learning approach
Sohrab Ferdowsi, Slava Voloshynovskiy, Dimche Kostadinov
The Residual Quantization (RQ) framework is revisited where the quantization distortion is being successively reduced in multi-layers. Inspired by the reverse-water-filling paradig…