Publications (34)
Deep Neural Networks Are Congestion Games: From Loss Landscape to Wardrop Equilibrium and Beyond
Nina Vesseron, Ievgen Redko, Charlotte Laclau
The theoretical analysis of deep neural networks (DNN) is arguably among the most challenging research directions in machine learning (ML) right now, as it requires from scientists…
User-friendly Foundation Model Adapters for Multivariate Time Series Classification
Vasilii Feofanov, Romain Ilbert, Malik Tiomoko +2
Foundation models, while highly effective, are often resource-intensive, requiring substantial inference time and memory. This paper addresses the challenge of making these models…
SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention
Romain Ilbert, Ambroise Odonnat, Vasilii Feofanov +4
Transformer-based architectures achieved breakthrough performance in natural language processing and computer vision, yet they remain inferior to simpler linear baselines in multiv…
Can LLMs predict the convergence of Stochastic Gradient Descent?
Oussama Zekri, Abdelhakim Benechehab, Ievgen Redko
Large-language models are notoriously famous for their impressive performance across a wide range of tasks. One surprising example of such impressive performance is a recently iden…
Vision Transformer Finetuning Benefits from Non-Smooth Components
Ambroise Odonnat, Laetitia Chapel, Romain Tavenard +1
The smoothness of the transformer architecture has been extensively studied in the context of generalization, training stability, and adversarial robustness. However, its role in t…
Learning representations that are closed-form Monge mapping optimal with application to domain adaptation
Oliver Struckmeier, Ievgen Redko, Anton Mallasto +3
Optimal transport (OT) is a powerful geometric tool used to compare and align probability measures following the least effort principle. Despite its widespread use in machine learn…
Leveraging Ensemble Diversity for Robust Self-Training in the Presence of Sample Selection Bias
Ambroise Odonnat, Vasilii Feofanov, Ievgen Redko
Self-training is a well-known approach for semi-supervised learning. It consists of iteratively assigning pseudo-labels to unlabeled data for which the model is confident and treat…
CO-Optimal Transport
Ievgen Redko, Titouan Vayer, Rémi Flamary +1
Optimal transport (OT) is a powerful geometric and probabilistic tool for finding correspondences and measuring similarity between two distributions. Yet, its original formulation…
Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
Romain Ilbert, Malik Tiomoko, Cosme Louart +4
In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensiona…
Optimal Self-Consistency for Efficient Reasoning with Large Language Models
Austin Feng, Marius Alonso, Ambroise Odonnat +2
Self-consistency (SC) is a widely used test-time inference technique for improving performance in chain-of-thought reasoning. It consists of generating multiple responses, or ``sam…
Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift
Renchunzi Xie, Ambroise Odonnat, Vasilii Feofanov +3
Estimating the test performance of a model, possibly under distribution shift, without having access to the ground-truth labels is a challenging, yet very important problem for the…
A survey on domain adaptation theory: learning bounds and theoretical guarantees
Ievgen Redko, Emilie Morvant, Amaury Habrard +2
All famous machine learning algorithms that comprise both supervised and semi-supervised learning work well only under a common assumption: the training and test data follow the sa…
Rank-one partitioning: formalization, illustrative examples, and a new cluster enhancing strategy
Charlotte Laclau, Franck Iutzeler, Ievgen Redko
In this paper, we introduce and formalize a rank-one partitioning learning paradigm that unifies partitioning methods that proceed by summarizing a data set using a single vector t…
Unbalanced CO-Optimal Transport
Quang Huy Tran, Hicham Janati, Nicolas Courty +4
Optimal transport (OT) compares probability distributions by computing a meaningful alignment between their samples. CO-optimal transport (COOT) takes this comparison further by in…
Layer by layer, module by module: Choose both for optimal OOD probing of ViT
Ambroise Odonnat, Vasilii Feofanov, Laetitia Chapel +2
Recent studies have observed that intermediate layers of foundation models often yield more discriminative representations than the final layer. While initially attributed to autor…
From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport
Quentin Bouniot, Ievgen Redko, Anton Mallasto +6
In the last decade, we have witnessed the introduction of several novel deep neural network (DNN) architectures exhibiting ever-increasing performance across diverse tasks. Explain…
Co-clustering through Optimal Transport
Charlotte Laclau, Ievgen Redko, Basarab Matei +2
In this paper, we present a novel method for co-clustering, an unsupervised learning approach that aims at discovering homogeneous groups of data instances and features by grouping…
Leveraging Generic Time Series Foundation Models for EEG Classification
Théo Gnassounou, Yessin Moakher, Shifeng Xie +2
Foundation models for time series are emerging as powerful general-purpose backbones, yet their potential for domain-specific biomedical signals such as electroencephalography (EEG…
Feature Selection for Unsupervised Domain Adaptation using Optimal Transport
Léo Gautheron, Ievgen Redko, Carole Lartizien
In this paper, we propose a new feature selection method for unsupervised domain adaptation based on the emerging optimal transportation theory. We build upon a recent theoretical…
Kernel Alignment for Unsupervised Transfer Learning
Ievgen Redko, Younès Bennani
The ability of a human being to extrapolate previously gained knowledge to other domains inspired a new family of methods in machine learning called transfer learning. Transfer lea…
CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
Shifeng Xie, Vasilii Feofanov, Ambroise Odonnat +7
Time series foundation models (TSFMs) have recently gained significant attention due to their strong zero-shot capabilities and widespread real-world applications. Such models typi…
Optimal Transport for Multi-source Domain Adaptation under Target Shift
Ievgen Redko, Nicolas Courty, Rémi Flamary +1
In this paper, we propose to tackle the problem of reducing discrepancies between multiple domains referred to as multi-source domain adaptation and consider it under the target sh…
Revisiting invariances and introducing priors in Gromov-Wasserstein distances
Pinar Demetci, Quang Huy Tran, Ievgen Redko +1
Gromov-Wasserstein distance has found many applications in machine learning due to its ability to compare measures across metric spaces and its invariance to isometric transformati…
All of the Fairness for Edge Prediction with Optimal Transport
Charlotte Laclau, Ievgen Redko, Manvi Choudhary +1
Machine learning and data mining algorithms have been increasingly used recently to support decision-making systems in many areas of high societal importance such as healthcare, ed…
Large Language Models as Markov Chains
Oussama Zekri, Ambroise Odonnat, Abdelhakim Benechehab +3
Large language models (LLMs) are remarkably efficient across a wide range of natural language processing tasks and well beyond them. However, a comprehensive theoretical analysis o…
Factored couplings in multi-marginal optimal transport via difference of convex programming
Quang Huy Tran, Hicham Janati, Ievgen Redko +2
Optimal transport (OT) theory underlies many emerging machine learning (ML) methods nowadays solving a wide range of tasks such as generative modeling, transfer learning and inform…
Zero-shot Model-based Reinforcement Learning using Large Language Models
Abdelhakim Benechehab, Youssef Attia El Hili, Ambroise Odonnat +6
The emerging zero-shot capabilities of Large Language Models (LLMs) have led to their applications in areas extending well beyond natural language processing tasks. In reinforcemen…
Meta Optimal Transport
Brandon Amos, Samuel Cohen, Giulia Luise +1
We study the use of amortized optimization to predict optimal transport (OT) maps from the input measures, which we call Meta OT. This helps repeatedly solve similar OT problems be…
Cross-lingual Document Retrieval using Regularized Wasserstein Distance
Georgios Balikas, Charlotte Laclau, Ievgen Redko +1
Many information retrieval algorithms rely on the notion of a good distance that allows to efficiently compare objects of different nature. Recently, a new promising metric called…
Mantis: Lightweight Foundation Model for Time Series Classification
Vasilii Feofanov, Songkang Wen, Shifeng Xie +10
While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly f…
Improving Few-Shot Learning through Multi-task Representation Learning Theory
Quentin Bouniot, Ievgen Redko, Romaric Audigier +2
In this paper, we consider the framework of multi-task representation (MTR) learning where the goal is to use source tasks to learn a representation that reduces the sample complex…
Theoretical Analysis of Domain Adaptation with Optimal Transport
Ievgen Redko, Amaury Habrard, Marc Sebban
Domain adaptation (DA) is an important and emerging field of machine learning that tackles the problem occurring when the distributions of training (source domain) and test (target…
MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategies
Vasilii Feofanov, Songkang Wen, Jianfeng Zhang +2
Developing foundation models for time series classification is of high practical relevance, as such models can serve as universal feature extractors for diverse downstream tasks. A…
Time Series Representations for Classification Lie Hidden in Pretrained Vision Transformers
Simon Roschmann, Quentin Bouniot, Vasilii Feofanov +2
Time series classification is a fundamental task in healthcare and industry, yet the development of time series foundation models (TSFMs) remains limited by the scarcity of publicl…