9 papers
An Analysis of Self-supervised Pre-training with Dependent Samples
Maximilian Fleissner, Debarghya Ghoshdastidar, Samory Kpotufe
Self-supervised learning relies on so-called data augmentations of unlabeled datapoints --- for example, masking random pixels in an image --- that should leave the…
Theoretical Foundations of Representation Learning using Unlabeled Data: Statistics and Optimization
Pascal Esser, Maximilian Fleissner, Debarghya Ghoshdastidar
Representation learning from unlabeled data has been extensively studied in statistics, data science and signal processing with a rich literature on techniques for dimension reduct…
Impact of Bottleneck Layers and Skip Connections on the Generalization of Linear Denoising Autoencoders
Jonghyun Ham, Maximilian Fleissner, Debarghya Ghoshdastidar
Modern deep neural networks exhibit strong generalization even in highly overparameterized regimes. Significant progress has been made to understand this phenomenon in the context…
A Probabilistic Model for Non-Contrastive Learning
Maximilian Fleissner, Pascal Esser, Debarghya Ghoshdastidar
Self-supervised learning (SSL) aims to find meaningful representations from unlabeled data by encoding semantic similarities through data augmentations. Despite its current popular…
Infinite Width Limits of Self Supervised Neural Networks
Maximilian Fleissner, Gautham Govind Anil, Debarghya Ghoshdastidar
The NTK is a widely used tool in the theoretical analysis of deep learning, allowing us to look at supervised deep neural networks through the lenses of kernel regression. Recently…
A Theoretical Characterization of Optimal Data Augmentations in Self-Supervised Learning
Shlomo Libo Feigin, Maximilian Fleissner, Debarghya Ghoshdastidar
Data augmentations play an important role in the recent success of self-supervised learning (SSL). While augmentations are commonly understood to encode invariances between differe…
Explainable Clustering of Mixture Models
Maximilian Fleissner, Maedeh Zarvandi, Debarghya Ghoshdastidar
The explainable clustering problem was first posed by Moshkovitz et al. (ICML 2020) and studies how well an axis-aligned decision tree with leaves can approximate a given clust…
Explaining Kernel Clustering via Decision Trees
Maximilian Fleissner, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Despite the growing popularity of explainable and interpretable machine learning, there is still surprisingly limited work on inherently interpretable clustering methods. Recently,…
Non-Parametric Representation Learning with Kernels
Pascal Esser, Maximilian Fleissner, Debarghya Ghoshdastidar
Unsupervised and self-supervised representation learning has become popular in recent years for learning useful features from unlabelled data. Representation learning has been most…