37 citations · 121 across the 28 of their papers we have counts for
16 papers · 1 filter
Spectral Signatures of Memorization Basins in Diffusion Trajectories
Achref Jaziri, Martin Mundt, Ahmed Imtiaz Humayun +1
Diffusion models are known to reproduce individual examples from their training data as a form of memorization with implications for generalization, privacy, and downstream use. Re…
The Geometric Structure of Models Learning Sparse Data
Thomas Walker, T. Mitchell Roddenberry, Ahmed Imtiaz Humayun +2
The manifold hypothesis (MH) is often used to explain how machine learning can overcome the curse of dimensionality. However, the MH is only applicable in regimes where the trainin…
The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts
Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero +1
The Linear Representation Hypothesis (LRH) identifies features of a trained deep network (DN) as linear directions in the activation spaces, i.e., output spaces of intermediate lay…
We Should Distinguish Unlearning From Untraining
Eleni Triantafillou, Ahmed Imtiaz Humayun, Monica Ribero +3
There has been a recent surge in interest about the question of how we can ``delete'' specific data points or behaviours from a trained model, a goal referred to as ``machine unlea…
GrokAlign: Geometric Characterisation and Acceleration of Grokking
Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero +1
A key challenge for the machine learning community is to understand and accelerate the training dynamics of deep networks that lead to delayed generalisation and emergent robustnes…
The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws
Tian Jin, Ahmed Imtiaz Humayun, Utku Evci +4
Pruning eliminates unnecessary parameters in neural networks; it offers a promising solution to the growing computational demands of large language models (LLMs). While many focus…
Self-Improving Diffusion Models with Synthetic Data
Sina Alemohammad, Ahmed Imtiaz Humayun, Shruti Agarwal +2
The artificial intelligence (AI) world is running out of real data for training increasingly large generative models, resulting in accelerating pressure to train on synthetic data.…
What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
Ahmed Imtiaz Humayun, Ibtihel Amara, Cristina Vasconcelos +7
Deep Generative Models are frequently used to learn continuous representations of complex data distributions using a finite number of samples. For any generative model, including p…
On the Geometry of Deep Learning
Randall Balestriero, Ahmed Imtiaz Humayun, Richard Baraniuk
In this paper, we overview one promising avenue of progress at the mathematical foundation of deep learning: the connection between deep networks and function approximation by affi…
Mitigating over-exploration in latent space optimization using LES
Omer Ronen, Ahmed Imtiaz Humayun, Richard Baraniuk +2
We develop Latent Exploration Score (LES) to mitigate over-exploration in Latent Space Optimization (LSO), a popular method for solving black-box discrete optimization problems. LS…
Deep Networks Always Grok and Here is Why
Ahmed Imtiaz Humayun, Randall Balestriero, Richard Baraniuk
Grokking, or delayed generalization, is a phenomenon where generalization in a deep neural network (DNN) occurs long after achieving near zero training error. Previous studies have…
Training Dynamics of Deep Network Linear Regions
Ahmed Imtiaz Humayun, Randall Balestriero, Richard Baraniuk
The study of Deep Network (DN) training dynamics has largely focused on the evolution of the loss function, evaluated on or around train and test set data points. In fact, many DN…
Self-Consuming Generative Models Go MAD
Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi +5
Seismic advances in generative AI algorithms for imagery, text, and other data types has led to the temptation to use synthetic data to train next-generation models. Repeating this…
No More Than 6ft Apart: Robust K-Means via Radius Upper Bounds
Ahmed Imtiaz Humayun, Randall Balestriero, Anastasios Kyrillidis +1
Centroid based clustering methods such as k-means, k-medoids and k-centers are heavily applied as a go-to tool in exploratory data analysis. In many cases, those methods are used t…
MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining
Ahmed Imtiaz Humayun, Randall Balestriero, Richard Baraniuk
Deep Generative Networks (DGNs) are extensively employed in Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and their variants to approximate the data mani…
End-to-end Sleep Staging with Raw Single Channel EEG using Deep Residual ConvNets
Ahmed Imtiaz Humayun, Asif Shahriyar Sushmit, Taufiq Hasan +1
Humans approximately spend a third of their life sleeping, which makes monitoring sleep an integral part of well-being. In this paper, a 34-layer deep residual ConvNet architecture…