181 citations · 702 across the 49 of their papers we have counts for
17 papers · 1 filter
DataComp-LM: In search of the next generation of training sets for language models
Jeffrey Li, Alex Fang, Georgios Smyrnis +56
We introduce DataComp for Language Models (DCLM), a testbed for controlled dataset experiments with the goal of improving language models. As part of DCLM, we provide a standardize…
Deep Learning for Accelerated and Robust MRI Reconstruction: a Review
Reinhard Heckel, Mathews Jacob, Akshay Chaudhari +2
Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides…
Approximating Positive Homogeneous Functions with Scale Invariant Neural Networks
Stefan Bamberger, Reinhard Heckel, Felix Krahmer
We investigate to what extent it is possible to solve linear inverse problems with networks. Due to the scaling invariance arising from the linearity, an optimal reconstruct…
Learning Provably Robust Estimators for Inverse Problems via Jittering
Anselm Krainovic, Mahdi Soltanolkotabi, Reinhard Heckel
Deep neural networks provide excellent performance for inverse problems such as denoising. However, neural networks can be sensitive to adversarial or worst-case perturbations. Thi…
Monotonic Risk Relationships under Distribution Shifts for Regularized Risk Minimization
Daniel LeJeune, Jiayu Liu, Reinhard Heckel
Machine learning systems are often applied to data that is drawn from a different distribution than the training distribution. Recent work has shown that for a variety of classific…
Test-time Recalibration of Conformal Predictors Under Distribution Shift Based on Unlabeled Examples
Fatih Furkan Yilmaz, Reinhard Heckel
Modern image classifiers are very accurate, but the predictions come without uncertainty estimates. Conformal predictors provide uncertainty estimates by computing a set of classes…
Regularization-wise double descent: Why it occurs and how to eliminate it
Fatih Furkan Yilmaz, Reinhard Heckel
The risk of overparameterized models, in particular deep neural networks, is often double-descent shaped as a function of the model size. Recently, it was shown that the risk as a…
Early Stopping in Deep Networks: Double Descent and How to Eliminate it
Reinhard Heckel, Fatih Furkan Yilmaz
Over-parameterized models, such as large deep networks, often exhibit a double descent phenomenon, whereas a function of model size, error first decreases, increases, and decreases…
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation
Reinhard Heckel, Mahdi Soltanolkotabi
Un-trained convolutional neural networks have emerged as highly successful tools for image recovery and restoration. They are capable of solving standard inverse problems such as d…
Reducing the Representation Error of GAN Image Priors Using the Deep Decoder
Mara Daniels, Paul Hand, Reinhard Heckel
Generative models, such as GANs, learn an explicit low-dimensional representation of a particular class of images, and so they may be used as natural image priors for solving inver…
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators
Reinhard Heckel, Mahdi Soltanolkotabi
Convolutional Neural Networks (CNNs) have emerged as highly successful tools for image generation, recovery, and restoration. A major contributing factor to this success is that co…
Image recognition from raw labels collected without annotators
Fatih Furkan Yilmaz, Reinhard Heckel
Image classification problems are typically addressed by first collecting examples with candidate labels, second cleaning the candidate labels manually, and third training a deep n…
Channel Normalization in Convolutional Neural Network avoids Vanishing Gradients
Zhenwei Dai, Reinhard Heckel
Normalization layers are widely used in deep neural networks to stabilize training. In this paper, we consider the training of convolutional neural networks with gradient descent o…
Regularizing linear inverse problems with convolutional neural networks
Reinhard Heckel
Deep convolutional neural networks trained on large datsets have emerged as an intriguing alternative for compressing images and solving inverse problems such as denoising and comp…
Approximate Ranking from Pairwise Comparisons
Reinhard Heckel, Max Simchowitz, Kannan Ramchandran +1
A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes accor…
The Sample Complexity of Online One-Class Collaborative Filtering
Reinhard Heckel, Kannan Ramchandran
We consider the online one-class collaborative filtering (CF) problem that consists of recommending items to users over time in an online fashion based on positive ratings only. Th…
Active Ranking from Pairwise Comparisons and when Parametric Assumptions Don't Help
Reinhard Heckel, Nihar B. Shah, Kannan Ramchandran +1
We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly c…