output
19922026
Showing 2021 · cs.LGShow all

66 papers · 2 filters

cs.LG2021

Learning from Mistakes based on Class Weighting with Application to Neural Architecture Search

Jay Gala, Pengtao Xie

Learning from mistakes is an effective learning approach widely used in human learning, where a learner pays greater focus on mistakes to circumvent them in the future to improve t…

cs.LG2021

Improving Differentiable Architecture Search with a Generative Model

Ruisi Zhang, Youwei Liang, Sai Ashish Somayajula +1

In differentiable neural architecture search (NAS) algorithms like DARTS, the training set used to update model weight and the validation set used to update model architectures are…

cs.LG2021

Turing-Universal Learners with Optimal Scaling Laws

Preetum Nakkiran

For a given distribution, learning algorithm, and performance metric, the rate of convergence (or data-scaling law) is the asymptotic behavior of the algorithm's test performance a…

cs.LG2021★ 4 cited

Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds

Yujia Huang, Huan Zhang, Yuanyuan Shi +2

Certified robustness is a desirable property for deep neural networks in safety-critical applications, and popular training algorithms can certify robustness of a neural network by…

cs.LG2021★ 1 cited

Improving Experience Replay with Successor Representation

Yizhi Yuan, Marcelo G Mattar

Prioritized experience replay is a reinforcement learning technique whereby agents speed up learning by replaying useful past experiences. This usefulness is quantified as the expe…

cs.LG2021★ 1 cited

ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees

Kuan-Lin Chen, Ching-Hua Lee, Harinath Garudadri +1

Models recently used in the literature proving residual networks (ResNets) are better than linear predictors are actually different from standard ResNets that have been widely used…

cs.LG2021

Realizable Learning is All You Need

Max Hopkins, Daniel M. Kane, Shachar Lovett +1

The equivalence of realizable and agnostic learnability is a fundamental phenomenon in learning theory. With variants ranging from classical settings like PAC learning and regressi…

cs.LG2021★ 7 cited

Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations

Xinyu Zhang, Ian Colbert, Ken Kreutz-Delgado +1

Quantization and pruning are core techniques used to reduce the inference costs of deep neural networks. State-of-the-art quantization techniques are currently applied to both the…

cs.LG2021★ 20 cited

Automatic Symmetry Discovery with Lie Algebra Convolutional Network

Nima Dehmamy, Robin Walters, Yanchen Liu +2

Existing equivariant neural networks require prior knowledge of the symmetry group and discretization for continuous groups. We propose to work with Lie algebras (infinitesimal gen…

cs.LG2021★ 77 cited

Applications and Techniques for Fast Machine Learning in Science

Allison McCarn Deiana, Nhan Tran, Joshua Agar +84

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time…

cs.LG2021★ 563 cited

Multitask Prompted Training Enables Zero-Shot Task Generalization

Victor Sanh, Albert Webson, Colin Raffel +38

Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a…

cs.LG2021★ 13 cited

Equivariant Subgraph Aggregation Networks

Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim +5

Message-passing neural networks (MPNNs) are the leading architecture for deep learning on graph-structured data, in large part due to their simplicity and scalability. Unfortunatel…

cs.LG2021★ 3 cited

Style Equalization: Unsupervised Learning of Controllable Generative Sequence Models

Jen-Hao Rick Chang, Ashish Shrivastava, Hema Swetha Koppula +2

Controllable generative sequence models with the capability to extract and replicate the style of specific examples enable many applications, including narrating audiobooks in diff…

cs.LG2021★ 55 cited

Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation

Stephen G. Penny, Timothy A. Smith, Tse-Chun Chen +4

Data assimilation (DA) is integrated with machine learning in order to perform entirely data-driven online state estimation. To achieve this, recurrent neural networks (RNNs) are i…

cs.LG2021

Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness

Fatemehsadat Mireshghallah, Taylor Berg-Kirkpatrick

Text style can reveal sensitive attributes of the author (e.g. race or age) to the reader, which can, in turn, lead to privacy violations and bias in both human and algorithmic dec…

cs.LG2021★ 2 cited

Outlier-Robust Sparse Estimation via Non-Convex Optimization

Yu Cheng, Ilias Diakonikolas, Rong Ge +3

We explore the connection between outlier-robust high-dimensional statistics and non-convex optimization in the presence of sparsity constraints, with a focus on the fundamental ta…

cs.LG2021

Efficient Hyperparameter Optimization for Differentially Private Deep Learning

Aman Priyanshu, Rakshit Naidu, Fatemehsadat Mireshghallah +1

Tuning the hyperparameters in the differentially private stochastic gradient descent (DPSGD) is a fundamental challenge. Unlike the typical SGD, private datasets cannot be used man…

cs.LG2021★ 47 cited

Deep Learning Enhanced Dynamic Mode Decomposition

Daniel J. Alford-Lago, Christopher W. Curtis, Alexander T. Ihler +1

Koopman operator theory shows how nonlinear dynamical systems can be represented as an infinite-dimensional, linear operator acting on a Hilbert space of observables of the system.…

cs.LG2021★ 22 cited

Simple, Fast, and Flexible Framework for Matrix Completion with Infinite Width Neural Networks

Adityanarayanan Radhakrishnan, George Stefanakis, Mikhail Belkin +1

Matrix completion problems arise in many applications including recommendation systems, computer vision, and genomics. Increasingly larger neural networks have been successful in m…

cs.LG2021

Bridge Networks: Relating Inputs through Vector-Symbolic Manipulations

Wilkie Olin-Ammentorp, Maxim Bazhenov

Despite rapid progress, current deep learning methods face a number of critical challenges. These include high energy consumption, catastrophic forgetting, dependance on global los…

cs.LG2021

Nonlinear Least Squares for Large-Scale Machine Learning using Stochastic Jacobian Estimates

Johannes J. Brust

For large nonlinear least squares loss functions in machine learning we exploit the property that the number of model parameters typically exceeds the data in one batch. This impli…

cs.LG2021★ 1 cited

Forster Decomposition and Learning Halfspaces with Noise

Ilias Diakonikolas, Daniel M. Kane, Christos Tzamos

A Forster transform is an operation that turns a distribution into one with good anti-concentration properties. While a Forster transform does not always exist, we show that any di…

cs.LG2021★ 74 cited

"Garbage In, Garbage Out" Revisited: What Do Machine Learning Application Papers Report About Human-Labeled Training Data?

R. Stuart Geiger, Dominique Cope, Jamie Ip +4

Supervised machine learning, in which models are automatically derived from labeled training data, is only as good as the quality of that data. This study builds on prior work that…

cs.LG2021★ 1 cited

Provably Efficient Multi-Task Reinforcement Learning with Model Transfer

Chicheng Zhang, Zhi Wang

We study multi-task reinforcement learning (RL) in tabular episodic Markov decision processes (MDPs). We formulate a heterogeneous multi-player RL problem, in which a group of play…