29 citations · 106 across the 17 of their papers we have counts for
8 papers · 1 filter
The Hessian perspective into the Nature of Convolutional Neural Networks
Sidak Pal Singh, Thomas Hofmann, Bernhard Schölkopf
While Convolutional Neural Networks (CNNs) have long been investigated and applied, as well as theorized, we aim to provide a slightly different perspective into their nature -- th…
Vanishing Curvature and the Power of Adaptive Methods in Randomly Initialized Deep Networks
Antonio Orvieto, Jonas Kohler, Dario Pavllo +2
This paper revisits the so-called vanishing gradient phenomenon, which commonly occurs in deep randomly initialized neural networks. Leveraging an in-depth analysis of neural chain…
Generative Minimization Networks: Training GANs Without Competition
Paulina Grnarova, Yannic Kilcher, Kfir Y. Levy +2
Many applications in machine learning can be framed as minimization problems and solved efficiently using gradient-based techniques. However, recent applications of generative mode…
BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Florian Schmidt, Thomas Hofmann
Measuring the quality of a generated sequence against a set of references is a central problem in many learning frameworks, be it to compute a score, to assign a reward, or to perf…
LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games
Leonard Adolphs, Thomas Hofmann
While Reinforcement Learning (RL) approaches lead to significant achievements in a variety of areas in recent history, natural language tasks remained mostly unaffected, due to the…
Autoregressive Text Generation Beyond Feedback Loops
Florian Schmidt, Stephan Mandt, Thomas Hofmann
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However…
Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy
Aryan Mokhtari, Alejandro Ribeiro
We consider empirical risk minimization for large-scale datasets. We introduce Ada Newton as an adaptive algorithm that uses Newton's method with adaptive sample sizes. The main id…
Variance Reduced Stochastic Gradient Descent with Neighbors
Thomas Hofmann, Aurelien Lucchi, Simon Lacoste-Julien +1
Stochastic Gradient Descent (SGD) is a workhorse in machine learning, yet its slow convergence can be a computational bottleneck. Variance reduction techniques such as SAG, SVRG an…