398 citations · 1.8k across the 163 of their papers we have counts for
5 papers · 2 filters
Language Modeling Is Compression
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne +9
It has long been established that predictive models can be transformed into lossless compressors and vice versa. Incidentally, in recent years, the machine learning community has f…
Levin Tree Search with Context Models
Laurent Orseau, Marcus Hutter, Levi H. S. Lelis
Levin Tree Search (LTS) is a search algorithm that makes use of a policy (a probability distribution over actions) and comes with a theoretical guarantee on the number of expansion…
U-Clip: On-Average Unbiased Stochastic Gradient Clipping
Bryn Elesedy, Marcus Hutter
U-Clip is a simple amendment to gradient clipping that can be applied to any iterative gradient optimization algorithm. Like regular clipping, U-Clip involves using gradients that…
Evaluating Representations with Readout Model Switching
Yazhe Li, Jorg Bornschein, Marcus Hutter
Although much of the success of Deep Learning builds on learning good representations, a rigorous method to evaluate their quality is lacking. In this paper, we treat the evaluatio…
Memory-Based Meta-Learning on Non-Stationary Distributions
Tim Genewein, Grégoire Delétang, Anian Ruoss +7
Memory-based meta-learning is a technique for approximating Bayes-optimal predictors. Under fairly general conditions, minimizing sequential prediction error, measured by the log l…