14.1k citations · 34.6k across the 53 of their papers we have counts for
9 papers · 1 filter
The Forward-Forward Algorithm: Some Preliminary Investigations
Geoffrey Hinton
The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investig…
Meta-Learning Fast Weight Language Models
Kevin Clark, Kelvin Guu, Ming-Wei Chang +3
Dynamic evaluation of language models (LMs) adapts model parameters at test time using gradient information from previous tokens and substantially improves LM performance. However,…
Testing GLOM's ability to infer wholes from ambiguous parts
Laura Culp, Sara Sabour, Geoffrey E. Hinton
The GLOM architecture proposed by Hinton [2021] is a recurrent neural network for parsing an image into a hierarchy of wholes and parts. When a part is ambiguous, GLOM assumes that…
Gaussian-Bernoulli RBMs Without Tears
Renjie Liao, Simon Kornblith, Mengye Ren +2
We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations. We propose a novel Gibbs-Langevin sampling alg…
A Generalist Framework for Panoptic Segmentation of Images and Videos
Ting Chen, Lala Li, Saurabh Saxena +2
Panoptic segmentation assigns semantic and instance ID labels to every pixel of an image. As permutations of instance IDs are also valid solutions, the task requires learning of hi…
Scaling Forward Gradient With Local Losses
Mengye Ren, Simon Kornblith, Renjie Liao +1
Forward gradient learning computes a noisy directional gradient and is a biologically plausible alternative to backprop for learning deep neural networks. However, the standard for…
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
Ting Chen, Ruixiang Zhang, Geoffrey Hinton
We present Bit Diffusion: a simple and generic approach for generating discrete data with continuous state and continuous time diffusion models. The main idea behind our approach i…
A Unified Sequence Interface for Vision Tasks
Ting Chen, Saurabh Saxena, Lala Li +3
While language tasks are naturally expressed in a single, unified, modeling framework, i.e., generating sequences of tokens, this has not been the case in computer vision. As a res…
Robust and Efficient Medical Imaging with Self-Supervision
Shekoofeh Azizi, Laura Culp, Jan Freyberg +31
Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optima…