286 citations · 823 across the 37 of their papers we have counts for
38 papers
URLOST: Unsupervised Representation Learning without Stationarity or Topology
Zeyu Yun, Juexiao Zhang, Yann LeCun +1
Unsupervised representation learning has seen tremendous progress. However, it is constrained by its reliance on domain specific stationarity and topology, a limitation not found i…
Efficient Decoding of Compositional Structure in Holistic Representations
Denis Kleyko, Connor Bybee, Ping-Chen Huang +4
We investigate the task of retrieving information from compositional distributed representations formed by Hyperdimensional Computing/Vector Symbolic Architectures and present nove…
Learning Internal Representations of 3D Transformations from 2D Projected Inputs
Marissa Connor, Bruno Olshausen, Christopher Rozell
When interacting in a three dimensional world, humans must estimate 3D structure from visual inputs projected down to two dimensional retinal images. It has been shown that humans…
Learning and generalization of compositional representations of visual scenes
E. Paxon Frady, Spencer Kent, Quinn Tran +3
Complex visual scenes that are composed of multiple objects, each with attributes, such as object name, location, pose, color, etc., are challenging to describe in order to train n…
Efficient Optimization with Higher-Order Ising Machines
Connor Bybee, Denis Kleyko, Dmitri E. Nikonov +3
A prominent approach to solving combinatorial optimization problems on parallel hardware is Ising machines, i.e., hardware implementations of networks of interacting binary spin va…
PIM: Video Coding using Perceptual Importance Maps
Evgenya Pergament, Pulkit Tandon, Oren Rippel +6
Human perception is at the core of lossy video compression, with numerous approaches developed for perceptual quality assessment and improvement over the past two decades. In the d…
Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution
Anthony Zador, Sean Escola, Blake Richards +24
Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in…
Minimalistic Unsupervised Learning with the Sparse Manifold Transform
Yubei Chen, Zeyu Yun, Yi Ma +2
We describe a minimalistic and interpretable method for unsupervised learning, without resorting to data augmentation, hyperparameter tuning, or other engineering designs, that ach…
Bispectral Neural Networks
Sophia Sanborn, Christian Shewmake, Bruno Olshausen +1
We present a neural network architecture, Bispectral Neural Networks (BNNs) for learning representations that are invariant to the actions of compact commutative groups on the spac…
Computing with Hypervectors for Efficient Speaker Identification
Ping-Chen Huang, Denis Kleyko, Jan M. Rabaey +2
We introduce a method to identify speakers by computing with high-dimensional random vectors. Its strengths are simplicity and speed. With only 1.02k active parameters and a 128-mi…
Neuromorphic Visual Scene Understanding with Resonator Networks
Alpha Renner, Lazar Supic, Andreea Danielescu +5
Analyzing a visual scene by inferring the configuration of a generative model is widely considered the most flexible and generalizable approach to scene understanding. Yet, one maj…
An Interactive Annotation Tool for Perceptual Video Compression
Evgenya Pergament, Pulkit Tandon, Kedar Tatwawadi +6
Human perception is at the core of lossy video compression and yet, it is challenging to collect data that is sufficiently dense to drive compression. In perceptual quality assessm…
Learning and Inference in Sparse Coding Models with Langevin Dynamics
Michael Y. -S. Fang, Mayur Mudigonda, Ryan Zarcone +2
We describe a stochastic, dynamical system capable of inference and learning in a probabilistic latent variable model. The most challenging problem in such models - sampling the po…
Integer Factorization with Compositional Distributed Representations
Denis Kleyko, Connor Bybee, Christopher J. Kymn +5
In this paper, we present an approach to integer factorization using distributed representations formed with Vector Symbolic Architectures. The approach formulates integer factoriz…
Computing on Functions Using Randomized Vector Representations
E. Paxon Frady, Denis Kleyko, Christopher J. Kymn +2
Vector space models for symbolic processing that encode symbols by random vectors have been proposed in cognitive science and connectionist communities under the names Vector Symbo…
Generalized Learning Vector Quantization for Classification in Randomized Neural Networks and Hyperdimensional Computing
Cameron Diao, Denis Kleyko, Jan M. Rabaey +1
Machine learning algorithms deployed on edge devices must meet certain resource constraints and efficiency requirements. Random Vector Functional Link (RVFL) networks are favored f…
Vector Symbolic Architectures as a Computing Framework for Emerging Hardware
Denis Kleyko, Mike Davies, E. Paxon Frady +8
This article reviews recent progress in the development of the computing framework vector symbolic architectures (VSA) (also known as hyperdimensional computing). This framework is…
Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors
Zeyu Yun, Yubei Chen, Bruno A Olshausen +1
Transformer networks have revolutionized NLP representation learning since they were introduced. Though a great effort has been made to explain the representation in transformers,…
Disentangling images with Lie group transformations and sparse coding
Ho Yin Chau, Frank Qiu, Yubei Chen +1
Discrete spatial patterns and their continuous transformations are two important regularities contained in natural signals. Lie groups and representation theory are mathematical to…
RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
Hong-Ye Hu, Dian Wu, Yi-Zhuang You +2
Flow-based generative models have become an important class of unsupervised learning approaches. In this work, we incorporate the key ideas of renormalization group (RG) and sparse…
Resonator networks for factoring distributed representations of data structures
E. Paxon Frady, Spencer Kent, Bruno A. Olshausen +1
The ability to encode and manipulate data structures with distributed neural representations could qualitatively enhance the capabilities of traditional neural networks by supporti…
Tent: Fully Test-time Adaptation by Entropy Minimization
Dequan Wang, Evan Shelhamer, Shaoteng Liu +2
A model must adapt itself to generalize to new and different data during testing. In this setting of fully test-time adaptation the model has only the test data and its own paramet…
Word Embedding Visualization Via Dictionary Learning
Juexiao Zhang, Yubei Chen, Brian Cheung +1
Co-occurrence statistics based word embedding techniques have proved to be very useful in extracting the semantic and syntactic representation of words as low dimensional continuou…
Dynamic Scale Inference by Entropy Minimization
Dequan Wang, Evan Shelhamer, Bruno Olshausen +1
Given the variety of the visual world there is not one true scale for recognition: objects may appear at drastically different sizes across the visual field. Rather than enumerate…