8 citations · 13 across the 9 of their papers we have counts for
6 papers · 1 filter
Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings
Anand Gopalakrishnan, Robert Csordás, Jürgen Schmidhuber +1
The attention mechanism in a Transformer architecture matches key to query based on both content -- the what -- and position in a sequence -- the where. We present an analysis indi…
Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning
Etai Littwin, Vimal Thilak, Anand Gopalakrishnan
Image-based Joint-Embedding Predictive Architecture (IJEPA) offers an attractive alternative to Masked Autoencoder (MAE) for representation learning using the Masked Image Modeling…
Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discovery
Anand Gopalakrishnan, Aleksandar Stanić, Jürgen Schmidhuber +1
Current state-of-the-art synchrony-based models encode object bindings with complex-valued activations and compute with real-valued weights in feedforward architectures. We argue f…
Exploring the Promise and Limits of Real-Time Recurrent Learning
Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber
Real-time recurrent learning (RTRL) for sequence-processing recurrent neural networks (RNNs) offers certain conceptual advantages over backpropagation through time (BPTT). RTRL req…
Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Aleksandar Stanić, Anand Gopalakrishnan, Kazuki Irie +1
Current state-of-the-art object-centric models use slots and attention-based routing for binding. However, this class of models has several conceptual limitations: the number of sl…
Unsupervised Learning of Temporal Abstractions with Slot-based Transformers
Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber +1
The discovery of reusable sub-routines simplifies decision-making and planning in complex reinforcement learning problems. Previous approaches propose to learn such temporal abstra…