7 papers
Joint Agent Memory and Exploration Learning via Novelty Signals
Shizuo Tian, Xiaohong Weng, Rui Kong +9
In open-ended environments, exploration is fundamental for autonomous agents, yet current language model agents struggle with this. Effective exploration requires memory, but retai…
Winfree Oscillatory Neural Network
Jiawen Dai, Yue Song
Oscillations and synchronization are widely believed to play a fundamental role in representation and computation. However, existing machine learning approaches based on synchroniz…
Deconfounded Lifelong Learning for Autonomous Driving via Dynamic Knowledge Spaces
Jiayuan Du, Yuebing Song, Yiming Zhao +6
End-to-End autonomous driving (E2E-AD) systems face challenges in lifelong learning, including catastrophic forgetting, difficulty in knowledge transfer across diverse scenarios, a…
Langevin Flows for Modeling Neural Latent Dynamics
Yue Song, T. Anderson Keller, Yisong Yue +2
Neural populations exhibit latent dynamical structures that drive time-evolving spiking activities, motivating the search for models that capture both intrinsic network dynamics an…
Is CLIP ideal? No. Can we fix it? Yes!
Raphi Kang, Yue Song, Georgia Gkioxari +1
Contrastive Language-Image Pre-Training (CLIP) is a popular method for learning multimodal latent spaces with well-organized semantics. Despite its wide range of applications, CLIP…
Morphological-Symmetry-Equivariant Heterogeneous Graph Neural Network for Robotic Dynamics Learning
Fengze Xie, Sizhe Wei, Yue Song +2
We present a morphological-symmetry-equivariant heterogeneous graph neural network, namely MS-HGNN, for robotic dynamics learning, that integrates robotic kinematic structures and…
Unsupervised Representation Learning from Sparse Transformation Analysis
Yue Song, Thomas Anderson Keller, Yisong Yue +2
There is a vast literature on representation learning based on principles such as coding efficiency, statistical independence, causality, controllability, or symmetry. In this pape…