4 papers
Escaping Plato's Cave: JAM for Aligning Independently Trained Vision and Language Models
Lauren Hyoseo Yoon, Yisong Yue, Been Kim
Independently trained vision and language models inhabit disjoint representational spaces, shaped by their respective modalities, objectives, and architectures. The Platonic Repres…
Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models
Saraswati Soedarmadji, Yunyue Wei, Chen Zhang +2
Discovering effective reward functions remains a fundamental challenge in motor control of high-dimensional musculoskeletal systems. While humans can describe movement goals explic…
EnCompass: Enhancing Agent Programming with Search Over Program Execution Paths
Zhening Li, Armando Solar-Lezama, Yisong Yue +1
We introduce a new approach to agent programming, the development of LLM-based agents. Current approaches to agent programming often entangle two aspects of agent design: the core…
Learning Time-Scale Invariant Population-Level Neural Representations
Eshani Patel, Yisong Yue, Geeling Chau
General-purpose foundation models for neural time series can help accelerate neuroscientific discoveries and enable applications such as brain computer interfaces (BCIs). A key com…