14 papers
VIScore: Diagnosing Planning-Relevant Quality in Latent World Models
Haiyu Wu, Randall Balestriero, Morgan Levine
Regulating the latent space to an isotropic Gaussian distribution provides a stable and information-maximized landscape for world model planning. However, the latent space property…
Latent Actions from Factorized Transition Effects under Agent Ambiguity
Heejeong Nam, Chandradithya S Jonnalagadda, Harshit Aggarwal +2
Latent Action Models (LAMs) learn action-like proxies from observation. However, in multi-object or distractor-rich scenes, observations contain not only agent motion but also dist…
OctoSense: Self-Supervised Learning for Multimodal Robot Perception
Anthony Bisulco, Jeremy Wang, Kostas Daniilidis +2
We present OctoSense, an open-source sensor platform with stereo RGB and event cameras, LiDAR, a thermal camera, an inertial measurement unit, RTK-corrected global positioning syst…
SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors
Pratyaksh Rao, Wancong Zhang, Randall Balestriero +2
Accurate dynamics models are critical for informed decision-making in robotic systems, particularly for agile aerial vehicles operating under uncertainty. Neural network dynamics m…
Sensorimotor World Models: Perception for Action via Inverse Dynamics
Petr Ivashkov, Randall Balestriero, Bernhard Schölkopf
Perception for action suggests that representations of the world should be shaped not by visual fidelity alone, but by their relevance for actions. At the same time, latent JEPA-st…
LEAP: Layer-skipping Efficiency via Adaptive Progression for Vision Transformer Distillation
Jiaqi Zhang, Ashton Lee, Anthony Wong +3
Vision Foundation Models (VFMs) with Vision Transformer (ViT) backbones, such as DINOv2, have become essential for downstream tasks like object recognition and semantic segmentatio…