8 citations · 14 across the 20 of their papers we have counts for
10 papers · 1 filter
Trajectory Planning without Trajectory Data: A Manifold-Guided Approach
Silong Yong, Anji Liu, Cunxi Dai +5
A common way for trajectory planning is to leverage generative models trained on large collections of expert trajectories. At inference time, the model generates executable traject…
Video2Skill: From Streaming Experience to Reusable Embodied Skills
Jianshu Zhang, Ce Zhang, Xiyuan Yang +6
Manipulation behaviors vary widely across objects and scenes, but they share a small set of reusable skills, and planning with these skills helps embodied agents generalize to new…
StreamScout: Learning When to Look Deeper for Streaming Video Understanding
Ce Zhang, Jing Bi, Jinxi He +9
Streaming video understanding requires answering questions that arrive at arbitrary moments over an unbounded video stream. Existing systems primarily focus on what to retain in a…
LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos
Ce Zhang, Jinxi He, Katia Sycara +1
Despite rapid progress in Multi-modal Large Language Models (MLLMs), understanding long-form videos is still bottlenecked by limited context windows. While recent keyframe sampling…
\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
Jianghui Wang, Silong Yong, Francesco Orabona +3
Low-Rank Adaptation (LoRA) has become a widely adopted technique for efficient neural network fine-tuning, decomposing model updates into low-rank matrices. However, LoRA remains c…
Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation
Muyi Bao, Yuxin Cai, Hang Xu +7
Vision-language models (VLMs) have become a common foundation for vision-and-language navigation in continuous environments (VLN-CE). Yet most VLM-based methods cast navigation as…
Jailbreaking Frontier Foundation Models Through Intention Deception
Xinhe Wang, Katia Sycara, Yaqi Xie
Large (vision-)language models exhibit remarkable capability but remain highly susceptible to jailbreaking. Existing safety training approaches aim to have the model learn a refusa…
Generalizable Dense Reward for Long-Horizon Robotic Tasks
Silong Yong, Stephen Sheng, Carl Qi +6
Existing robotic foundation policies are trained primarily via large-scale imitation learning. While such models demonstrate strong capabilities, they often struggle with long-hori…
Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memory
Ce Zhang, Jinxi He, Junyi He +2
Multi-modal Large Language Models (MLLMs) have achieved remarkable performance across a wide range of visual reasoning tasks, yet their vulnerability to safety risks remains a pres…
pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoning
Zhanpeng Luo, Ce Zhang, Silong Yong +6
Multi-modal Large Language Models (MLLMs) have demonstrated strong capabilities in general-purpose perception and reasoning, but they still struggle with tasks that require spatial…