112 citations · 304 across the 149 of their papers we have counts for
62 papers · 1 filter
Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow
Karthik Dharmarajan, Wenlong Huang, Jiajun Wu +2
Generative video modeling has emerged as a compelling tool to zero-shot reason about plausible physical interactions for open-world manipulation. Yet, it remains a challenge to tra…
Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing
Yuwen Li, Wei Zhang, Zelong Huang +8
Enabling Large Language Models (LLMs) to reliably invoke external tools remains a critical bottleneck for autonomous agents. Existing approaches suffer from three fundamental chall…
Autoregressive Flow Matching for Motion Prediction
Johnathan Xie, Stefan Stojanov, Cristobal Eyzaguirre +2
Motion prediction has been studied in different contexts with models trained on narrow distributions and applied to downstream tasks in human motion prediction and robotics. Simult…
Empowering LLMs with Structural Role Inference for Zero-Shot Graph Learning
Heng Zhang, Jing Liu, Jiajun Wu +8
Large Language Models have emerged as a promising approach for graph learning due to their powerful reasoning capabilities. However, existing methods exhibit systematic performance…
UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models
Jiajun Wu, Jian Yang, Wei Zhang +6
Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, their effectiveness heavily relies on supervised training with extensive l…
VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression
Kyle Sargent, Ruiqi Gao, Philipp Henzler +5
Evaluations of image compression performance which include human preferences have generally found that naive distortion functions such as MSE are insufficiently aligned to human pe…
WonderZoom: Multi-Scale 3D World Generation
Jin Cao, Hong-Xing Yu, Jiajun Wu
We present WonderZoom, a novel approach to generating 3D scenes with contents across multiple spatial scales from a single image. Existing 3D world generation models remain limited…
From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
Jian Yang, Xianglong Liu, Weifeng Lv +68
Large language models (LLMs) have fundamentally transformed automated software development by enabling direct translation of natural language descriptions into functional code, dri…
A Tool Bottleneck Framework for Clinically-Informed and Interpretable Medical Image Understanding
Christina Liu, Alan Q. Wang, Joy Hsu +2
Recent tool-use frameworks powered by vision-language models (VLMs) improve image understanding by grounding model predictions with specialized tools. Broadly, these frameworks lev…
ART: Articulated Reconstruction Transformer
Zizhang Li, Cheng Zhang, Zhengqin Li +7
We introduce ART, Articulated Reconstruction Transformer -- a category-agnostic, feed-forward model that reconstructs complete 3D articulated objects from only sparse, multi-state…
Seeing the Wind from a Falling Leaf
Zhiyuan Gao, Jiageng Mao, Hong-Xing Yu +5
A longstanding goal in computer vision is to model motions from videos, while the representations behind motions, i.e. the invisible physical interactions that cause objects to def…
Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Driving
Haohong Lin, Yunzhi Zhang, Wenhao Ding +2
End-to-end (E2E) autonomous driving models have demonstrated strong performance in open-loop evaluations but often suffer from cascading errors and poor generalization in closed-lo…
ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
Qineng Wang, Wenlong Huang, Yu Zhou +8
Embodied cognition argues that intelligence arises from sensorimotor interaction rather than passive observation. It raises an intriguing question: do modern vision-language models…
10 Open Challenges Steering the Future of Vision-Language-Action Models
Soujanya Poria, Navonil Majumder, Chia-Yu Hung +7
Due to their ability of follow natural language instructions, vision-language-action (VLA) models are increasingly prevalent in the embodied AI arena, following the widespread succ…
TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System
Yanjie Ze, Siheng Zhao, Weizhuo Wang +6
Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effe…
Exploringand Unleashing the Power of Large Language Models in CI/CD Configuration Translation
Chong Wang, Chen Zhang, Jiajun Wu +4
Continuous Integration (CI) is a cornerstone of modern collaborative software development, and numerous CI platforms are available. Differences in maintenance overhead, reliability…
Web-Scale Collection of Video Data for 4D Animal Reconstruction
Brian Nlong Zhao, Jiajun Wu, Shangzhe Wu
Computer vision for animals holds great promise for wildlife research but often depends on large-scale data, while existing collection methods rely on controlled capture setups. Re…
Rotate Both Ways: Time-and-Order RoPE for Generative Recommendation
Xiaokai Wei, Jiajun Wu, Daiyao Yi +2
Generative recommenders, typically transformer-based autoregressive models, predict the next item or action from a user's interaction history. Their effectiveness depends on how th…
VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
Kangrui Wang, Pingyue Zhang, Zihan Wang +13
A key challenge in training Vision-Language Model (VLM) agents, compared to Language Model (LLM) agents, lies in the shift from textual states to complex visual observations. This…
The Layout Is the Model: On Action-Item Coupling in Generative Recommendation
Xiaokai Wei, Jiajun Wu, Daiyao Yi +2
Generative Recommendation (GR) models treat a user's interaction history as a sequence to be autoregressively predicted. When both items and actions (e.g., watch time, purchase, co…
Coupled Diffusion Sampling for Training-Free Multi-View Image Editing
Hadi Alzayer, Yunzhi Zhang, Chen Geng +2
We present an inference-time diffusion sampling method to perform multi-view consistent image editing using pre-trained 2D image editing models. These models can independently prod…
Ctrl-VI: Controllable Video Synthesis via Variational Inference
Haoyi Duan, Yunzhi Zhang, Yilun Du +1
Many video workflows benefit from a mixture of user controls with varying granularity, from exact 4D object trajectories and camera paths to coarse text prompts, while existing vid…
From Programs to Poses: Factored Real-World Scene Generation via Learned Program Libraries
Joy Hsu, Emily Jin, Jiajun Wu +1
Real-world scenes, such as those in ScanNet, are difficult to capture, with highly limited data available. Generating realistic scenes with varied object poses remains an open and…
Dual-stage and Lightweight Patient Chart Summarization for Emergency Physicians
Jiajun Wu, Swaleh Zaidi, Braden Teitge +4
Electronic health records (EHRs) contain extensive unstructured clinical data that can overwhelm emergency physicians trying to identify critical information. We present a two-stag…