26 papers
Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training
Shuyuan Tu, Qi Tian, Yinming Huang +8
Natively training joint video-audio generation models at higher resolutions empowers them to learn richer visual details and sharper motion dynamics. However, full attention incurs…
LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation
Zijie Diao, Yitong Chen, Sicheng Xie +7
General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodie…
Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Hands
Zhenjie Yang, Yideng Zhang, Dongjie Zhang +21
Tactile sensing provides contact information that can be difficult to infer from vision alone, but tactile hardware for dexterous hands has not converged to a common design. Dexter…
VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
Yinming Huang, Shuyuan Tu, Xi Yan +6
Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual…
Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring
Zhenxin Li, Nadine Chang, Xinglong Sun +8
Behavior policies are often formulated as continuous generative models, whose iterative denoising processes are expressive but difficult to interpret and prone to producing implaus…
Agentic Visual Generation: From Generative Models to Agentic Control
Yinming Huang, Shuyuan Tu, Xi Yan +7
Visual generation is evolving from generative models used through a single invocation into agentic control processes that can plan, select tools, inspect intermediate synthesized o…
CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
Yuqian Zhang, Yao Shi, Kexin Huang +7
Zero-shot text-to-speech (TTS) now supports interactive assistants, personalized media, and accessibility tools. All TTS systems require faithful linguistic rendering, consistent s…
ETA: A New Agentic Paradigm for Embodied Tasks
Yitong Chen, Zezheng Huai, Sixian Li +7
When will robots have their ChatGPT moment? Such a breakthrough requires a general-purpose robot that can handle unfamiliar tasks in unfamiliar environments, remain controllable ov…
EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration
Zuhao Ge, Yuchen Zhou, Weitao Zhou +8
Robust embodied robots should be able to recover from failures and retry tasks in order to operate reliably in unstructured and noisy real-world environments. Achieving this capabi…
Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation
Shengqi Xu, Guojin Zhong, Yang Liu +7
Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation. These sensors achieve high spatial resolution and multi-dimensional…
Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
Junhao Shi, Zezheng Huai, Siyin Wang +7
Building persistent embodied agents in unstructured environments demands unified orchestration of heterogeneous tools spanning both cyber (APIs, IoT) and physical (manipulation, na…
MambaADv2: Evolving Duality-enhanced State Space Model for Unsupervised Anomaly Detection
Xiaobin Hu, Haoyang He, Bo Yin +5
While recent advancements in anomaly detection have demonstrated the efficacy of CNN- and Transformer-based approaches, these architectures face inherent limitations: CNNs struggle…
ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation
Tianyi Lu, Hui Zhang, Zijie Diao +8
Most Vision-Language-Action (VLA) models map observations directly to actions without explicit reasoning, limiting their capacity for reasoning-intensive long-horizon tasks. To add…
UniDexTok: A Unified Dexterous Hand Tokenizer from Real Data
Dong Fang, Youjun Wu, Yuanxin Zhong +4
Dexterous hands are essential for fine-grained manipulation, but their hardware designs vary substantially across embodiments. Differences in kinematics, joint definitions, and deg…
ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations
Junke Wang, Xiao Wang, Jiacheng Pan +16
This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-token prediction framework.…
IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder
Yitong Chen, Zijie Diao, Junke Wang +5
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spac…
OmniGen-AR: AutoRegressive Any-to-Image Generation
Junke Wang, Xun Wang, Qiushan Guo +4
Autoregressive (AR) models have demonstrated strong potential in visual generation, offering superior performance with simple architectures and optimization objectives. However, ex…
Teach Multimodal Recommendation Model to See via Personalized Visual Extraction and Adaptive Learning
Yutong Li, Xinyi Zhang, Ziyi Ye +2
Multimodal sequential recommendation (MSR) incorporates textual and visual information to improve recommendation quality. However, recent studies and our empirical analysis show th…
Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data
Linqi Yin, Shiduo Zhang, Shenling Qiu +11
Vision-language models (VLMs) are powerful general-purpose reasoners, yet converting them into robot control policies (VLAs) is surprisingly difficult. The root cause is a two-fold…
Coarse-to-Control: Action-Token Planning for Vision-Language-Action Models
Jinhao Wu, Shiduo Zhang, Yicheng Liu +9
Most vision-language-action (VLA) models map observations directly to actions without explicit intermediate planning, which limits performance on long-horizon tasks where early mis…
Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation
Shuyuan Tu, Qi Tian, Zihan Yang +9
Current open-source diffusion models struggle to generate stable and synchronized audio-visual content, particularly in scenarios demanding complex semantic reasoning. The root cau…
Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance
Runze Wang, Yuqian Fu, Yu Li +7
Vision-language-action (VLA) models have shown strong potential for generalist robot manipulation, yet they remain limited by insufficient spatial reasoning, particularly in determ…
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
Xinlei Yu, Zhangquan Chen, Yongbo He +36
Latent space is rapidly emerging as a native substrate for language-based models. While modern systems are still commonly understood through explicit token-level generation, an inc…
Back to Basics: Revisiting ASR in the Age of Voice Agents
Geeyang Tay, Wentao Ma, Jaewon Lee +8
Automatic speech recognition (ASR) systems have achieved near-human accuracy on curated benchmarks, yet still fail in real-world voice agents under conditions that current evaluati…