11 papers
StepAudio 3 Realtime Technical Report
Bin Lin, Bo Zhao, Boyang Zhang +87
Realtime spoken interaction demands deep reasoning, prompt responses, and fluid turn-taking. We present StepAudio 3 Realtime, an audio-language foundation model organized around a…
StepAudio 3 Gen Technical Report
Bin Lin, Bo Zhao, Boyang Wang +68
We introduce StepAudio 3 Gen, a general-purpose audio generation model that supports zero-shot text-to-speech (TTS), voice design, vocal generation, sound effects, music, vibe spee…
NeuROK: Generative 4D Neural Object Kinematics
Chen Geng, Guangzhao He, Yue Gao +3
Data-driven approaches have revolutionized 3D vision, enabling transformers to effectively reconstruct and generate static 3D objects. However, generating simulative 4D dynamics --…
StepAudio 2.5 Technical Report
Bin Lin, Bo Zhao, Boyong Wu +98
Unified audio-language modeling has emerged as a prominent trend in modern speech systems, promising to bring the reasoning capabilities of large language models to auditory tasks.…
GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates
Youngjoong Kwon, Yao He, Heejung Choi +4
We present a feed-forward human performance capture method that renders novel views of a performer from a monocular RGB stream. A key challenge in this setting is the lack of suffi…
Choreographing a World of Dynamic Objects
Yanzhe Lyu, Chen Geng, Karthik Dharmarajan +4
Dynamic objects in our physical 4D (3D + time) world are constantly evolving, deforming, and interacting with other objects, leading to diverse 4D scene dynamics. In this paper, we…
ART: Articulated Reconstruction Transformer
Zizhang Li, Cheng Zhang, Zhengqin Li +7
We introduce ART, Articulated Reconstruction Transformer -- a category-agnostic, feed-forward model that reconstructs complete 3D articulated objects from only sparse, multi-state…
Coupled Diffusion Sampling for Training-Free Multi-View Image Editing
Hadi Alzayer, Yunzhi Zhang, Chen Geng +2
We present an inference-time diffusion sampling method to perform multi-view consistent image editing using pre-trained 2D image editing models. These models can independently prod…
Category-Agnostic Neural Object Rigging
Guangzhao He, Chen Geng, Shangzhe Wu +1
The motion of deformable 4D objects lies in a low-dimensional manifold. To better capture the low dimensionality and enable better controllability, traditional methods have devised…
Anymate: A Dataset and Baselines for Learning 3D Object Rigging
Yufan Deng, Yuhao Zhang, Chen Geng +2
Rigging and skinning are essential steps to create realistic 3D animations, often requiring significant expertise and manual effort. Traditional attempts at automating these proces…
Birth and Death of a Rose
Chen Geng, Yunzhi Zhang, Shangzhe Wu +1
We study the problem of generating temporal object intrinsics -- temporally evolving sequences of object geometry, reflectance, and texture, such as a blooming rose -- from pre-tra…