9 citations · 14 across the 20 of their papers we have counts for
24 papers
PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
Siqi Yang, Yimu Wang, Ziqi Cai +6
A multiplayer world model must predict how the independent actions of several players change a shared environment, including changes outside a player's current field of view. A poi…
SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
Yunheng Liu, Ziqi Cai, Siqi Yang +7
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent train…
StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generation
Kaiqi Liu, Haoxuan Zeng, Jingqi Liu +7
Recent advancements in generative models are pushing video generation toward unbounded streaming audio-video generation for real-time interactive worlds. However, existing benchmar…
FlowDance: Music-Driven Dance Video Generation with Parallel Pose and RGB Streams
Genying Li, Boda Lin, Jiachen Li +5
Music-driven dance video synthesis aims to animate a reference person according to a given music clip. The task is challenging because it requires a model to jointly learn music-to…
MASS: Multiplayer World Models with Authoritative Shared State
Ziqi Cai, Siqi Yang, Yimu Wang +6
Current video world models struggle in multiplayer environments because they entangle world state with view-dependent visual latents, leading to redundant compute, view inconsisten…
MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control
Kaiqi Liu, Yunyao Mao, Ziqi Cai +8
While recent generative models produce high-fidelity videos, they struggle with the complex narrative control required for coherent multi-shot audio-visual generation. Existing met…
Video Generation Models Are Inherent Lighting Estimators
Ziqi Cai, Shuchen Weng, Kaiqi Liu +5
Recovering dynamic environment maps from a single in-the-wild video is crucial for photorealistic rendering, yet remains a challenge. Recent video generation models can produce pho…
InstructAV2AV: Instruction-Guided Audio-Video Joint Editing
Haojie Zheng, Yixin Yang, Siqi Yang +2
Recent diffusion-based methods have achieved impressive progress in video content manipulation. However, they typically ignore the accompanying audio, leaving the audio disjointed…
A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation
Peixuan Zhang, Chang Zhou, Ziyuan Zhang +8
The surging demand for adapting long-form cinematic content into short videos has motivated the need for versatile automatic video compilation systems. However, existing compilatio…
ReContraster: Making Your Posters Stand Out with Regional Contrast
Peixuan Zhang, Zijian Jia, Ziqi Cai +3
Effective poster design requires rapidly capturing attention and clearly conveying messages. Inspired by the ``contrast effects'' principle, we propose ReContraster, the first trai…
Lighting-grounded Video Generation with Renderer-based Agent Reasoning
Ziqi Cai, Taoyu Yang, Zheng Chang +4
Diffusion models have achieved remarkable progress in video generation, but their controllability remains a major limitation. Key scene factors such as layout, lighting, and camera…
Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors
Peixuan Zhang, Shuchen Weng, Jiajun Tang +2
Social media platforms enable users to express emotions by posting text with accompanying images. In this paper, we propose the Affective Image Filter (AIF) task, which aims to ref…
STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative
Peixuan Zhang, Zijian Jia, Kaiqi Liu +3
While recent advancements in generative models have achieved remarkable visual fidelity in video synthesis, creating coherent multi-shot narratives remains a significant challenge.…
AVI-Edit: Audio-sync Video Instance Editing with Granularity-Aware Mask Refiner
Haojie Zheng, Shuchen Weng, Jingqi Liu +3
Recent advancements in video generation highlight that realistic audio-visual synchronization is crucial for engaging content creation. However, existing video editing methods larg…
Personalized Image Filter: Mastering Your Photographic Style
Chengxuan Zhu, Shuchen Weng, Jiacong Fang +4
Photographic style, as a composition of certain photographic concepts, is the charm behind renowned photographers. But learning and transferring photographic style need a profound…
PanoWan: Lifting Diffusion Video Generation Models to 360° with Latitude/Longitude-aware Mechanisms
Yifei Xia, Shuchen Weng, Siqi Yang +6
Panoramic video generation enables immersive 360° content creation, valuable in applications that demand scene-consistent world exploration. However, existing panoramic video gener…
Audio-Sync Video Generation with Multi-Stream Temporal Control
Shuchen Weng, Haojie Zheng, Zheng Chang +3
Audio is inherently temporal and closely synchronized with the visual world, making it a naturally aligned and expressive control signal for controllable video generation (e.g., mo…
Affective Image Editing: Shaping Emotional Factors via Text Descriptions
Peixuan Zhang, Shuchen Weng, Chengxuan Zhu +4
In daily life, images as common affective stimuli have widespread applications. Despite significant progress in text-driven image editing, there is limited work focusing on underst…
L-C4: Language-Based Video Colorization for Creative and Consistent Color
Zheng Chang, Shuchen Weng, Huan Ouyang +3
Automatic video colorization is inherently an ill-posed problem because each monochrome frame has multiple optional color candidates. Previous exemplar-based video colorization met…
VIRES: Video Instance Repainting via Sketch and Text Guided Generation
Shuchen Weng, Haojie Zheng, Peixuan Zhang +4
We introduce VIRES, a video instance repainting method with sketch and text guidance, enabling video instance repainting, replacement, generation, and removal. Existing approaches…
Colorizing Monochromatic Radiance Fields
Yean Cheng, Renjie Wan, Shuchen Weng +3
Though Neural Radiance Fields (NeRF) can produce colorful 3D representations of the world by using a set of 2D images, such ability becomes non-existent when only monochromatic ima…
Language-guided Image Reflection Separation
Haofeng Zhong, Yuchen Hong, Shuchen Weng +2
This paper studies the problem of language-guided reflection separation, which aims at addressing the ill-posed reflection separation problem by introducing language descriptions t…
L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors
Zheng Chang, Shuchen Weng, Peixuan Zhang +3
Language-based colorization produces plausible and visually pleasing colors under the guidance of user-friendly natural language descriptions. Previous methods implicitly assume th…
STS Classification with Dual-stream CNN
Shuchen Weng, Wenbo Li, Yi Zhang +1
The structured time series (STS) classification problem requires the modeling of interweaved spatiotemporal dependency. most previous STS classification methods model the spatial a…