3 citations · 5 across the 18 of their papers we have counts for
20 papers
Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework
Dongxu Ge, Shansong Liu, Cheng Gong +3
As an important subfield of cross-modal generation, synthesizing static visual content in the form of images from audio, namely audio-to-image (A2I) generation, has attracted incre…
VTouch++: A Multimodal Dataset with Vision-Based Tactile Enhancement for Bimanual Manipulation
Qianxi Hua, Xinyue Li, Zheng Yan +4
Embodied intelligence has advanced rapidly in recent years; however, bimanual manipulation-especially in contact-rich tasks remains challenging. This is largely due to the lack of…
High-Fidelity Generative Audio Compression at 0.275kbps
Hao Ma, Ruihao Jing, Shansong Liu +4
High-fidelity general audio compression at ultra-low bitrates is crucial for applications ranging from low-bandwidth communication to generative audio-language modeling. Traditiona…
Rare Word Recognition and Translation Without Fine-Tuning via Task Vector in Speech Models
Ruihao Jing, Cheng Gong, Yu Jiang +5
Rare words remain a critical bottleneck for speech-to-text systems. While direct fine-tuning improves recognition of target words, it often incurs high cost, catastrophic forgettin…
Towards Multimodal Query-Based Spatial Audio Source Extraction
Chenxin Yu, Hao Ma, Xu Li +4
Query-based audio source extraction seeks to recover a target source from a mixture conditioned on a query. Existing approaches are largely confined to single-channel audio, leavin…
Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimation
Shiyuan Yin, Chenjia Bai, Zihao Zhang +4
Large language models (LLMs) demonstrate advanced reasoning abilities, enabling robots to understand natural language instructions and generate high-level plans with appropriate gr…
FoleySpace: Vision-Aligned Binaural Spatial Audio Generation
Lei Zhao, Rujin Chen, Chi Zhang +2
Recently, with the advancement of AIGC, deep learning-based video-to-audio (V2A) technology has garnered significant attention. However, existing research mostly focuses on mono au…
: A Multimodal, Multi-Label, Multilingual Prompt Database for Speech Generation
Boyu Zhu, Cheng Gong, Muyang Wu +5
Recent advancements in zero-shot speech generation have enabled models to synthesize speech that mimics speaker identity and speaking style from speech prompts. However, these mode…
ERF-BA-TFD+: A Multimodal Model for Audio-Visual Deepfake Detection
Xin Zhang, Jiaming Chu, Jian Zhao +5
Deepfake detection is a critical task in identifying manipulated multimedia content. In real-world scenarios, deepfake content can manifest across multiple modalities, including au…
AI Flow: Perspectives, Scenarios, and Approaches
Hongjun An, Wenhan Hu, Sida Huang +11
Pioneered by the foundational information theory by Claude Shannon and the visionary framework of machine intelligence by Alan Turing, the convergent evolution of information and c…
HAMLET: A Hierarchical and Adaptive Multi-Agent Framework for Live Embodied Theatrics
Shufan Jiang, Sizhou Chen, Chios Chen +3
Creating an immersive and interactive theatrical experience is a long-term goal in the field of interactive narrative. The emergence of large language models (LLMs) provides a new…
Conditional Video Generation for High-Efficiency Video Compression
Fangqiu Yi, Jingyu Xu, Jiawei Shao +2
Perceptual studies demonstrate that conditional diffusion models excel at reconstructing video content aligned with human visual perception. Building on this insight, we propose a…
Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection
Yuchu Jiang, Jiaming Chu, Jian Zhao +5
The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classificat…
Towards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
Chenyou Fan, Fangzheng Yan, Chenjia Bai +4
Learning a generalizable bimanual manipulation policy is extremely challenging for embodied agents due to the large action space and the need for coordinated arm movements. Existin…
Metric-Solver: Sliding Anchored Metric Depth Estimation from a Single Image
Tao Wen, Jiepeng Wang, Yabo Chen +3
Accurate and generalizable metric depth estimation is crucial for various computer vision applications but remains challenging due to the diverse depth scales encountered in indoor…
OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding
Dianbing Xi, Jiepeng Wang, Yuanzhi Liang +5
In this paper, we propose a novel framework for controllable video diffusion, OmniVDiff , aiming to synthesize and comprehend multiple video visual content in a single diffusion mo…
Bridging the Gap between Continuous and Informative Discrete Representations by Random Product Quantization
Xueqing Li, Hao Ma, Zehan Li +8
Self-supervised learning (SSL) has become a core technique in speech processing, but the high dimensionality of its representations makes discretization essential for improving eff…
NFIG: Multi-Scale Autoregressive Image Generation via Frequency Ordering
Zhihao Huang, Xi Qiu, Yukuo Ma +5
Autoregressive models have achieved significant success in image generation. However, unlike the inherent hierarchical structure of image information in the spectral domain, standa…
UniForm: A Unified Multi-Task Diffusion Transformer for Audio-Video Generation
Lei Zhao, Linfeng Feng, Dongxu Ge +5
With the rise of diffusion models, audio-video generation has been revolutionized. However, most existing methods rely on separate modules for each modality, with limited explorati…
VAST 1.0: A Unified Framework for Controllable and Consistent Video Generation
Chi Zhang, Yuanzhi Liang, Xi Qiu +2
Generating high-quality videos from textual descriptions poses challenges in maintaining temporal coherence and control over subject motion. We propose VAST (Video As Storyboard fr…