5 citations · 22 across the 25 of their papers we have counts for
13 papers · 1 filter
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
Zhen Ye, Peiwen Sun, Jiahe Lei +9
Recent advancements in audio generation have been significantly propelled by the capabilities of Large Language Models (LLMs). The existing research on audio LLM has primarily focu…
Importance Weighting Can Help Large Language Models Self-Improve
Chunyang Jiang, Chi-min Chan, Wei Xue +2
Large language models (LLMs) have shown remarkable capability in numerous tasks and applications. However, fine-tuning LLMs using high-quality datasets under external supervision r…
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
Peijie Dong, Lujun Li, Yuedong Zhong +8
In this paper, we present the first structural binarization method for LLM compression to less than 1-bit precision. Although LLMs have achieved remarkable performance, their memor…
MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions
Xiaowei Chi, Yatian Wang, Aosong Cheng +16
Massive multi-modality datasets play a significant role in facilitating the success of large video-language models. However, current video-language datasets primarily provide text…
LLMs Meet Multimodal Generation and Editing: A Survey
Yingqing He, Zhaoyang Liu, Jingye Chen +13
With the recent advancement in large language models (LLMs), there is a growing interest in combining LLMs with multimodal learning. Previous surveys of multimodal large language m…
Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attention
Mengfei Li, Xiaoxiao Long, Yixun Liang +6
Despite recent advancements in the Large Reconstruction Model (LRM) demonstrating impressive results, when extending its input from single image to multiple images, it exhibits ine…
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling
Zeyue Tian, Zhaoyang Liu, Ruibin Yuan +6
In this work, we systematically study music generation conditioned solely on the video. First, we present a large-scale dataset comprising 360K video-music pairs, including various…
FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generation
Jianyi Chen, Wei Xue, Xu Tan +3
Singing Accompaniment Generation (SAG), which generates instrumental music to accompany input vocals, is crucial to developing human-AI symbiotic art creation systems. The state-of…
CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild
Xingqun Qi, Hengyuan Zhang, Yatian Wang +10
Deriving co-speech 3D gestures has seen tremendous progress in virtual avatar animation. Yet, the existing methods often produce stiff and unreasonable gestures with unseen human s…
Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention
Peng Li, Yuan Liu, Xiaoxiao Long +10
In this paper, we introduce Era3D, a novel multiview diffusion method that generates high-resolution multiview images from a single-view image. Despite significant advancements in…
FlashSpeech: Efficient Zero-Shot Speech Synthesis
Zhen Ye, Zeqian Ju, Haohe Liu +10
Recent progress in large-scale zero-shot speech synthesis has been significantly advanced by language models and diffusion models. However, the generation process of both methods i…
ChatMusician: Understanding and Generating Music Intrinsically with LLM
Ruibin Yuan, Hanfeng Lin, Yi Wang +32
While Large Language Models (LLMs) demonstrate impressive capabilities in text generation, we find that their ability has yet to be generalized to music, humanity's creative langua…
CoMoSVC: Consistency Model-based Singing Voice Conversion
Yiwen Lu, Zhen Ye, Wei Xue +3
The diffusion-based Singing Voice Conversion (SVC) methods have achieved remarkable performances, producing natural audios with high similarity to the target timbre. However, the i…