4 citations · 6 across the 12 of their papers we have counts for
12 papers
GaMMA: Towards Joint Global-Temporal Music Understanding in Large Multimodal Models
Zuyao You, Zhesong Yu, Mingyu Liu +3
In this paper, we propose GaMMA, a state-of-the-art (SoTA) large multimodal model (LMM) designed to achieve comprehensive musical content understanding. GaMMA inherits the streamli…
ByteComposer: a Human-like Melody Composition Method based on Language Model Agent
Xia Liang, Xingjian Du, Jiaju Lin +3
Large Language Models (LLM) have shown encouraging progress in multimodal understanding and generation tasks. However, how to design a human-aligned and interpretable melody compos…
MINT: Boosting Audio-Language Model via Multi-Target Pre-Training and Instruction Tuning
Hang Zhao, Yifei Xin, Zhesong Yu +3
In the realm of audio-language pre-training (ALP), the challenge of achieving cross-modal alignment is significant. Moreover, the integration of audio inputs with diverse distribut…
Joint Music and Language Attention Models for Zero-shot Music Tagging
Xingjian Du, Zhesong Yu, Jiaju Lin +2
Music tagging is a task to predict the tags of music recordings. However, previous music tagging research primarily focuses on close-set music tagging tasks which can not be genera…
ByteCover3: Accurate Cover Song Identification on Short Queries
Xingjian Du, Zijie Wang, Xia Liang +3
Deep learning based methods have become a paradigm for cover song identification (CSI) in recent years, where the ByteCover systems have achieved state-of-the-art results on all th…
Graph Contrastive Learning with Implicit Augmentations
Huidong Liang, Xingjian Du, Bilei Zhu +3
Existing graph contrastive learning methods rely on augmentation techniques based on random perturbations (e.g., randomly adding or dropping edges and nodes). Nevertheless, alterin…
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Ke Chen, Xingjian Du, Bilei Zhu +3
Audio classification is an important task of mapping audio samples into their corresponding labels. Recently, the transformer model with self-attention mechanisms has been adopted…
Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data
Ke Chen, Xingjian Du, Bilei Zhu +3
Deep learning techniques for separating audio into different sound sources face several challenges. Standard architectures require training separate models for different types of a…
Attention-based cross-modal fusion for audio-visual voice activity detection in musical video streams
Yuanbo Hou, Zhesong Yu, Xia Liang +4
Many previous audio-visual voice-related works focus on speech, ignoring the singing voice in the growing number of musical video streams on the Internet. For processing diverse mu…
Rule-embedded network for audio-visual voice activity detection in live musical video streams
Yuanbo Hou, Yi Deng, Bilei Zhu +2
Detecting anchor's voice in live musical streams is an important preprocessing for music and speech signal processing. Existing approaches to voice activity detection (VAD) primari…
Contrastive Unsupervised Learning for Audio Fingerprinting
Zhesong Yu, Xingjian Du, Bilei Zhu +1
The rise of video-sharing platforms has attracted more and more people to shoot videos and upload them to the Internet. These videos mostly contain a carefully-edited background au…
ByteCover: Cover Song Identification via Multi-Loss Training
Xingjian Du, Zhesong Yu, Bilei Zhu +2
We present in this paper ByteCover, which is a new feature learning method for cover song identification (CSI). ByteCover is built based on the classical ResNet model, and two majo…