63 citations · 95 across the 20 of their papers we have counts for
12 papers · 1 filter
GME: Improving Universal Multimodal Retrieval by Multimodal LLMs
Xin Zhang, Yanzhao Zhang, Wen Xie +7
Universal Multimodal Retrieval (UMR) aims to enable search across various modalities using a unified model, where queries and candidates can consist of pure text, images, or a comb…
Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Image
Yu Zhao, Hao Fei, Xiangtai Li +6
In the visual spatial understanding (VSU) area, spatial image-to-text (SI2T) and spatial text-to-image (ST2I) are two fundamental tasks that appear in dual form. Existing methods f…
Grammar Induction from Visual, Speech and Text
Yu Zhao, Hao Fei, Shengqiong Wu +3
Grammar Induction could benefit from rich heterogeneous signals, such as text, vision, and acoustics. In the process, features from distinct modalities essentially serve complement…
SpeechEE: A Novel Benchmark for Speech Event Extraction
Bin Wang, Meishan Zhang, Hao Fei +5
Event extraction (EE) is a critical direction in the field of information extraction, laying an important foundation for the construction of structured knowledge bases. EE from tex…
An End-to-End Model for Photo-Sharing Multi-modal Dialogue Generation
Peiming Guo, Sinuo Liu, Yanzhao Zhang +4
Photo-Sharing Multi-modal dialogue generation requires a dialogue agent not only to generate text responses but also to share photos at the proper moment. Using image text caption…
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval
Xin Zhang, Yanzhao Zhang, Dingkun Long +10
We present systematic efforts in building long-context multilingual text representation model (TRM) and reranker from scratch for text retrieval. We first introduce a text encoder…
LLM-Driven Multimodal Opinion Expression Identification
Bonian Jia, Huiyao Chen, Yueheng Sun +2
Opinion Expression Identification (OEI) is essential in NLP for applications ranging from voice assistants to depression diagnosis. This study extends OEI to encompass multimodal i…
Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment
Hao Fei, Shengqiong Wu, Meishan Zhang +3
While pre-training large-scale video-language models (VLMs) has shown remarkable potential for various downstream video-language tasks, existing VLMs can still suffer from certain…
AutoSurvey: Large Language Models Can Automatically Write Surveys
Yidong Wang, Qi Guo, Wenjin Yao +10
This paper introduces AutoSurvey, a speedy and well-organized methodology for automating the creation of comprehensive literature surveys in rapidly evolving fields like artificial…
Recognizing Everything from All Modalities at Once: Grounded Multimodal Universal Information Extraction
Meishan Zhang, Hao Fei, Bin Wang +4
In the field of information extraction (IE), tasks across a wide range of modalities and their combinations have been traditionally studied in isolation, leaving a gap in deeply re…
Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training
Longhui Zhang, Dingkun Long, Meishan Zhang +3
Chinese sequence labeling tasks are heavily reliant on accurate word boundary demarcation. Although current pre-trained language models (PLMs) have achieved substantial gains on th…
In-Context Learning for Few-Shot Nested Named Entity Recognition
Meishan Zhang, Bin Wang, Hao Fei +1
In nested Named entity recognition (NER), entities are nested with each other, and thus requiring more data annotations to address. This leads to the development of few-shot nested…