4 citations · 4 across the 12 of their papers we have counts for
12 papers
Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings
Peixi Wu, Ke Mei, Feipeng Ma +15
Multimodal Large Language Models (MLLMs) have emerged as a promising foundation for universal multimodal embeddings. Recent studies have shown that reasoning-driven generative mult…
Countering the Over-Reliance Trap: Mitigating Object Hallucination for LVLMs via a Self-Validation Framework
Shiyu Liu, Xinyi Wen, Zhibin Lan +2
Despite progress in Large Vision Language Models (LVLMs), object hallucination remains a critical issue in image captioning task, where models generate descriptions of non-existent…
Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignment
Yan Gao, Yazheng Yang, Zhibin Lan +5
Code-switching (CS) speech translation (ST) aims to translate speech that alternates between multiple languages into a target language text, posing significant challenges due to th…
UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
Zhibin Lan, Liqiang Niu, Fandong Meng +2
The remarkable success of multimodal large language models (MLLMs) has driven advances in multimodal embeddings, yet existing models remain inherently discriminative, limiting thei…
PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models
Wanru Zhuang, Wenbo Li, Zhibin Lan +3
Text Image Machine Translation (TIMT) aims to translate texts embedded within an image into another language. Current TIMT studies primarily focus on providing translations for all…
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning
Zhibin Lan, Liqiang Niu, Fandong Meng +2
Universal multimodal embedding models play a critical role in tasks such as interleaved image-text retrieval, multimodal RAG, and multimodal clustering. However, our empirical resu…
"I've Heard of You!": Generate Spoken Named Entity Recognition Data for Unseen Entities
Jiawei Yu, Xiang Geng, Yuang Li +8
Spoken named entity recognition (NER) aims to identify named entities from speech, playing an important role in speech processing. New named entities appear every day, however, ann…
Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Training
Wenbo Li, Guohao Li, Zhibin Lan +5
Diffusion-based text-to-image models have demonstrated impressive achievements in diversity and aesthetics but struggle to generate images with legible visual texts. Existing backb…
AVG-LLaVA: An Efficient Large Multimodal Model with Adaptive Visual Granularity
Zhibin Lan, Liqiang Niu, Fandong Meng +3
Recently, large multimodal models (LMMs) have achieved significant advancements. When dealing with high-resolution images, dominant LMMs typically divide them into multiple local i…
Translatotron-V(ison): An End-to-End Model for In-Image Machine Translation
Zhibin Lan, Liqiang Niu, Fandong Meng +3
In-image machine translation (IIMT) aims to translate an image containing texts in source language into an image containing translations in target language. In this regard, convent…
A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges
Huangjun Shen, Liangying Shao, Wenbo Li +3
In recent years, multi-modal machine translation has attracted significant interest in both academia and industry due to its superior performance. It takes both textual and visual…
Exploring Better Text Image Translation with Multimodal Codebook
Zhibin Lan, Jiawei Yu, Xiang Li +5
Text image translation (TIT) aims to translate the source texts embedded in the image to target translations, which has a wide range of applications and thus has important research…