7 papers
FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation
Guangyu Sun, Shlok Kumar Mishra, Wentao Bao +8
Traditional multimodal representation learning and generation are two stages: a contrastive or self-supervised visual encoder is trained first, followed by a separate downstream ge…
MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
Xuanming Cui, Shlok Kumar Mishra, Wentao Bao +6
Universal multimodal embedding (UME) increasingly demands encoder's capacity for handling a broad range of tasks and modalities with increased complexity. Prior scaling methods eit…
An Event is Worth One Token: Event Tokenization for Industrial-scale LLM Recommendation
Fan Xia, Zhaoheng Zheng, Iman Setayesh +12
LLM-based recommendation has scaled along model capacity and sequence length, yet each position encodes only text, semantic IDs, or a few categorical features, discarding rich user…
TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens
Jianpeng Cheng, Xian Wu, Jiangfan Zhang +16
Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradigm, a generative model produc…
Reason to Contrast: A Cascaded Multimodal Retrieval Framework
Xuanming Cui, Hong-You Chen, Hao Yu +10
Traditional multimodal retrieval systems rely primarily on bi-encoder architectures, where performance is closely tied to embedding dimensionality. Recent work, Think-Then-Embed (T…
CAMPHOR: Collaborative Agents for Multi-input Planning and High-Order Reasoning On Device
Yicheng Fu, Raviteja Anantha, Jianpeng Cheng
While server-side Large Language Models (LLMs) demonstrate proficiency in function calling and complex reasoning, deploying Small Language Models (SLMs) directly on devices brings…
UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity
Yicheng Fu, Raviteja Anantha, Prabal Vashisht +2
Generating user intent from a sequence of user interface (UI) actions is a core challenge in comprehensive UI understanding. Recent advancements in multimodal large language models…