9 papers
Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval
Zelong Sun, Jun Wang, Kaicheng Yang +3
Unified multimodal retrieval aims to identify candidates that satisfy complex user intent expressed through heterogeneous inputs. Although Large Vision-Language Model (LVLM)-based…
UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function
Zhichao Wang, Bin Bi, Zixu Zhu +6
By pretraining on trillions of tokens, an LLM gains the capability of text generation. However, to enhance its utility and reduce potential harm, SFT and alignment are applied sequ…
DanQing: An Up-to-Date Large-Scale Chinese Vision-Language Pre-training Dataset
Hengyu Shen, Tiancheng Gu, Bin Qin +10
Vision-Language Pre-training (VLP) models have achieved remarkable success by leveraging large-scale image-text pairs. While English-centric models like CLIP and SigLIP benefit fro…
Memento 2: Learning by Stateful Reflective Memory
Jun Wang
We present a theoretical study of continual and experiential learning in large language model agents that combine episodic memory with reinforcement learning. We argue that the key…
Can Synthetic Images Serve as Effective and Efficient Class Prototypes?
Dianxing Shi, Dingjie Fu, Yuqiao Liu +1
Vision-Language Models (VLMs) have shown strong performance in zero-shot image classification tasks. However, existing methods, including Contrastive Language-Image Pre-training (C…
OmniInfer: System-Wide Acceleration Techniques for Optimizing LLM Serving Throughput and Latency
Jun Wang, Yunxiang Yao, Wenwei Kuang +11
Large Language Models drive a wide range of modern AI applications but impose substantial challenges on large-scale serving systems due to intensive computation, strict latency con…