23 citations · 36 across the 20 of their papers we have counts for
7 papers · 1 filter
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Jianhong Tu, Zhuohao Ni, Nicholas Crispino +8
We present a novel visual instruction tuning strategy to improve the zero-shot task generalization of multimodal large language models by building a firm text-only knowledge base.…
Multimodal Reranking for Knowledge-Intensive Visual Question Answering
Haoyang Wen, Honglei Zhuang, Hamed Zamani +2
Knowledge-intensive visual question answering requires models to effectively use external knowledge to help answer visual questions. A typical pipeline includes a knowledge retriev…
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
Zhuowan Li, Cheng Li, Mingyang Zhang +2
Retrieval Augmented Generation (RAG) has been a powerful tool for Large Language Models (LLMs) to efficiently process overly lengthy contexts. However, recent LLMs like Gemini-1.5…
Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization
Hamed Zamani, Michael Bendersky
This paper introduces Stochastic RAG--a novel approach for end-to-end optimization of retrieval-augmented generation (RAG) models that relaxes the simplifying assumptions of margin…
Bridging the Preference Gap between Retrievers and LLMs
Zixuan Ke, Weize Kong, Cheng Li +3
Large Language Models (LLMs) have demonstrated superior results across a wide range of tasks, and Retrieval-augmented Generation (RAG) is an effective way to enhance the performanc…
Unlocking the `Why' of Buying: Introducing a New Dataset and Benchmark for Purchase Reason and Post-Purchase Experience
Tao Chen, Siqi Zuo, Cheng Li +3
In business and marketing, analyzing the reasons behind buying is a fundamental step towards understanding consumer behaviors, shaping business strategies, and predicting market ou…
PRewrite: Prompt Rewriting with Reinforcement Learning
Weize Kong, Spurthi Amba Hombaiah, Mingyang Zhang +2
Prompt engineering is critical for the development of LLM-based applications. However, it is usually done manually in a "trial and error" fashion that can be time consuming, ineffe…