23 citations · 36 across the 20 of their papers we have counts for
21 papers
OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning
Krista Opsahl-Ong, Arnav Singhvi, Jasmine Collins +10
We introduce OfficeQA Pro, a benchmark for evaluating AI agents on grounded, multi-document reasoning over a large and heterogeneous document corpus. The corpus consists of U.S. Tr…
KARL: Knowledge Agents via Reinforcement Learning
Jonathan D. Chang, Andrew Drozdov, Shubham Toshniwal +23
We present a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic sea…
Predicting Task Performance with Context-aware Scaling Laws
Kyle Montgomery, David Park, Jianhong Tu +4
Scaling laws have transformed our understanding of large language models by linking upstream metrics like cross-entropy loss to design factors such as model size, training data, an…
Pathways of Thoughts: Multi-Directional Thinking for Long-form Personalized Question Answering
Alireza Salemi, Cheng Li, Mingyang Zhang +7
Personalization is well studied in search and recommendation, but personalized question answering remains underexplored due to challenges in inferring preferences from long, noisy,…
LLaVA-RE: Binary Image-Text Relevancy Evaluation with Multimodal Large Language Model
Tao Sun, Oliver Liu, JinJin Li +1
Multimodal generative AI usually involves generating image or text responses given inputs in another modality. The evaluation of image-text relevancy is essential for measuring res…
Refract ICL: Rethinking Example Selection in the Era of Million-Token Models
Arjun R. Akula, Kazuma Hashimoto, Krishna Srinivasan +3
The emergence of long-context large language models (LLMs) has enabled the use of hundreds, or even thousands, of demonstrations for in-context learning (ICL) - a previously imprac…
Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation
Alireza Salemi, Cheng Li, Mingyang Zhang +6
Personalized text generation requires a unique ability of large language models (LLMs) to learn from context that they often do not encounter during their standard training. One wa…
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Jianhong Tu, Zhuohao Ni, Nicholas Crispino +8
We present a novel visual instruction tuning strategy to improve the zero-shot task generalization of multimodal large language models by building a firm text-only knowledge base.…
Multimodal Reranking for Knowledge-Intensive Visual Question Answering
Haoyang Wen, Honglei Zhuang, Hamed Zamani +2
Knowledge-intensive visual question answering requires models to effectively use external knowledge to help answer visual questions. A typical pipeline includes a knowledge retriev…
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
Zhuowan Li, Cheng Li, Mingyang Zhang +2
Retrieval Augmented Generation (RAG) has been a powerful tool for Large Language Models (LLMs) to efficiently process overly lengthy contexts. However, recent LLMs like Gemini-1.5…
Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization
Hamed Zamani, Michael Bendersky
This paper introduces Stochastic RAG--a novel approach for end-to-end optimization of retrieval-augmented generation (RAG) models that relaxes the simplifying assumptions of margin…
Bridging the Preference Gap between Retrievers and LLMs
Zixuan Ke, Weize Kong, Cheng Li +3
Large Language Models (LLMs) have demonstrated superior results across a wide range of tasks, and Retrieval-augmented Generation (RAG) is an effective way to enhance the performanc…
Unlocking the `Why' of Buying: Introducing a New Dataset and Benchmark for Purchase Reason and Post-Purchase Experience
Tao Chen, Siqi Zuo, Cheng Li +3
In business and marketing, analyzing the reasons behind buying is a fundamental step towards understanding consumer behaviors, shaping business strategies, and predicting market ou…
PRewrite: Prompt Rewriting with Reinforcement Learning
Weize Kong, Spurthi Amba Hombaiah, Mingyang Zhang +2
Prompt engineering is critical for the development of LLM-based applications. However, it is usually done manually in a "trial and error" fashion that can be time consuming, ineffe…
It's All Relative! -- A Synthetic Query Generation Approach for Improving Zero-Shot Relevance Prediction
Aditi Chaudhary, Karthik Raman, Michael Bendersky
Recent developments in large language models (LLMs) have shown promise in their ability to generate synthetic query-document pairs by prompting with as few as 8 demonstrations. Thi…
Take One Step at a Time to Know Incremental Utility of Demonstration: An Analysis on Reranking for Few-Shot In-Context Learning
Kazuma Hashimoto, Karthik Raman, Michael Bendersky
In-Context Learning (ICL) is an emergent capability of Large Language Models (LLMs). Only a few demonstrations enable LLMs to be used as blackbox for new tasks. Previous studies ha…
Predicting Text Preference Via Structured Comparative Reasoning
Jing Nathan Yan, Tianqi Liu, Justin T Chiu +9
Comparative reasoning plays a crucial role in text preference prediction; however, large language models (LLMs) often demonstrate inconsistencies in their reasoning. While approach…
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling
Yaqing Wang, Jialin Wu, Tanmaya Dabral +8
Large language models (LLMs) and vision language models (VLMs) demonstrate excellent performance on a wide range of tasks by scaling up parameter counts from O(10^9) to O(10^{12})…
Automated Evaluation of Personalized Text Generation using Large Language Models
Yaqing Wang, Jiepu Jiang, Mingyang Zhang +4
Personalized text generation presents a specialized mechanism for delivering content that is specific to a user's personal context. While the research progress in this area has bee…
Learning to Rewrite Prompts for Personalized Text Generation
Cheng Li, Mingyang Zhang, Qiaozhu Mei +2
Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized mode…
Ambiguity-Aware In-Context Learning with Large Language Models
Lingyu Gao, Aditi Chaudhary, Krishna Srinivasan +3
In-context learning (ICL) i.e. showing LLMs only a few task-specific demonstrations has led to downstream gains with no task-specific fine-tuning required. However, LLMs are sensit…