15 citations · 32 across the 10 of their papers we have counts for
12 papers
CoverSynth: Coverage-Guided Generation of Retrieval Evaluation Sets
Anahita Sharma, Apaar Shanker, Rishav Chakravarti +6
Enterprise agents need retrieval evaluation sets that expose domain-specific failures and guide developers towards efficient deployment tradeoffs. Neither expert-human-authored set…
Coverage, Not Averages: Semantic Stratification for Trustworthy Retrieval Evaluation
Andrew Klearman, Radu Revutchi, Rohin Garg +3
Retrieval quality is the primary bottleneck for accuracy and robustness in retrieval-augmented generation (RAG). Current evaluation relies on heuristically constructed query sets,…
CoddLLM: Empowering Large Language Models for Data Analytics
Jiani Zhang, Hengrui Zhang, Rishav Chakravarti +6
Large Language Models (LLMs) have the potential to revolutionize data analytics by simplifying tasks such as data discovery and SQL query synthesis through natural language interac…
Capturing Row and Column Semantics in Transformer Based Question Answering over Tables
Michael Glass, Mustafa Canim, Alfio Gliozzo +7
Transformer based architectures are recently used for the task of answering questions over tables. In order to improve the accuracy on this task, specialized pre-training technique…
Towards Confident Machine Reading Comprehension
Rishav Chakravarti, Avirup Sil
There has been considerable progress on academic benchmarks for the Reading Comprehension (RC) task with State-of-the-Art models closing the gap with human performance on extractiv…
The TechQA Dataset
Vittorio Castelli, Rishav Chakravarti, Saswati Dana +18
We introduce TechQA, a domain-adaptation question answering dataset for the technical support domain. The TechQA corpus highlights two real-world issues from the automated customer…
Ensembling Strategies for Answering Natural Questions
Anthony Ferritto, Lin Pan, Rishav Chakravarti +4
Many of the top question answering systems today utilize ensembling to improve their performance on tasks such as the Stanford Question Answering Dataset (SQuAD) and Natural Questi…
Frustratingly Easy Natural Question Answering
Lin Pan, Rishav Chakravarti, Anthony Ferritto +5
Existing literature on Question Answering (QA) mostly focuses on algorithmic novelty, data augmentation, or increasingly large pre-trained language models like XLNet and RoBERTa. A…
Span Selection Pre-training for Question Answering
Michael Glass, Alfio Gliozzo, Rishav Chakravarti +5
BERT (Bidirectional Encoder Representations from Transformers) and related pre-trained Transformers have provided large gains across many language understanding tasks, achieving a…
CFO: A Framework for Building Production NLP Systems
Rishav Chakravarti, Cezar Pendus, Andrzej Sakrajda +8
This paper introduces a novel orchestration framework, called CFO (COMPUTATION FLOW ORCHESTRATOR), for building, experimenting with, and deploying interactive NLP (Natural Language…
A Study on Passage Re-ranking in Embedding based Unsupervised Semantic Search
Md Faisal Mahbub Chowdhury, Vijil Chenthamarakshan, Rishav Chakravarti +1
State of the art approaches for (embedding based) unsupervised semantic search exploits either compositional similarity (of a query and a passage) or pair-wise word (or term) simil…
Improved Answer Selection with Pre-Trained Word Embeddings
Rishav Chakravarti, Jiri Navratil, Cicero Nogueira dos Santos
This paper evaluates existing and newly proposed answer selection methods based on pre-trained word embeddings. Word embeddings are highly effective in various natural language pro…