584 citations · 803 across the 16 of their papers we have counts for
16 papers
MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations
Sara Rosenthal, Yannis Katsis, Vraj Shah +3
We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models. We release a benchmark of 666 t…
A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks
Sara Rosenthal, Maeda Hanafi, Yannis Katsis +2
Grounding conversations in existing passages, known as Retrieval-Augmented Generation (RAG), is an important aspect of Chat-Based Assistants powered by Large Language Models (LLMs)…
RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits
Kshitij Fadnis, Sara Rosenthal, Maeda Hanafi +2
Retrieval Augmented Generation (RAG) is an important aspect of conversing with Large Language Models (LLMs) when factually correct information is important. LLMs may provide answer…
A Library of LLM Intrinsics for Retrieval-Augmented Generation
Marina Danilevsky, Kristjan Greenewald, Chulaka Gunasekara +13
In the developer community for large language models (LLMs), there is not yet a clean pattern analogous to a software library, to support very large scale collaboration. Even for t…
MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
Yannis Katsis, Sara Rosenthal, Kshitij Fadnis +7
Retrieval-augmented generation (RAG) has recently become a very popular task for Large Language Models (LLMs). Evaluating them on multi-turn RAG conversations, where the system is…
Leveraging Large Language Models to Enhance Domain Expert Inclusion in Data Science Workflows
Jasmine Y. Shih, Vishal Mohanty, Yannis Katsis +1
Domain experts can play a crucial role in guiding data scientists to optimize machine learning models while ensuring contextual relevance for downstream use. However, in current wo…
InspectorRAGet: An Introspection Platform for RAG Evaluation
Kshitij Fadnis, Siva Sankalp Patel, Odellia Boni +4
Large Language Models (LLM) have become a popular approach for implementing Retrieval Augmented Generation (RAG) systems, and a significant amount of effort has been spent on build…
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture
Bingsheng Yao, Ishan Jindal, Lucian Popa +8
Real-world domain experts (e.g., doctors) rarely annotate only a decision label in their day-to-day workflow without providing explanations. Yet, existing low-resource learning tec…
SPOT: Knowledge-Enhanced Language Representations for Information Extraction
Jiacheng Li, Yannis Katsis, Tyler Baldwin +4
Knowledge-enhanced pre-trained models for language representation have been shown to be more effective in knowledge base construction tasks (i.e.,~relation extraction) than languag…
Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours
Eyal Shnarch, Alon Halfon, Ariel Gera +19
Text classification can be useful in many real-world scenarios, saving a lot of time for end users. However, building a custom classifier typically requires coding skills and ML kn…
Abstractified Multi-instance Learning (AMIL) for Biomedical Relation Extraction
William Hogan, Molly Huang, Yannis Katsis +5
Relation extraction in the biomedical domain is a challenging task due to a lack of labeled data and a long-tail distribution of fact triples. Many works leverage distant supervisi…
AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industry
Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar +8
Recent advances in transformers have enabled Table Question Answering (Table QA) systems to achieve high accuracy and SOTA results on open domain datasets like WikiTableQuestions a…
Theoretical Rule-based Knowledge Graph Reasoning by Connectivity Dependency Discovery
Canlin Zhang, Chun-Nan Hsu, Yannis Katsis +2
Discovering precise and interpretable rules from knowledge graphs is regarded as an essential challenge, which can improve the performances of many downstream tasks and even provid…
A Survey of the State of Explainable AI for Natural Language Processing
Marina Danilevsky, Kun Qian, Ranit Aharonov +3
Recent years have seen important advances in the quality of state-of-the-art models, but this has come at the expense of models becoming less interpretable. This survey presents an…
CORD-19: The COVID-19 Open Research Dataset
Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar +25
The COVID-19 Open Research Dataset (CORD-19) is a growing resource of scientific papers on COVID-19 and related historical coronavirus research. CORD-19 is designed to facilitate t…
Efficient Approximate Query Answering over Sensor Data with Deterministic Error Guarantees
Jaqueline Brito, Korhan Demirkaya, Boursier Etienne +3
With the recent proliferation of sensor data, there is an increasing need for the efficient evaluation of analytical queries over multiple sensor datasets. The magnitude of such da…