7 citations · 19 across the 13 of their papers we have counts for
13 papers
DRBENCHER: Can Your Agent Identify the Entity, Retrieve Its Properties and Do the Math?
Young-Suk Lee, Ramon Fernandez Astudillo, Radu Florian
Deep research agents increasingly interleave web browsing with multi-step computation, yet existing benchmarks evaluate these capabilities in isolation, creating a blind spot in as…
MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
Yannis Katsis, Sara Rosenthal, Kshitij Fadnis +7
Retrieval-augmented generation (RAG) has recently become a very popular task for Large Language Models (LLMs). Evaluating them on multi-turn RAG conversations, where the system is…
Multi-Document Grounded Multi-Turn Synthetic Dialog Generation
Young-Suk Lee, Chulaka Gunasekara, Danish Contractor +2
We introduce a technique for multi-document grounded multi-turn synthetic dialog generation that incorporates three main ideas. First, we control the overall dialog flow using taxo…
Self-Refinement of Language Models from External Proxy Metrics Feedback
Keshav Ramji, Young-Suk Lee, Ramón Fernandez Astudillo +5
It is often desirable for Large Language Models (LLMs) to capture multiple objectives when providing a response. In document-grounded response generation, for example, agent respon…
Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs
Young-Suk Lee, Md Arafat Sultan, Yousef El-Kurdi +4
Using in-context learning (ICL) for data generation, techniques such as Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023) can train strong conversation…
AMR Parsing with Instruction Fine-tuned Pre-trained Language Models
Young-Suk Lee, Ramón Fernandez Astudillo, Radu Florian +2
Instruction fine-tuned language models on a collection of instruction annotated datasets (FLAN) have shown highly effective to improve model performance and generalization to unsee…
A Benchmark for Generalizable and Interpretable Temporal Question Answering over Knowledge Bases
Sumit Neelam, Udit Sharma, Hima Karanam +22
Knowledge Base Question Answering (KBQA) tasks that involve complex reasoning are emerging as an important research direction. However, most existing KBQA datasets focus primarily…
DocAMR: Multi-Sentence AMR Representation and Evaluation
Tahira Naseem, Austin Blodgett, Sadhana Kumaravel +7
Despite extensive research on parsing of English sentences into Abstraction Meaning Representation (AMR) graphs, which are compared to gold graphs via the Smatch metric, full-docum…
Learning Cross-Lingual IR from an English Retriever
Yulong Li, Martin Franz, Md Arafat Sultan +3
We present DR.DECR (Dense Retrieval with Distillation-Enhanced Cross-Lingual Representation), a new cross-lingual information retrieval (CLIR) system trained using multi-stage know…
Maximum Bayes Smatch Ensemble Distillation for AMR Parsing
Young-Suk Lee, Ramon Fernandez Astudillo, Thanh Lam Hoang +3
AMR parsing has experienced an unprecendented increase in performance in the last three years, due to a mixture of effects including architecture improvements and transfer learning…
Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing
Jiawei Zhou, Tahira Naseem, Ramón Fernandez Astudillo +3
Predicting linearized Abstract Meaning Representation (AMR) graphs using pre-trained sequence-to-sequence Transformer models has recently led to large improvements on AMR parsing b…
Ensembling Graph Predictions for AMR Parsing
Hoang Thanh Lam, Gabriele Picco, Yufang Hou +5
In many machine learning tasks, models are trained to predict structure data such as graphs. For example, in natural language processing, it is very common to parse texts into depe…
SYGMA: System for Generalizable Modular Question Answering OverKnowledge Bases
Sumit Neelam, Udit Sharma, Hima Karanam +21
Knowledge Base Question Answering (KBQA) tasks that in-volve complex reasoning are emerging as an important re-search direction. However, most KBQA systems struggle withgeneralizab…