2 papers
cs.CL2024
Syn-QA2: Evaluating False Assumptions in Long-tail Questions with Synthetic QA Datasets
Ashwin Daswani, Rohan Sawant, Najoung Kim
Sensitivity to false assumptions (or false premises) in information-seeking questions is critical for robust question-answering (QA) systems. Recent work has shown that false assum…
cs.CL2023
SLOG: A Structural Generalization Benchmark for Semantic Parsing
Bingzhi Li, Lucia Donatelli, Alexander Koller +3
The goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often focus on lexical gene…