API-Assisted Code Generation for Question Answering on Varied Table Structures
arXiv:2310.14687 · doi:10.18653/v1/2023.emnlp-main.897
Abstract
A persistent challenge to table question answering (TableQA) by generating executable programs has been adapting to varied table structures, typically requiring domain-specific logical forms. In response, this paper introduces a unified TableQA framework that: (1) provides a unified representation for structured tables as multi-index Pandas data frames, (2) uses Python as a powerful querying language, and (3) uses few-shot prompting to translate NL questions into Python programs, which are executable on Pandas data frames. Furthermore, to answer complex relational questions with extended program functionality and external knowledge, our framework allows customized APIs that Python programs can call. We experiment with four TableQA datasets that involve tables of different structures -- relational, multi-table, and hierarchical matrix shapes -- and achieve prominent improvements over past state-of-the-art systems. In ablation studies, we (1) show benefits from our multi-index representation and APIs over baselines that use only an LLM, and (2) demonstrate that our approach is modular and can incorporate additional APIs.
EMNLP 2023 camera ready, 13 pages, 11 figures
References in corpus (9)
- Evaluating Large Language Models Trained on Code
- StarCoder: may the source be with you!
- Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
- PAL: Program-aided Language Models
- TUTA: Tree-based Transformers for Generally Structured Table Pre-training
- Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing
- TAPEX: Table Pre-training via Learning a Neural SQL Executor
- DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction
- Binding Language Models in Symbolic Languages