9 papers
Provably Complete Generalized Planning with LLMs
Katharina Stein, Chaahat Jain, Jörg Hoffmann +1
Generalized planning aims to compute a plan that solves all instances of a planning domain. Recent work has used LLMs to automatically generate and debug such generalized plans in…
Positional Biases Shift as Inputs Approach Context Window Limits
Blerta Veseli, Julian Chibane, Mariya Toneva +1
Large Language Models (LLMs) often struggle to use information across long inputs effectively. Prior work has identified positional biases, such as the Lost in the Middle (LiM) eff…
Improved Generalized Planning with LLMs through Strategy Refinement and Reflection
Katharina Stein, Nils Hodel, Daniel Fišer +3
LLMs have recently been used to generate Python programs representing generalized plans in PDDL planning, i.e., plans that generalize across the tasks of a given PDDL domain. Previ…
Language models can learn implicit multi-hop reasoning, but only if they have lots of training data
Yuekun Yao, Yupei Du, Dawei Zhu +2
Implicit reasoning is the ability of a language model to solve multi-hop reasoning tasks in a single forward pass, without chain of thought. We investigate this capability using GP…
Scope-enhanced Compositional Semantic Parsing for DRT
Xiulin Yang, Jonas Groschwitz, Alexander Koller +1
Discourse Representation Theory (DRT) distinguishes itself from other semantic representation frameworks by its ability to model complex semantic and discourse phenomena through st…
Fine-grained Controllable Text Generation through In-context Learning with Feedback
Sarubi Thillainathan, Alexander Koller
We present a method for rewriting an input sentence to match specific values of nontrivial linguistic features, such as dependency depth. In contrast to earlier work, our method us…
Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planning
Katharina Stein, Daniel Fišer, Jörg Hoffmann +1
Large language models (LLMs) have revolutionized a large variety of NLP tasks. An active debate is to what extent they can do reasoning and planning. Prior work has assessed the la…
Predicting generalization performance with correctness discriminators
Yuekun Yao, Alexander Koller
The ability to predict an NLP model's accuracy on unseen, potentially out-of-distribution data is a prerequisite for trustworthiness. We present a novel model that establishes uppe…
SLOG: A Structural Generalization Benchmark for Semantic Parsing
Bingzhi Li, Lucia Donatelli, Alexander Koller +3
The goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often focus on lexical gene…