13 papers
Model Space Reasoning as Search in Feedback Space for Planning Domain Generation
James Oswald, Daniel Obolensky, Volodymyr Varha +5
The generation of planning domains from natural language descriptions remains an open problem even with the advent of large language models and reasoning models. Recent work sugges…
Seemingly Simple Planning Problems are Computationally Challenging: The Countdown Game
Michael Katz, Harsha Kokel, Sarath Sreedharan
There is a broad consensus that the inability to form long-term plans is one of the key limitations of current foundational models and agents. However, the existing planning benchm…
ACPBench Hard: Unrestrained Reasoning about Action, Change, and Planning
Harsha Kokel, Michael Katz, Kavitha Srinivas +1
The ACPBench dataset provides atomic reasoning tasks required for efficient planning. The dataset is aimed at distilling the complex plan generation task into separate atomic reaso…
ACPBench: Reasoning about Action, Change, and Planning
Harsha Kokel, Michael Katz, Kavitha Srinivas +1
There is an increasing body of work using Large Language Models (LLMs) as agents for orchestrating workflows and making decisions in domains that require planning and multi-step re…
Thought of Search: Planning with Language Models Through The Lens of Efficiency
Michael Katz, Harsha Kokel, Kavitha Srinivas +1
Among the most important properties of algorithms investigated in computer science are soundness, completeness, and complexity. These properties, however, are rarely analyzed for t…
QueryGym: Step-by-Step Interaction with Relational Databases
Haritha Ananthakrishnan, Harsha Kokel, Kelsey Sikes +4
We introduce QueryGym, an interactive environment for building, testing, and evaluating LLM-based query planning agents. Existing frameworks often tie agents to specific query lang…