KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval

Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran +5

We study the ability of state-of-the art models to answer constraint satisfaction queries for information retrieval (e.g., 'a list of ice cream shops in San Diego'). In the past, s…