25 citations · 27 across the 7 of their papers we have counts for
3 papers · 1 filter
Plancraft: an evaluation dataset for planning with LLM agents
Gautier Dagan, Frank Keller, Alex Lascarides
We present Plancraft, a multi-modal evaluation dataset for LLM agents. Plancraft has both a text-only and multi-modal interface, based on the Minecraft crafting GUI. We include the…
CAST: Cross-modal Alignment Similarity Test for Vision Language Models
Gautier Dagan, Olga Loginova, Anil Batra
Vision Language Models (VLMs) are typically evaluated with Visual Question Answering (VQA) tasks which assess a model's understanding of scenes. Good VQA performance is taken as ev…
Getting the most out of your tokenizer for pre-training and domain adaptation
Gautier Dagan, Gabriel Synnaeve, Baptiste Rozière
Tokenization is an understudied and often neglected component of modern LLMs. Most published works use a single tokenizer for all experiments, often borrowed from another model, wi…