5 papers
Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization
Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti +2
Moving beyond evaluations that collapse performance across heterogeneous prompts toward fine-grained evaluation at the prompt level, or within relatively homogeneous subsets, is ne…
Benchmarking Diversity in Image Generation via Attribute-Conditional Human Evaluation
Isabela Albuquerque, Ira Ktena, Olivia Wiles +4
Despite advances in generation quality, current text-to-image (T2I) models often lack diversity, generating homogeneous outputs. This work introduces a framework to address the nee…
Dynamic Classifier-Free Diffusion Guidance via Online Feedback
Pinelopi Papalampidi, Olivia Wiles, Ira Ktena +5
Classifier-free guidance (CFG) is a cornerstone of text-to-image diffusion models, yet its effectiveness is limited by the use of static guidance scales. This "one-size-fits-all" a…
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings
Olivia Wiles, Chuhan Zhang, Isabela Albuquerque +11
While text-to-image (T2I) generative models have become ubiquitous, they do not necessarily generate images that align with a given prompt. While previous work has evaluated T2I al…
Evaluating Numerical Reasoning in Text-to-Image Models
Ivana KajiÄ, Olivia Wiles, Isabela Albuquerque +4
Text-to-image generative models are capable of producing high-quality images that often faithfully depict concepts described using natural language. In this work, we comprehensivel…