4 papers
Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss
Niccolò Ajroldi, Diana Alexandra Onutu, Haider Al-Tahan +4
We study the scaling behavior of learning rate and batch size in pretraining dense large language models on English-prevalent corpora. Beyond scaling jointly optimal learning rates…
Topographic Constraints Shape Brain-Like Component Structure in Auditory Models
Haider Al-Tahan, Mayukh Deb, Jenelle Feather +1
If topography is a fundamental feature of the brain, it should influence both how neurons are arranged in space (i.e. explain brain maps) and how information is structured within t…
UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
Haider Al-Tahan, Quentin Garrido, Randall Balestriero +3
Significant research efforts have been made to scale and improve vision-language model (VLM) training approaches. Yet, with an ever-growing number of benchmarks, researchers are ta…
An Introduction to Vision-Language Modeling
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay +38
Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guid…