7 papers
GPIC: A Giant Permissive Image Corpus for Visual Generation
Keshigeyan Chandrasegaran, Kyle Sargent, Suchir Agarwal +6
Studying scalable methods for visual generative modeling requires large, accessible, and stable datasets. We introduce GPIC, a Giant Permissive Image Corpus of approximately 28 tri…
Sliding Window Recurrences for Sequence Models
Dragos Secrieru, Garyk Brixi, Yoshua Bengio +3
Multi-hybrid architectures are poised to take over language modeling due to better quality and performance. We introduce a hierarchical decomposition framework for linear recurrenc…
VRPAgent: LLM-Driven Discovery of Heuristic Operators for Vehicle Routing Problems
André Hottung, Federico Berto, Chuanbo Hua +9
Designing high-performing heuristics for vehicle routing problems (VRPs) is a complex task that requires both intuition and deep domain knowledge. Large language model (LLM)-based…
Exploring Diffusion Transformer Designs via Grafting
Keshigeyan Chandrasegaran, Michael Poli, Daniel Y. Fu +9
Designing model architectures requires decisions such as selecting operators (e.g., attention, convolution) and configurations (e.g., depth, width). However, evaluating the impact…
Quantifying Memory Utilization with Effective State-Size
Rom N. Parnichkun, Neehal Tumma, Armin W. Thomas +6
The need to develop a general framework for architecture analysis is becoming increasingly important, given the expanding design space of sequence models. To this end, we draw insi…
Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale
Jerome Ku, Eric Nguyen, David W. Romero +13
We introduce convolutional multi-hybrid architectures, with a design grounded on two simple observations. First, operators in hybrid models can be tailored to token manipulation ta…