Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale

Jerome Ku, Eric Nguyen, David W. Romero +13

We introduce convolutional multi-hybrid architectures, with a design grounded on two simple observations. First, operators in hybrid models can be tailored to token manipulation ta…