1.7k citations · 2k across the 14 of their papers we have counts for
7 papers · 1 filter
Jointly Training Large Autoregressive Multimodal Models
Emanuele Aiello, Lili Yu, Yixin Nie +2
In recent years, advances in the large-scale pretraining of language and text-to-image models have revolutionized the field of machine learning. Yet, integrating these two modaliti…
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
Lili Yu, Bowen Shi, Ramakanth Pasunuru +24
We present CM3Leon (pronounced "Chameleon"), a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling both text and images. C…
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
Lili Yu, Dániel Simig, Colin Flaherty +3
Autoregressive transformers are spectacular models for short sequences but scale poorly to long sequences such as high-resolution images, podcasts, code, or books. We proposed Mega…
LIMA: Less Is More for Alignment
Chunting Zhou, Pengfei Liu, Puxin Xu +12
Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and re…
VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation
Xilun Chen, Lili Yu, Wenhan Xiong +3
We propose a new two-stage pre-training framework for video-to-text generation tasks such as video captioning and video question answering: A generative encoder-decoder model is fi…
Descriptor-Enabled Rational Design of High-Entropy Materials Over Vast Chemical Spaces
Dibyendu Dey, Liangbo Liang, Liping Yu
The practically unlimited high-dimensional composition space of high-entropy materials (HEMs) has emerged as an exciting platform for functional materials design and discovery. How…