Video Pre-trained Transformer: A Multimodal Mixture of Pre-trained Experts

Kastan Day, Daniel Christl, Rohan Salvi +1

We present Video Pre-trained Transformer. VPT uses four SOTA encoder models from prior work to convert a video into a sequence of compact embeddings. Our backbone, based on a refer…