activity
20172023
most citedVATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

339 citations · 576 across the 7 of their papers we have counts for

collaborators
Showing 2023Show all

2 papers · 1 filter

cs.CV2023★ 22 cited

VideoPoet: A Large Language Model for Zero-Shot Video Generation

Dan Kondratyuk, Lijun Yu, Xiuye Gu +28

We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals. VideoPoet employs a decoder-on…

cs.CV2023★ 6 cited

Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception

Hassan Akbari, Dan Kondratyuk, Yin Cui +3

We present Integrated Multimodal Perception (IMP), a simple and scalable multimodal multi-task training and modeling approach. IMP integrates multimodal inputs including image, vid…