One Model, Multiple Modalities: A Sparsely Activated Approach for Text, Sound, Image, Video and Code

Yong Dai, Duyu Tang, Liangxin Liu +7

People perceive the world with multiple senses (e.g., through hearing sounds, reading words and seeing objects). However, most existing AI systems only process an individual modali…