Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics

Xiaoyuan Guo, Jiali Duan, C. -C. Jay Kuo +2

Language modality within the vision language pretraining framework is innately discretized, endowing each word in the language vocabulary a semantic meaning. In contrast, visual mo…