Transforming LLMs into Cross-modal and Cross-lingual Retrieval Systems

Frank Palma Gomez, Ramon Sanabria, Yun-hsuan Sung +3

Large language models (LLMs) are trained on text-only data that go far beyond the languages with paired speech and text data. At the same time, Dual Encoder (DE) based retrieval sy…