BAT-CLIP: Trimodal Alignment of Brain, Audio and Text

Suhyun Kim, Jinmo Han, Danny Dongyeop Han +9

Decoding and interpreting naturalistic speech from the brain increasingly relies on alignment to pretrained speech and language representation spaces. However, current CLIP-style b…