Cross-stitched Multi-modal Encoders

Karan Singla, Daniel Pressel, Ryan Price +3

In this paper, we propose a novel architecture for multi-modal speech and text input. We combine pretrained speech and text encoders using multi-headed cross-modal attention and jo…