Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Model

Hung-Chieh Fang, Nai-Xuan Ye, Yi-Jen Shih +5

Recent advances in self-supervised speech models have shown significant improvement in many downstream tasks. However, these models predominantly centered on frame-level training o…