Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking

Katia Vendrame, Bolaji Yusuf, Santosh Kesiraju +3

End-to-end spoken dialogue state tracking (DST) is made difficult by the tandem of having to handle speech input and data scarcity. Combining speech foundation encoders and large l…