35 citations · 68 across the 10 of their papers we have counts for
7 papers · 1 filter
Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning
Livia Qian, Gabriel Skantze
Backchannels (e.g., `yeah', `mhm', and `right') are short, non-interruptive feedback signals whose lexical form and prosody jointly convey pragmatic meaning. While prior computatio…
Resolving References in Visually-Grounded Dialogue via Text Generation
Bram Willemsen, Livia Qian, Gabriel Skantze
Vision-language models (VLMs) have shown to be effective at image retrieval based on simple text queries, but text-image retrieval based on conversational input remains a challenge…
How "open" are the conversations with open-domain chatbots? A proposal for Speech Event based evaluation
A. Seza Doğruöz, Gabriel Skantze
Open-domain chatbots are supposed to converse freely with humans without being restricted to a topic, task or domain. However, the boundaries and/or contents of open-domain convers…
CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddings
Gabriel Skantze, Bram Willemsen
This paper presents CoLLIE: a simple, yet effective model for continual learning of how language is grounded in vision. Given a pre-trained multimodal embedding model, where langua…
TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog
Erik Ekstedt, Gabriel Skantze
Syntactic and pragmatic completeness is known to be important for turn-taking prediction, but so far machine learning models of turn-taking have used such linguistic information in…
Multimodal Continuous Turn-Taking Prediction Using Multiscale RNNs
Matthew Roddy, Gabriel Skantze, Naomi Harte
In human conversational interactions, turn-taking exchanges can be coordinated using cues from multiple modalities. To design spoken dialog systems that can conduct fluid interacti…
Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMs
Matthew Roddy, Gabriel Skantze, Naomi Harte
For spoken dialog systems to conduct fluid conversational interactions with users, the systems must be sensitive to turn-taking cues produced by a user. Models should be designed s…