89 citations · 129 across the 22 of their papers we have counts for
23 papers
Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs
Houman Mehrafarin, Arash Eshghi, Ioannis Konstas
Evaluating Large Language Models (LLMs) on reasoning benchmarks demonstrates their ability to solve compositional questions. However, little is known of whether these models engage…
Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models
Javier Chiyah-Garcia, Alessandro Suglia, Arash Eshghi
In dialogue, the addressee may initially misunderstand the speaker and respond erroneously, often prompting the speaker to correct the misunderstanding in the next turn with a Thir…
Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modeling
Georgios Pantazopoulos, Malvina Nikandrou, Alessandro Suglia +2
This study explores replacing Transformers in Visual Language Models (VLMs) with Mamba, a recent structured state space model (SSM) that demonstrates promising performance in seque…
AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding
Alessandro Suglia, Claudio Greco, Katie Baker +5
AI personal assistants deployed via robots or wearables require embodied understanding to collaborate with humans effectively. However, current Vision-Language Models (VLMs) primar…
Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplers
Georgios Pantazopoulos, Alessandro Suglia, Oliver Lemon +1
An effective method for combining frozen large language models (LLM) and visual encoders involves a resampler module that creates a `visual prompt' which is provided to the LLM, al…
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion
Georgios Pantazopoulos, Malvina Nikandrou, Amit Parekh +6
Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and ob…
Learning to generate and corr- uh I mean repair language in real-time
Arash Eshghi, Arash Ashrafzadeh
In conversation, speakers produce language incrementally, word by word, while continuously monitoring the appropriateness of their own contribution in the dynamically unfolding con…
No that's not what I meant: Handling Third Position Repair in Conversational Question Answering
Vevake Balaraman, Arash Eshghi, Ioannis Konstas +1
The ability to handle miscommunication is crucial to robust and faithful conversational AI. People usually deal with miscommunication immediately as they detect it, using highly sy…
'What are you referring to?' Evaluating the Ability of Multi-Modal Dialogue Models to Process Clarificational Exchanges
Javier Chiyah-Garcia, Alessandro Suglia, Arash Eshghi +1
Referential ambiguities arise in dialogue when a referring expression does not uniquely identify the intended referent for the addressee. Addressees usually detect such ambiguities…
The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering
Sabrina Chiesurin, Dimitris Dimakopoulos, Marco Antonio Sobrevilla Cabezudo +4
Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. "unfaithful" with respect to a rationale as retrieved from a kno…
Exploring Multi-Modal Representations for Ambiguity Detection & Coreference Resolution in the SIMMC 2.0 Challenge
Javier Chiyah-Garcia, Alessandro Suglia, José Lopes +2
Anaphoric expressions, such as pronouns and referential descriptions, are situated with respect to the linguistic context of prior turns, as well as, the immediate visual environme…
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations
Miruna Clinciu, Arash Eshghi, Helen Hastie
As transparency becomes key for robotics and AI, it will be necessary to evaluate the methods through which transparency is provided, including automatically generated natural lang…
Data-Efficient Goal-Oriented Conversation with Dialogue Knowledge Transfer Networks
Igor Shalyminov, Sungjin Lee, Arash Eshghi +1
Goal-oriented dialogue systems are now being widely adopted in industry where it is of key importance to maintain a rapid prototyping cycle for new products and domains. Data-drive…
Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia
Angus Addlesee, Arash Eshghi, Ioannis Konstas
Dialogue technologies such as Amazon's Alexa have the potential to transform the healthcare industry. However, current systems are not yet naturally interactive: they are often tur…
Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach
Igor Shalyminov, Sungjin Lee, Arash Eshghi +1
Learning with minimal data is one of the key challenges in the development of practical, production-ready goal-oriented dialogue systems. In a real-world enterprise setting where d…
Benchmarking Natural Language Understanding Services for building Conversational Agents
Xingkun Liu, Arash Eshghi, Pawel Swietojanski +1
We have recently seen the emergence of several publicly available Natural Language Understanding (NLU) toolkits, which map user utterances to structured, but more abstract, Dialogu…
Multi-Task Learning for Domain-General Spoken Disfluency Detection in Dialogue Systems
Igor Shalyminov, Arash Eshghi, Oliver Lemon
Spontaneous spoken dialogue is often disfluent, containing pauses, hesitations, self-corrections and false starts. Processing such phenomena is essential in understanding a speaker…
The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings
Yanchao Yu, Arash Eshghi, Gregory Mills +1
We motivate and describe a new freely available human-human dialogue dataset for interactive learning of visually grounded word meanings through ostensive definition by a tutor to…
Training an adaptive dialogue policy for interactive learning of visually grounded word meanings
Yanchao Yu, Arash Eshghi, Oliver Lemon
We present a multi-modal dialogue system for interactive learning of perceptually grounded word meanings from a human tutor. The system integrates an incremental, semantic parsing/…
Learning how to learn: an adaptive dialogue agent for incrementally learning visually grounded word meanings
Yanchao Yu, Arash Eshghi, Oliver Lemon
We present an optimised multi-modal dialogue agent for interactive learning of visually grounded word meanings from a human tutor, trained on real human-human tutoring data. Within…
Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars
Arash Eshghi, Igor Shalyminov, Oliver Lemon
We investigate an end-to-end method for automatically inducing task-based dialogue systems from small amounts of unannotated dialogue data. It combines an incremental semantic gram…
Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena
Igor Shalyminov, Arash Eshghi, Oliver Lemon
Natural, spontaneous dialogue proceeds incrementally on a word-by-word basis; and it contains many sorts of disfluency such as mid-utterance/sentence hesitations, interruptions, an…
Bootstrapping incremental dialogue systems: using linguistic knowledge to learn from minimal data
Dimitrios Kalatzis, Arash Eshghi, Oliver Lemon
We present a method for inducing new dialogue systems from very small amounts of unannotated dialogue data, showing how word-level exploration using Reinforcement Learning (RL), co…