21 citations · 77 across the 26 of their papers we have counts for
26 papers
Milo, a Fully Autonomous Indoor/Outdoor Robotic Guide Dog
Florian Golemo, Joanna Wolski, Joel Ruben Antony Moniz +1
Many Blind and Low-Vision (BLV) people rely on guide dogs for moment-to-moment navigation, such as staying on path and avoiding obstacles and pedestrians. However, guide dogs are e…
Anthropogenic Regional Adaptation in Multimodal Vision-Language Model
Samuel Cahyawijaya, Peerat Limkonchotiwat, Tack Hwa Wong +45
While the field of vision-language (VL) has achieved remarkable success in integrating visual and textual information across multiple languages and domains, there is still no dedic…
Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection
Atharva Kulkarni, Yuan Zhang, Joel Ruben Antony Moniz +5
Hallucinations pose a significant obstacle to the reliability and widespread adoption of language models, yet their accurate measurement remains a persistent challenge. While many…
Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia
Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz +89
Southeast Asia (SEA) is a region of extraordinary linguistic and cultural diversity, yet it remains significantly underrepresented in vision-language (VL) research. This often resu…
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages
Holy Lovenia, Rahmad Mahendra, Salsabil Maulana Akbar +58
Southeast Asia (SEA) is a region rich in linguistic diversity and cultural variety, with over 1,300 indigenous languages and a population of 671 million people. However, prevailing…
ReALM: Reference Resolution As Language Modeling
Joel Ruben Antony Moniz, Soundarya Krishnan, Melis Ozyildirim +4
Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds. This context includes both previous turns and…
SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking
Atharva Kulkarni, Bo-Hsiang Tseng, Joel Ruben Antony Moniz +3
In-context learning with Large Language Models (LLMs) has emerged as a promising avenue of research in Dialog State Tracking (DST). However, the best-performing in-context learning…
Can Large Language Models Understand Context?
Yilun Zhu, Joel Ruben Antony Moniz, Shruti Bhargava +6
Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent. Howe…
MARRS: Multimodal Reference Resolution System
Halim Cagri Ates, Shruti Bhargava, Site Li +15
Successfully handling context is essential for any dialog understanding task. This context maybe be conversational (relying on previous user queries or system responses), visual (r…
STEER: Semantic Turn Extension-Expansion Recognition for Voice Assistants
Leon Liyang Zhang, Jiarui Lu, Joel Ruben Antony Moniz +5
In the context of a voice assistant system, steering refers to the phenomenon in which a user issues a follow-up command attempting to direct or clarify a previous turn. We propose…
Intelligent Assistant Language Understanding On Device
Cecilia Aas, Hisham Abdelsalam, Irina Belousova +20
It has recently become feasible to run personal digital assistants on phones and other personal devices. In this paper we describe a design for a natural language understanding sys…
5IDER: Unified Query Rewriting for Steering, Intent Carryover, Disfluencies, Entity Carryover and Repair
Jiarui Lu, Bo-Hsiang Tseng, Joel Ruben Antony Moniz +4
Providing voice assistants the ability to navigate multi-turn conversations is a challenging problem. Handling multi-turn interactions requires the system to understand various con…
On Efficiently Acquiring Annotations for Multilingual Models
Joel Ruben Antony Moniz, Barun Patra, Matthew R. Gormley
When tasked with supporting multiple languages for a given problem, two approaches have arisen: training a model for each language with the annotation budget divided equally among…
MMIU: Dataset for Visual Intent Understanding in Multimodal Assistants
Alkesh Patel, Joel Ruben Antony Moniz, Roman Nguyen +3
In multimodal assistant, where vision is also one of the input modalities, the identification of user intent becomes a challenging task as visual input can influence the outcome. C…
Using Pause Information for More Accurate Entity Recognition
Sahas Dendukuri, Pooja Chitkara, Joel Ruben Antony Moniz +3
Entity tags in human-machine dialog are integral to natural language understanding (NLU) tasks in conversational assistants. However, current systems struggle to accurately parse s…
DEXTER: Deep Encoding of External Knowledge for Named Entity Recognition in Virtual Assistants
Deepak Muralidharan, Joel Ruben Antony Moniz, Weicheng Zhang +6
Named entity recognition (NER) is usually developed and tested on text from well-written sources. However, in intelligent voice assistants, where NER is an important component, inp…
CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues
Bo-Hsiang Tseng, Shruti Bhargava, Jiarui Lu +4
Anaphora and ellipses are two common phenomena in dialogues. Without resolving referring expressions and information omission, dialogue systems may fail to generate consistent and…
Noise Robust Named Entity Understanding for Voice Assistants
Deepak Muralidharan, Joel Ruben Antony Moniz, Sida Gao +21
Named Entity Recognition (NER) and Entity Linking (EL) play an essential role in voice assistant interaction, but are challenging due to the special difficulties associated with sp…
Learning to Relate from Captions and Bounding Boxes
Sarthak Garg, Joel Ruben Antony Moniz, Anshu Aviral +1
In this work, we propose a novel approach that predicts the relationships between various entities in an image in a weakly supervised manner by relying on image captions and object…
LucidDream: Controlled Temporally-Consistent DeepDream on Videos
Joel Ruben Antony Moniz, Eunsu Kang, Barnabás Póczos
In this work, we aim to propose a set of techniques to improve the controllability and aesthetic appeal when DeepDream, which uses a pre-trained neural network to modify images by…
Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces
Barun Patra, Joel Ruben Antony Moniz, Sarthak Garg +2
Recent work on bilingual lexicon induction (BLI) has frequently depended either on aligned bilingual lexicons or on distribution matching, often with an assumption about the isomet…
Compression and Localization in Reinforcement Learning for ATARI Games
Joel Ruben Antony Moniz, Barun Patra, Sarthak Garg
Deep neural networks have become commonplace in the domain of reinforcement learning, but are often expensive in terms of the number of parameters needed. While compressing deep ne…
Unsupervised Depth Estimation, 3D Face Rotation and Replacement
Joel Ruben Antony Moniz, Christopher Beckham, Simon Rajotte +2
We present an unsupervised approach for learning to estimate three dimensional (3D) facial structure from a single image while also predicting 3D viewpoint transformations that mat…
Nested LSTMs
Joel Ruben Antony Moniz, David Krueger
We propose Nested LSTMs (NLSTM), a novel RNN architecture with multiple levels of memory. Nested LSTMs add depth to LSTMs via nesting as opposed to stacking. The value of a memory…