360 citations · 375 across the 14 of their papers we have counts for
14 papers
Action100M: A Large-scale Video Action Dataset
Delong Chen, Tejaswi Kasarla, Yejin Bang +6
Inferring physical actions from visual observations is a fundamental capability for advancing machine intelligence in the physical world. Achieving this requires large-scale, open-…
VL-JEPA: Joint Embedding Predictive Architecture for Vision-language
Delong Chen, Mustafa Shukor, Theo Moutakanni +7
We introduce VL-JEPA, a vision-language model built on a Joint Embedding Predictive Architecture (JEPA). Instead of autoregressively generating tokens as in classical VLMs, VL-JEPA…
Planning with Reasoning using Vision Language World Model
Delong Chen, Theo Moutakanni, Willy Chung +4
Effective planning requires strong world models, but high-level world models that can understand and reason about actions with semantic and temporal abstraction remain largely unde…
Embodied AI Agents: Modeling the World
Pascale Fung, Yoram Bachrach, Asli Celikyilmaz +18
This paper describes our research on AI agents embodied in visual, virtual or physical forms, enabling them to interact with both users and their environments. These agents, which…
WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning
Delong Chen, Willy Chung, Yejin Bang +2
Humans are known to have an internal "world model" that enables us to carry out action planning based on world states. AI agents need to have such a world model for action planning…
Contrastive Learning for Inference in Dialogue
Etsuko Ishii, Yan Xu, Bryan Wilie +4
Inference, especially those derived from inductive processes, is a crucial component in our conversation to complement the information implicitly or explicitly conveyed by a speake…
InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems
Willy Chung, Samuel Cahyawijaya, Bryan Wilie +2
Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP), yet remain under-explored for task-oriented dialogue systems (TODS), especially…
PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems
Bryan Wilie, Yan Xu, Willy Chung +3
Grounding dialogue response generation on external knowledge is proposed to produce informative and engaging responses. However, current knowledge-grounded dialogue (KGD) systems o…
Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition
Samuel Cahyawijaya, Holy Lovenia, Willy Chung +3
Speech emotion recognition plays a crucial role in human-computer interactions. However, most speech emotion recognition research is biased toward English-speaking adults, which hi…
InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning
Samuel Cahyawijaya, Holy Lovenia, Tiezheng Yu +2
Large language models (LLMs) that are tuned with instructions have demonstrated remarkable capabilities in various tasks and languages. However, their ability to generalize to unde…
Learn What NOT to Learn: Towards Generative Safety in Chatbots
Leila Khalatbari, Yejin Bang, Dan Su +4
Conversational models that are generative and open-domain are particularly susceptible to generating unsafe content since they are trained on web-based social data. Prior approache…
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee +10
This paper proposes a framework for quantitatively evaluating interactive LLMs such as ChatGPT using publicly available data sets. We carry out an extensive technical evaluation of…
Every picture tells a story: Image-grounded controllable stylistic story generation
Holy Lovenia, Bryan Wilie, Romain Barraud +3
Generating a short story out of an image is arduous. Unlike image captioning, story generation from an image poses multiple challenges: preserving the story coherence, appropriatel…
Clozer: Adaptable Data Augmentation for Cloze-style Reading Comprehension
Holy Lovenia, Bryan Wilie, Willy Chung +4
Task-adaptive pre-training (TAPT) alleviates the lack of labelled data and provides performance lift by adapting unlabelled data to downstream task. Unfortunately, existing adaptat…