200 citations · 407 across the 22 of their papers we have counts for
26 papers
MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment
Hanxian Huang, Igor Fedorov, Andrey Gromov +14
Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The most useful OD-LLMs produce nea…
Short Data, Long Context: Distilling Positional Knowledge in Transformers
Patrick Huber, Ernie Chang, Chinnadhurai Sankar +4
Extending the context window of language models typically requires expensive long-context pre-training, posing significant challenges for both training efficiency and data collecti…
MobileLLM-Pro Technical Report
Patrick Huber, Ernie Chang, Wei Wen +16
Efficient on-device language models around 1 billion parameters are essential for powering low-latency AI applications on mobile and wearable devices. However, achieving strong per…
CoSMoEs: Compact Sparse Mixture of Experts
Patrick Huber, Akshat Shrivastava, Ernie Chang +3
Sparse Mixture of Expert (MoE) models are popular foundational architectures at large scale, however, under-explored at smaller sizes. Here, we show how to enable Compact Sparse Mi…
Step by Step to Fairness: Attributing Societal Bias in Task-oriented Dialogue Systems
Hsuan Su, Rebecca Qian, Chinnadhurai Sankar +4
Recent works have shown considerable improvements in task-oriented dialogue (TOD) systems by utilizing pretrained large language models (LLMs) in an end-to-end manner. However, the…
Continual Dialogue State Tracking via Example-Guided Question Answering
Hyundong Cho, Andrea Madotto, Zhaojiang Lin +5
Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services in diminishing performance on prev…
AUTODIAL: Efficient Asynchronous Task-Oriented Dialogue Model
Prajjwal Bhargava, Pooyan Amini, Shahin Shayandeh +1
As large dialogue models become commonplace in practice, the problems surrounding high compute requirements for training, inference and larger memory footprint still persists. In t…
Data-Efficiency with a Single GPU: An Exploration of Transfer Methods for Small Language Models
Alon Albalak, Akshat Shrivastava, Chinnadhurai Sankar +2
Multi-task learning (MTL), instruction tuning, and prompting have recently been shown to improve the generalizability of large language models to new tasks. However, the benefits o…
KETOD: Knowledge-Enriched Task-Oriented Dialogue
Zhiyu Chen, Bing Liu, Seungwhan Moon +3
Existing studies in dialogue system research mostly treat task-oriented dialogue and chit-chat as separate domains. Towards building a human-like assistant that can converse natura…
Database Search Results Disambiguation for Task-Oriented Dialog Systems
Kun Qian, Ahmad Beirami, Satwik Kottur +5
As task-oriented dialog systems are becoming increasingly popular in our lives, more realistic tasks have been proposed and explored. However, new practical challenges arise. For i…
Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics
Hyundong Cho, Chinnadhurai Sankar, Christopher Lin +5
Recent works that revealed the vulnerability of dialogue state tracking (DST) models to distributional shifts have made holistic comparisons on robustness and qualitative analyses…
Robustness through Data Augmentation Loss Consistency
Tianjian Huang, Shaunak Halbe, Chinnadhurai Sankar +5
While deep learning through empirical risk minimization (ERM) has succeeded at achieving human-level performance at a variety of complex tasks, ERM is not robust to distribution sh…
Annotation Inconsistency and Entity Bias in MultiWOZ
Kun Qian, Ahmad Beirami, Zhouhan Lin +4
MultiWOZ is one of the most popular multi-domain task-oriented dialog datasets, containing 10K+ annotated dialogs covering eight domains. It has been widely accepted as a benchmark…
DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue
Hung Le, Chinnadhurai Sankar, Seungwhan Moon +3
A video-grounded dialogue system is required to understand both dialogue, which contains semantic dependencies from turn to turn, and video, which contains visual cues of spatial a…
ProFormer: Towards On-Device LSH Projection Based Transformers
Chinnadhurai Sankar, Sujith Ravi, Zornitsa Kozareva
At the heart of text based neural models lay word representations, which are powerful but occupy a lot of memory making it challenging to deploy to devices with memory constraints…
Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning
Arvind Neelakantan, Semih Yavuz, Sharan Narang +5
Task-oriented dialog presents a difficult challenge encompassing multiple problems including multi-turn language understanding and generation, knowledge retrieval and reasoning, an…
Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset
Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar +7
A significant barrier to progress in data-driven approaches to building dialog systems is the lack of high quality, goal-oriented conversational data. To help satisfy this elementa…
On-Device Text Representations Robust To Misspellings via Projections
Chinnadhurai Sankar, Sujith Ravi, Zornitsa Kozareva
Recently, there has been a strong interest in developing natural language applications that live on personal devices such as mobile phones, watches and IoT with the objective to pr…
Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study
Chinnadhurai Sankar, Sandeep Subramanian, Christopher Pal +2
Neural generative models have been become increasingly popular when building conversational agents. They offer flexibility, can be easily adapted to new domains, and require minima…
Deep Reinforcement Learning For Modeling Chit-Chat Dialog With Discrete Attributes
Chinnadhurai Sankar, Sujith Ravi
Open domain dialog systems face the challenge of being repetitive and producing generic responses. In this paper, we demonstrate that by conditioning the response generation on int…
Transferable Neural Projection Representations
Chinnadhurai Sankar, Sujith Ravi, Zornitsa Kozareva
Neural word representations are at the core of many state-of-the-art natural language processing models. A widely used approach is to pre-train, store and look up word or character…
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
Sarath Chandar, Chinnadhurai Sankar, Eugene Vorontsov +2
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training, as the sequence length incre…
The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach
Iulian Vlad Serban, Chinnadhurai Sankar, Michael Pieper +2
Deep reinforcement learning has recently shown many impressive successes. However, one major obstacle towards applying such methods to real-world problems is their lack of data-eff…
A Deep Reinforcement Learning Chatbot (Short Version)
Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain +15
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capa…