92 citations · 523 across the 22 of their papers we have counts for
26 papers
Online Continual Learning of End-to-End Speech Recognition Models
Muqiao Yang, Ian Lane, Shinji Watanabe
Continual Learning, also known as Lifelong Learning, aims to continually learn from new data as it becomes available. While prior research on continual learning in automatic speech…
Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding
Yifan Peng, Siddharth Dalmia, Ian Lane +1
Conformer has proven to be effective in many speech processing tasks. It combines the benefits of extracting local dependencies using convolutions and global dependencies using sel…
Identifying Actions for Sound Event Classification
Benjamin Elizalde, Radu Revutchi, Samarjit Das +3
In Psychology, actions are paramount for humans to identify sound events. In Machine Learning (ML), action recognition achieves high accuracy; however, it has not been asked whethe…
Learning Question-Guided Video Representation for Multi-Turn Video Question Answering
Guan-Lin Chao, Abhinav Rastogi, Semih Yavuz +3
Understanding and conversing about dynamic scenes is one of the key capabilities of AI agents that navigate the environment and convey useful information to humans. Video question…
BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer
Guan-Lin Chao, Ian Lane
An important yet rarely tackled problem in dialogue state tracking (DST) is scalability for dynamic ontology (e.g., movie, restaurant) and unseen slot values. We focus on a specifi…
Speaker-Targeted Audio-Visual Models for Speech Recognition in Cocktail-Party Environments
Guan-Lin Chao, William Chan, Ian Lane
Speech recognition in cocktail-party environments remains a significant challenge for state-of-the-art speech recognition systems, as it is extremely difficult to extract an acoust…
Speaker Diarization With Lexical Information
Tae Jin Park, Kyu Han, Ian Lane +1
This work presents a novel approach to leverage lexical information for speaker diarization. We introduce a speaker diarization system that can directly integrate lexical as well a…
Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention
Ming Zeng, Haoxiang Gao, Tong Yu +4
Deep neural networks, including recurrent networks, have been successfully applied to human activity recognition. Unfortunately, the final representation learned by recurrent netwo…
Adversarial Learning of Task-Oriented Neural Dialog Models
Bing Liu, Ian Lane
In this work, we propose an adversarial learning method for reward estimation in reinforcement learning (RL) based task-oriented dialog models. Most of the current RL based task-or…
Semi-Supervised Convolutional Neural Networks for Human Activity Recognition
Ming Zeng, Tong Yu, Xiao Wang +3
Labeled data used for training activity recognition classifiers are usually limited in terms of size and diversity. Thus, the learned model may not generalize well when used in rea…
The CAPIO 2017 Conversational Speech Recognition System
Kyu J. Han, Akshay Chandrashekaran, Jungsuk Kim +1
In this paper we show how we have achieved the state-of-the-art performance on the industry-standard NIST 2000 Hub5 English evaluation set. We explore densely connected LSTMs, insp…
Multi-Domain Adversarial Learning for Slot Filling in Spoken Language Understanding
Bing Liu, Ian Lane
The goal of this paper is to learn cross-domain representations for slot filling task in spoken language understanding (SLU). Most of the recently published SLU models are domain-s…
Customized Nonlinear Bandits for Online Response Selection in Neural Conversation Models
Bing Liu, Tong Yu, Ian Lane +1
Dialog response selection is an important step towards natural response generation in conversational agents. Existing work on neural conversational models mainly focuses on offline…
Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models
Bing Liu, Ian Lane
In this paper, we present a deep reinforcement learning (RL) framework for iterative dialog policy optimization in end-to-end task-oriented dialog systems. Popular approaches in le…
An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialog
Bing Liu, Ian Lane
We present a novel end-to-end trainable neural network model for task-oriented dialog systems. The model is able to track dialog state, issue API calls to knowledge base (KB), and…
Dialog Context Language Modeling with Recurrent Neural Networks
Bing Liu, Ian Lane
In this work, we propose contextual language models that incorporate dialog level discourse information into language modeling. Previous works on contextual language model treat pr…
Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networks
Bing Liu, Ian Lane
Speaker intent detection and semantic slot filling are two critical tasks in spoken language understanding (SLU) for dialogue systems. In this paper, we describe a recurrent neural…
Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
Bing Liu, Ian Lane
Attention-based encoder-decoder neural network models have recently shown promising results in machine translation and speech recognition. In this work, we propose an attention-bas…
An Approach for Self-Training Audio Event Detectors Using Web Data
Benjamin Elizalde, Ankit Shah, Siddharth Dalmia +5
Audio Event Detection (AED) aims to recognize sounds within audio and video recordings. AED employs machine learning algorithms commonly trained and tested on annotated datasets. H…
Experiments on the DCASE Challenge 2016: Acoustic Scene Classification and Sound Event Detection in Real Life Recording
Benjamin Elizalde, Anurag Kumar, Ankit Shah +4
In this paper we present our work on Task 1 Acoustic Scene Classi- fication and Task 3 Sound Event Detection in Real Life Recordings. Among our experiments we have low-level and hi…
City-Identification of Flickr Videos Using Semantic Acoustic Features
Benjamin Elizalde, Guan-Lin Chao, Ming Zeng +1
City-identification of videos aims to determine the likelihood of a video belonging to a set of cities. In this paper, we present an approach using only audio, thus we do not use a…
AudioPairBank: Towards A Large-Scale Tag-Pair-Based Audio Content Analysis
Sebastian Sager, Benjamin Elizalde, Damian Borth +3
Recently, sound recognition has been used to identify sounds, such as car and river. However, sounds have nuances that may be better described by adjective-noun pairs such as slow…
Environmental Noise Embeddings for Robust Speech Recognition
Suyoun Kim, Bhiksha Raj, Ian Lane
We propose a novel deep neural network architecture for speech recognition that explicitly employs knowledge of the background environmental noise within a deep neural network acou…
Recurrent Models for Auditory Attention in Multi-Microphone Distance Speech Recognition
Suyoun Kim, Ian Lane
Integration of multiple microphone data is one of the key ways to achieve robust speech recognition in noisy environments or when the speaker is located at some distance from the i…