281 citations · 1.6k across the 109 of their papers we have counts for
30 papers · 1 filter
NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks
Fahim Dalvi, Avery Nortonsmith, D. Anthony Bau +4
We present a toolkit to facilitate the interpretation and understanding of neural network models. The toolkit provides several methods to identify salient neurons with respect to t…
What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models
Fahim Dalvi, Nadir Durrani, Hassan Sajjad +3
Despite the remarkable evolution of deep neural networks in natural language processing (NLP), their interpretability remains a challenge. Previous work largely focused on what the…
Domain Attentive Fusion for End-to-end Dialect Identification with Unknown Target Domain
Suwon Shon, Ahmed Ali, James Glass
End-to-end deep learning language or dialect identification systems operate on the spectrogram or other acoustic feature and directly generate identification scores for each class.…
A Study of the Complexity and Accuracy of Direction of Arrival Estimation Methods Based on GCC-PHAT for a Pair of Close Microphones
Francois Grondin, James Glass
This paper investigates the accuracy of various Generalized Cross-Correlation with Phase Transform (GCC-PHAT) methods for a close pair of microphones. We investigate interpolation-…
Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion
Suwon Shon, Tae-Hyun Oh, James Glass
In this paper, we present a multi-modal online person verification system using both speech and visual signals. Inspired by neuroscientific findings on the association of voice and…
On The Inductive Bias of Words in Acoustics-to-Word Models
Hao Tang, James Glass
Acoustics-to-word models are end-to-end speech recognizers that use words as targets without relying on pronunciation dictionaries or graphemes. These models are notoriously diffic…
Towards Unsupervised Speech-to-Text Translation
Yu-An Chung, Wei-Hung Weng, Schrasing Tong +1
We present a framework for building speech-to-text translation (ST) systems using only monolingual speech and text corpora, in other words, speech utterances from a source language…
Identifying and Controlling Important Neurons in Neural Machine Translation
Anthony Bau, Yonatan Belinkov, Hassan Sajjad +3
Neural machine translation (NMT) models learn representations containing substantial linguistic information. However, it is not clear if such information is fully distributed or if…
SVD-PHAT: A Fast Sound Source Localization Method
Francois Grondin, James Glass
This paper introduces a new localization method called SVD-PHAT. The SVD-PHAT method relies on Singular Value Decomposition of the SRP-PHAT projection matrix. A k-d tree is also pr…
Detecting egregious responses in neural sequence-to-sequence models
Tianxing He, James Glass
In this work, we attempt to answer a critical question: whether there exists some input sequence that will cause a well-trained discrete-space neural network sequence-to-sequence (…
Predicting Factuality of Reporting and Bias of News Media Sources
Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov +2
We present a study on predicting the factuality of reporting and bias of news media. While previous work has focused on studying the veracity of claims or documents, here we are in…
Unsupervised Representation Learning of Speech for Dialect Identification
Suwon Shon, Wei-Ning Hsu, James Glass
In this paper, we explore the use of a factorized hierarchical variational autoencoder (FHVAE) model to learn an unsupervised latent representation for dialect identification (DID)…
Frame-level speaker embeddings for text-independent speaker recognition and analysis of end-to-end model
Suwon Shon, Hao Tang, James Glass
In this paper, we propose a Convolutional Neural Network (CNN) based speaker recognition model for extracting robust speaker embeddings. The embedding can be extracted efficiently…
MCE 2018: The 1st Multi-target Speaker Detection and Identification Challenge Evaluation (MCE) Plan, Dataset and Baseline System
Suwon Shon, Najim Dehak, Douglas Reynolds +1
The Multitarget Challenge aims to assess how well current speech technology is able to determine whether or not a recorded utterance was spoken by one of a large number of 'blackli…
On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition
Hao Tang, James Glass
Recurrent neural networks have been the dominant models for many speech and language processing tasks. However, we understand little about the behavior and the class of functions r…
Unsupervised Adaptation with Interpretable Disentangled Representations for Distant Conversational Speech Recognition
Wei-Ning Hsu, Hao Tang, James Glass
The current trend in automatic speech recognition is to leverage large amounts of labeled data to train supervised neural network models. Unfortunately, obtaining data for a wide r…
A Study of Enhancement, Augmentation, and Autoencoder Methods for Domain Adaptation in Distant Speech Recognition
Hao Tang, Wei-Ning Hsu, Francois Grondin +1
Speech recognizers trained on close-talking speech do not generalize to distant speech and the word error rate degradation can be as large as 40% absolute. Most studies focus on ta…
Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data
Wei-Ning Hsu, James Glass
Multimodal sensory data resembles the form of information perceived by humans for learning, and are easy to obtain in large quantities. Compared to unimodal data, synchronization o…
On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference
Adam Poliak, Yonatan Belinkov, James Glass +1
We propose a process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena. We use…
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
Yu-An Chung, Wei-Hung Weng, Schrasing Tong +1
Recent research has shown that word embedding spaces learned from text corpora of different languages can be aligned without any parallel data supervision. Inspired by the success…
Integrating Stance Detection and Fact Checking in a Unified Corpus
Ramy Baly, Mitra Mohtarami, James Glass +3
A reasonable approach for fact checking a claim involves retrieving potentially relevant documents from different sources (e.g., news websites, social media, etc.), determining the…
Convolutional Neural Networks and Language Embeddings for End-to-End Dialect Recognition
Suwon Shon, Ahmed Ali, James Glass
Dialect identification (DID) is a special case of general language identification (LID), but a more challenging problem due to the linguistic similarity between dialects. In this p…
Automatic Stance Detection Using End-to-End Memory Networks
Mitra Mohtarami, Ramy Baly, James Glass +3
We present a novel end-to-end memory network for stance detection, which jointly (i) predicts whether a document agrees, disagrees, discusses or is unrelated with respect to a give…
Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech
David Harwath, Galen Chuang, James Glass
In this paper, we explore the learning of neural network embeddings for natural images and speech waveforms describing the content of those images. These embeddings are learned dir…