102 citations · 327 across the 27 of their papers we have counts for
30 papers
From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognition
Colin Lea, Zifang Huang, Lauren Tooley +6
Consumer speech recognition systems do not work as well for many people with speech diferences, such as stuttering, relative to the rest of the general population. However, what is…
CALM: Contrastive Aligned Audio-Language Multirate and Multimodal Representations
Vin Sachidananda, Shao-Yen Tseng, Erik Marchi +2
Deriving multimodal representations of audio and lexical inputs is a central problem in Natural Language Understanding (NLU). In this paper, we present Contrastive Aligned Audio-La…
Analysis and Tuning of a Voice Assistant System for Dysfluent Speech
Vikramjit Mitra, Zifang Huang, Colin Lea +8
Dysfluencies and variations in speech pronunciation can severely degrade speech recognition performance, and for many individuals with moderate-to-severe speech disorders, voice op…
Unsupervised Speech Representation Learning for Behavior Modeling using Triplet Enhanced Contextualized Networks
Haoqi Li, Brian Baucom, Shrikanth Narayanan +1
Speech encodes a wealth of information related to human behavior and has been used in a variety of automated behavior recognition tasks. However, extracting behavioral information…
Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies
Nikolaos Flemotomos, Victor R. Martinez, Zhuohao Chen +13
With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care to assist in training, supervision, and…
Confusion2vec 2.0: Enriching Ambiguous Spoken Language Representations with Subwords
Prashanth Gurunath Shivakumar, Panayiotis Georgiou, Shrikanth Narayanan
Word vector representations enable machines to encode human language for spoken language understanding and processing. Confusion2vec, motivated from human speech production and per…
Speaker Diarization with Lexical Information
Tae Jin Park, Kyu J. Han, Jing Huang +4
This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition. We propose a speaker diarization system that c…
Automatic prediction of suicidal risk in military couples using multimodal interaction cues from couples conversations
Sandeep Nallan Chakravarthula, Md Nasir, Shao-Yen Tseng +6
Suicide is a major societal challenge globally, with a wide range of risk factors, from individual health, psychological and behavioral elements to socio-economic aspects. Military…
An analysis of observation length requirements for machine understanding of human behaviors from spoken language
Sandeep Nallan Chakravarthula, Brian Baucom, Shrikanth Narayanan +1
The task of quantifying human behavior by observing interaction cues is an important and useful one across a range of domains in psychological research and practice. Machine learni…
Linguistically Aided Speaker Diarization Using Speaker Role Information
Nikolaos Flemotomos, Panayiotis Georgiou, Shrikanth Narayanan
Speaker diarization relies on the assumption that speech segments corresponding to a particular speaker are concentrated in a specific region of the speaker space; a region which r…
Speaker-invariant Affective Representation Learning via Adversarial Training
Haoqi Li, Ming Tu, Jing Huang +2
Representation learning for speech emotion recognition is challenging due to labeled data sparsity issue and lack of gold standard references. In addition, there is much variabilit…
Linking emotions to behaviors through deep transfer learning
Haoqi Li, Brian Baucom, Panayiotis Georgiou
Human behavior refers to the way humans act and interact. Understanding human behavior is a cornerstone of observational practice, especially in psychotherapy. An important cue of…
RNN based Incremental Online Spoken Language Understanding
Prashanth Gurunath Shivakumar, Naveen Kumar, Panayiotis Georgiou +1
Spoken Language Understanding (SLU) typically comprises of an automatic speech recognition (ASR) followed by a natural language understanding (NLU) module. The two modules process…
Multimodal Embeddings from Language Models
Shao-Yen Tseng, Panayiotis Georgiou, Shrikanth Narayanan
Word embeddings such as ELMo have recently been shown to model word semantics with greater efficacy through contextualized learning on large-scale language corpora, resulting in si…
Behavior Gated Language Models
Prashanth Gurunath Shivakumar, Shao-Yen Tseng, Panayiotis Georgiou +1
Most current language modeling techniques only exploit co-occurrence, semantic and syntactic information from the sequence of words. However, a range of information such as the sta…
Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions using Speech and Language
Sandeep Nallan Chakravarthula, Haoqi Li, Shao-Yen Tseng +2
Cancer impacts the quality of life of those diagnosed as well as their spouse caregivers, in addition to potentially influencing their day-to-day behaviors. There is evidence that…
Modeling Interpersonal Linguistic Coordination in Conversations using Word Mover's Distance
Md Nasir, Sandeep Nallan Chakravarthula, Brian Baucom +3
Linguistic coordination is a well-established phenomenon in spoken conversations and often associated with positive social behaviors and outcomes. While there have been many attemp…
Spoken Language Intent Detection using Confusion2Vec
Prashanth Gurunath Shivakumar, Mu Yang, Panayiotis Georgiou
Decoding speaker's intent is a crucial part of spoken language understanding (SLU). The presence of noise or errors in the text transcriptions, in real life scenarios make the task…
Speaker Diarization With Lexical Information
Tae Jin Park, Kyu Han, Ian Lane +1
This work presents a novel approach to leverage lexical information for speaker diarization. We introduce a speaker diarization system that can directly integrate lexical as well a…
Multi-label Multi-task Deep Learning for Behavioral Coding
James Gibson, David C. Atkins, Torrey Creed +3
We propose a methodology for estimating human behaviors in psychotherapy sessions using mutli-label and multi-task learning paradigms. We discuss the problem of behavioral coding i…
Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities
Prashanth Gurunath Shivakumar, Panayiotis Georgiou
Word vector representations are a crucial part of Natural Language Processing (NLP) and Human Computer Interaction. In this paper, we propose a novel word vector representation, Co…
Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings
Shao-Yen Tseng, Brian Baucom, Panayiotis Georgiou
Unsupervised learning has been an attractive method for easily deriving meaningful data representations from vast amounts of unlabeled data. These representations, or embeddings, o…
Multimodal Speaker Segmentation and Diarization using Lexical and Acoustic Cues via Sequence to Sequence Neural Networks
Tae Jin Park, Panayiotis Georgiou
While there has been substantial amount of work in speaker diarization recently, there are few efforts in jointly employing lexical and acoustic information for speaker segmentatio…
Modeling Interpersonal Influence of Verbal Behavior in Couples Therapy Dyadic Interactions
Sandeep Nallan Chakravarthula, Brian Baucom, Panayiotis Georgiou
Dyadic interactions among humans are marked by speakers continuously influencing and reacting to each other in terms of responses and behaviors, among others. Understanding how int…