58 citations · 156 across the 20 of their papers we have counts for
20 papers
SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning
Bin Wang, Zhengyuan Liu, Xin Huang +4
We present SeaEval, a benchmark for multilingual foundation models. In addition to characterizing how these models understand and reason with natural language, we also investigate…
EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Models
Perry Lam, Huayun Zhang, Nancy F. Chen +1
Neural models are known to be over-parameterized, and recent work has shown that sparse text-to-speech (TTS) models can outperform dense models. Although a plethora of sparse metho…
Taking Quantitative Genomics into the Wild
Susan E. Johnston, Nancy Chen, Emily B. Josephs
A key goal in studies of ecology and evolution is understanding the causes of phenotypic diversity in nature. Most traits of interest, such as those relating to morphology, life-hi…
A Transformer-based Neural Language Model that Synthesizes Brain Activation Maps from Free-Form Text Queries
Gia H. Ngo, Minh Nguyen, Nancy F. Chen +1
Neuroimaging studies are often limited by the number of subjects and cognitive processes that can be feasibly interrogated. However, a rapidly growing number of neuroscientific stu…
Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues
Hung Le, Nancy F. Chen, Steven C. H. Hoi
Compared to traditional visual question answering, video-grounded dialogues require additional reasoning over dialogue context to answer questions in a multi-turn setting. Previous…
N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking
Taha Aksu, Zhengyuan Liu, Min-Yen Kan +1
Augmentation of task-oriented dialogues has followed standard methods used for plain-text such as back-translation, word-level manipulation, and paraphrasing despite its richly ann…
Senone-aware Adversarial Multi-task Training for Unsupervised Child to Adult Speech Adaptation
Richeng Duan, Nancy F. Chen
Acoustic modeling for child speech is challenging due to the high acoustic variability caused by physiological differences in the vocal tract. The dearth of publicly available data…
An End-to-End Document-Level Neural Discourse Parser Exploiting Multi-Granularity Representations
Ke Shi, Zhengyuan Liu, Nancy F. Chen
Document-level discourse parsing, in accordance with the Rhetorical Structure Theory (RST), remains notoriously challenging. Challenges include the deep structure of document-level…
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues
Hung Le, Doyen Sahoo, Nancy F. Chen +1
Video-grounded dialogues are very challenging due to (i) the complexity of videos which contain both spatial and temporal variations, and (ii) the complexity of user utterances whi…
Domain-shift Conditioning using Adaptable Filtering via Hierarchical Embeddings for Robust Chinese Spell Check
Minh Nguyen, Gia H. Ngo, Nancy F. Chen
Spell check is a useful application which processes noisy human-generated text. Spell check for Chinese poses unresolved problems due to the large number of characters, the sparse…
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues
Hung Le, Doyen Sahoo, Chenghao Liu +2
Building an end-to-end conversational agent for multi-domain task-oriented dialogues has been an open challenge for two main reasons. First, tracking dialogue states of multiple do…
Multimodal Transformer with Pointer Network for the DSTC8 AVSD Challenge
Hung Le, Nancy F. Chen
Audio-Visual Scene-Aware Dialog (AVSD) is an extension from Video Question Answering (QA) whereby the dialogue agent is required to generate natural language responses to address u…
Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networks
Minh Nguyen, Gia H. Ngo, Nancy F. Chen
Logographs (Chinese characters) have recursive structures (i.e. hierarchies of sub-units in logographs) that contain phonological and semantic information, as developmental psychol…
Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding
Jiewen Wu, Luis Fernando D'Haro, Nancy F. Chen +2
We propose an architecture to jointly learn word and label embeddings for slot filling in spoken language understanding. The proposed approach encodes labels using a combination of…
Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversations
Zhengyuan Liu, Angela Ng, Sheldon Lee +2
Due to the lack of publicly available resources, conversation summarization has received far less attention than text summarization. As the purpose of conversations is to exchange…
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems
Hung Le, Doyen Sahoo, Nancy F. Chen +1
Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is conducted based on visual and audio aspects of a given video, is significantly more challenging than traditio…
Fast Prototyping a Dialogue Comprehension System for Nurse-Patient Conversations on Symptom Monitoring
Zhengyuan Liu, Hazel Lim, Nur Farah Ain Binte Suhaimi +9
Data for human-human spoken dialogues for research and development are currently very limited in quantity, variety, and sources; such data are even scarcer in healthcare. In this w…
Phonology-Augmented Statistical Framework for Machine Transliteration using Limited Linguistic Resources
Gia H. Ngo, Minh Nguyen, Nancy F. Chen
Transliteration converts words in a source language (e.g., English) into words in a target language (e.g., Vietnamese). This conversion considers the phonological structure of the…
Multimodal neural pronunciation modeling for spoken languages with logographic origin
Minh Nguyen, Gia H. Ngo, Nancy F. Chen
Graphemes of most languages encode pronunciation, though some are more explicit than others. Languages like Spanish have a straightforward mapping between its graphemes and phoneme…
Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and Text
Zhe Wang, Kingsley Kuan, Mathieu Ravaut +13
The YouTube-8M video classification challenge requires teams to classify 0.7 million videos into one or more of 4,716 classes. In this Kaggle competition, we placed in the top 3% o…