342 citations · 1.2k across the 26 of their papers we have counts for
29 papers
Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding
Yingting Li, Ambuj Mehrish, Shuai Zhao +5
Fine-tuning is widely used as the default algorithm for transfer learning from pre-trained models. Parameter inefficiency can however arise when, during transfer learning, all the…
Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design computer agents with intelligent capabilities such as understanding, reasoning, and l…
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering
Alex Wilf, Martin Q. Ma, Paul Pu Liang +2
Creating artificial social intelligence - algorithms that can understand the nuances of multi-person interactions - is an exciting and emerging challenge in processing facial expre…
Relay Variational Inference: A Method for Accelerated Encoderless VI
Amir Zadeh, Santiago Benoit, Louis-Philippe Morency
Variational Inference (VI) offers a method for approximating intractable likelihoods. In neural VI, inference of approximate posteriors is commonly done using an encoder. Alternati…
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis
Wei Han, Hui Chen, Alexander Gelbukh +3
Multimodal sentiment analysis aims to extract and integrate semantic information collected from multiple modalities to recognize the expressed emotions and sentiment in multimodal…
M2H2: A Multimodal Multiparty Hindi Dataset For Humor Recognition in Conversations
Dushyant Singh Chauhan, Gopendra Vikram Singh, Navonil Majumder +5
Humor recognition in conversations is a challenging task that has recently gained popularity due to its importance in dialogue understanding, including in multimodal settings (i.e.…
StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations
Amir Zadeh, Santiago Benoit, Louis-Philippe Morency
In this paper we present an approach for training deep generative models solely based on solving determined systems of linear equations. A network that uses this approach, called a…
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences
Jianing Yang, Yongxin Wang, Ruitao Yi +5
Human communication is multimodal in nature; it is through multiple modalities such as language, voice, and facial expressions, that opinions and emotions are expressed. Data in th…
Multimodal Research in Vision and Language: A Review of Current and Emerging Trends
Shagun Uppal, Sarthak Bhagat, Devamanyu Hazarika +4
Deep Learning and its applications have cascaded impactful research and development with a diverse range of modalities present in the real-world data. More recently, this has enhan…
What Gives the Answer Away? Question Answering Bias Analysis on Video QA Datasets
Jianing Yang, Yuying Zhu, Yongxin Wang +3
Question answering biases in video QA datasets can mislead multimodal model to overfit to QA artifacts and jeopardize the model's ability to generalize. Understanding how strong th…
Improving Aspect-Level Sentiment Analysis with Aspect Extraction
Navonil Majumder, Rishabh Bhardwaj, Soujanya Poria +4
Aspect-based sentiment analysis (ABSA), a popular research area in NLP has two distinct parts -- aspect extraction (AE) and labeling the aspects with sentiment polarity (ALSA). Alt…
Pseudo-Encoded Stochastic Variational Inference
Amir Zadeh, Smon Hessner, Yao-Chong Lim +1
Posterior inference in directed graphical models is commonly done using a probabilistic encoder (a.k.a inference model) conditioned on the input. Often this inference model is trai…
Factorized Multimodal Transformer for Multimodal Sequential Learning
Amir Zadeh, Chengfeng Mao, Kelly Shi +4
The complex world around us is inherently multimodal and sequential (continuous). Information is scattered across different modalities and requires multiple continuous sensors to b…
WildMix Dataset and Spectro-Temporal Transformer Model for Monoaural Audio Source Separation
Amir Zadeh, Tianjun Ma, Soujanya Poria +1
Monoaural audio source separation is a challenging research area in machine learning. In this area, a mixture containing multiple audio sources is given, and a model is expected to…
Integrating Multimodal Information in Large Pretrained Transformers
Wasifur Rahman, Md. Kamrul Hasan, Sangwu Lee +4
Recent Transformer-based contextual word representations, including BERT and XLNet, have shown state-of-the-art performance in multiple disciplines within NLP. Fine-tuning the trai…
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor
Md Kamrul Hasan, Wasifur Rahman, Amir Zadeh +5
Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (visi…
Variational Auto-Decoder: A Method for Neural Generative Modeling from Incomplete Data
Amir Zadeh, Yao-Chong Lim, Paul Pu Liang +1
Learning a generative model from partial data (data with missingness) is a challenging area of machine learning research. We study a specific implementation of the Auto-Encoding Va…
Approaching the Gamow window with stored ions: Direct measurement of Xe(p,) in the ESR storage ring
J. Glorius, C. Langer, Z. Slavkovská +63
We report the first measurement of low-energy proton-capture cross sections of Xe in a heavy ion storage ring. Xe ions of five different beam energies betwe…
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Yansen Wang, Ying Shen, Zhun Liu +3
Humans convey their intentions through the usage of both verbal and nonverbal behaviors during face-to-face communication. Speaker intentions often vary dynamically depending on di…
Multimodal Local-Global Ranking Fusion for Emotion Recognition
Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
Emotion recognition is a core research area at the intersection of artificial intelligence and human communication analysis. It is a significant technical challenge since humans di…
Multimodal Language Analysis with Recurrent Multistage Fusion
Paul Pu Liang, Ziyin Liu, Amir Zadeh +1
Computational modeling of human multimodal language is an emerging research area in natural language processing spanning the language, visual and acoustic modalities. Comprehending…
Learning Factorized Multimodal Representations
Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh +2
Learning multimodal representations is a fundamentally complex research problem due to the presence of multiple heterogeneous sources of information. Although the presence of multi…
Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan +3
Multimodal research is an emerging field of artificial intelligence, and one of the main research problems in this field is multimodal fusion. The fusion of multimodal data is the…
Memory Fusion Network for Multi-view Sequential Learning
Amir Zadeh, Paul Pu Liang, Navonil Mazumder +3
Multi-view sequential learning is a fundamental problem in machine learning dealing with multi-view sequences. In a multi-view sequence, there exists two forms of interactions betw…