342 citations · 1.2k across the 26 of their papers we have counts for
14 papers · 1 filter
Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design computer agents with intelligent capabilities such as understanding, reasoning, and l…
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering
Alex Wilf, Martin Q. Ma, Paul Pu Liang +2
Creating artificial social intelligence - algorithms that can understand the nuances of multi-person interactions - is an exciting and emerging challenge in processing facial expre…
Relay Variational Inference: A Method for Accelerated Encoderless VI
Amir Zadeh, Santiago Benoit, Louis-Philippe Morency
Variational Inference (VI) offers a method for approximating intractable likelihoods. In neural VI, inference of approximate posteriors is commonly done using an encoder. Alternati…
StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations
Amir Zadeh, Santiago Benoit, Louis-Philippe Morency
In this paper we present an approach for training deep generative models solely based on solving determined systems of linear equations. A network that uses this approach, called a…
Pseudo-Encoded Stochastic Variational Inference
Amir Zadeh, Smon Hessner, Yao-Chong Lim +1
Posterior inference in directed graphical models is commonly done using a probabilistic encoder (a.k.a inference model) conditioned on the input. Often this inference model is trai…
Factorized Multimodal Transformer for Multimodal Sequential Learning
Amir Zadeh, Chengfeng Mao, Kelly Shi +4
The complex world around us is inherently multimodal and sequential (continuous). Information is scattered across different modalities and requires multiple continuous sensors to b…
WildMix Dataset and Spectro-Temporal Transformer Model for Monoaural Audio Source Separation
Amir Zadeh, Tianjun Ma, Soujanya Poria +1
Monoaural audio source separation is a challenging research area in machine learning. In this area, a mixture containing multiple audio sources is given, and a model is expected to…
Integrating Multimodal Information in Large Pretrained Transformers
Wasifur Rahman, Md. Kamrul Hasan, Sangwu Lee +4
Recent Transformer-based contextual word representations, including BERT and XLNet, have shown state-of-the-art performance in multiple disciplines within NLP. Fine-tuning the trai…
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor
Md Kamrul Hasan, Wasifur Rahman, Amir Zadeh +5
Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (visi…
Variational Auto-Decoder: A Method for Neural Generative Modeling from Incomplete Data
Amir Zadeh, Yao-Chong Lim, Paul Pu Liang +1
Learning a generative model from partial data (data with missingness) is a challenging area of machine learning research. We study a specific implementation of the Auto-Encoding Va…
Multimodal Language Analysis with Recurrent Multistage Fusion
Paul Pu Liang, Ziyin Liu, Amir Zadeh +1
Computational modeling of human multimodal language is an emerging research area in natural language processing spanning the language, visual and acoustic modalities. Comprehending…
Learning Factorized Multimodal Representations
Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh +2
Learning multimodal representations is a fundamentally complex research problem due to the presence of multiple heterogeneous sources of information. Although the presence of multi…
Memory Fusion Network for Multi-view Sequential Learning
Amir Zadeh, Paul Pu Liang, Navonil Mazumder +3
Multi-view sequential learning is a fundamental problem in machine learning dealing with multi-view sequences. In a multi-view sequence, there exists two forms of interactions betw…
Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning
Minghai Chen, Sen Wang, Paul Pu Liang +3
With the increasing popularity of video sharing websites such as YouTube and Facebook, multimodal sentiment analysis has received increasing attention from the scientific community…