234 citations · 440 across the 31 of their papers we have counts for
34 papers
Self-Supervised Object Goal Navigation with In-Situ Finetuning
So Yeon Min, Yao-Hung Hubert Tsai, Wei Ding +4
A household robot should be able to navigate to target objects without requiring users to first annotate everything in their home. Most current approaches to object navigation do n…
Greedy Modality Selection via Approximate Submodular Maximization
Runxiang Cheng, Gargi Balasubramaniam, Yifei He +2
Multimodal learning considers learning from multi-modality data, aiming to fuse heterogeneous sources of information. However, it is not always feasible to leverage all available m…
Towards Multimodal Multitask Scene Understanding Models for Indoor Mobile Agents
Yao-Hung Hubert Tsai, Hanlin Goh, Ali Farhadi +1
The perception system in personalized mobile agents requires developing indoor scene understanding models, which can understand 3D geometries, capture objectiveness, analyze human…
Paraphrasing Is All You Need for Novel Object Captioning
Cheng-Fu Yang, Yao-Hung Hubert Tsai, Wan-Cyuan Fan +3
Novel object captioning (NOC) aims to describe images containing objects without observing their ground truth captions during training. Due to the absence of caption annotation, ca…
Scale dependant layer for self-supervised nuclei encoding
Peter Naylor, Yao-Hung Hubert Tsai, Marick Laé +1
Recent developments in self-supervised learning give us the possibility to further reduce human intervention in multi-step pipelines where the focus evolves around particular objec…
Conditional Contrastive Learning with Kernel
Yao-Hung Hubert Tsai, Tianqin Li, Martin Q. Ma +4
Conditional contrastive learning frameworks consider the conditional sampling procedure that constructs positive or negative data pairs conditioned on specific variables. Fair cont…
Learning Weakly-Supervised Contrastive Representations
Yao-Hung Hubert Tsai, Tianqin Li, Weixin Liu +3
We argue that a form of the valuable information provided by the auxiliary information is its implied data clustering information. For instance, considering hashtags as auxiliary i…
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai +3
Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no l…
Integrating Auxiliary Information in Self-supervised Learning
Yao-Hung Hubert Tsai, Tianqin Li, Weixin Liu +3
This paper presents to integrate the auxiliary information (e.g., additional attributes for data such as the hashtags for Instagram images) in the self-supervised learning process.…
Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning
Martin Q. Ma, Yao-Hung Hubert Tsai, Paul Pu Liang +4
Contrastive self-supervised learning (SSL) learns an embedding space that maps similar data pairs closer and dissimilar data pairs farther apart. Despite its success, one issue has…
A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning
Yao-Hung Hubert Tsai, Shaojie Bai, Louis-Philippe Morency +1
In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approa…
Self-supervised Representation Learning with Relative Predictive Coding
Yao-Hung Hubert Tsai, Martin Q. Ma, Muqiao Yang +3
This paper introduces Relative Predictive Coding (RPC), a new contrastive representation learning objective that maintains a good balance among training stability, minibatch size s…
Unsupervised Domain Adaptation for Visual Navigation
Shangda Li, Devendra Singh Chaplot, Yao-Hung Hubert Tsai +3
Advances in visual navigation methods have led to intelligent embodied navigation agents capable of learning meaningful representations from raw RGB images and perform a wide varie…
Self-supervised Learning from a Multi-view Perspective
Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov +1
As a subset of unsupervised representation learning, self-supervised representation learning adopts self-defined signals as supervision and uses the learned representation for down…
Neural Methods for Point-wise Dependency Estimation
Yao-Hung Hubert Tsai, Han Zhao, Makoto Yamada +2
Since its inception, the neural estimation of mutual information (MI) has demonstrated the empirical success of modeling expected dependency between high-dimensional random variabl…
Feature Robust Optimal Transport for High-dimensional Data
Mathis Petrovich, Chao Liang, Ryoma Sato +6
Optimal transport is a machine learning problem with applications including distribution comparison, feature selection, and generative adversarial networks. In this paper, we propo…
Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis
Yao-Hung Hubert Tsai, Martin Q. Ma, Muqiao Yang +2
The human language can be expressed through multiple sources of information known as modalities, including tones of voice, facial gestures, and spoken language. Recent multimodal l…
Capsules with Inverted Dot-Product Attention Routing
Yao-Hung Hubert Tsai, Nitish Srivastava, Hanlin Goh +1
We introduce a new routing algorithm for capsule networks, in which a child capsule is routed to a parent based only on agreement between the parent's state and the child's vote. T…
Complex Transformer: A Framework for Modeling Complex-Valued Sequence
Muqiao Yang, Martin Q. Ma, Dongyu Li +2
While deep learning has received a surge of interest in a variety of fields in recent years, major deep learning models barely use complex numbers. However, speech, signal and audi…
LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport
Yanbin Liu, Makoto Yamada, Yao-Hung Hubert Tsai +3
Estimating mutual information is an important statistics and machine learning problem. To estimate the mutual information from data, a common practice is preparing a set of paired…
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel
Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada +2
Transformer is a powerful architecture that achieves superior performance on various sequence learning tasks, including neural machine translation, language understanding, and sequ…
Learning Neural Networks with Adaptive Regularization
Han Zhao, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov +1
Feed-forward neural networks can be understood as a combination of an intermediate representation and a linear hypothesis. While most previous works aim to diversify the representa…
Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
Paul Pu Liang, Zhun Liu, Yao-Hung Hubert Tsai +3
There has been an increased interest in multimodal language processing including multimodal dialog, question answering, sentiment analysis, and speech recognition. However, natural…
Strong and Simple Baselines for Multimodal Utterance Embeddings
Paul Pu Liang, Yao Chong Lim, Yao-Hung Hubert Tsai +2
Human language is a rich multimodal signal consisting of spoken words, facial expressions, body gestures, and vocal intonations. Learning representations for these spoken utterance…