46 citations · 113 across the 19 of their papers we have counts for
19 papers
Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training
Yao Wei, Yanchao Sun, Ruijie Zheng +7
We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dep…
TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning
Ruijie Zheng, Xiyao Wang, Yanchao Sun +5
Despite recent progress in reinforcement learning (RL) from raw pixel data, sample inefficiency continues to present a substantial obstacle. Prior works have attempted to address t…
SMART: Self-supervised Multi-task pretrAining with contRol Transformers
Yanchao Sun, Shuang Ma, Ratnesh Madaan +3
Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining repr…
Masked Autoencoders for Egocentric Video Understanding @ Ego4D Challenge 2022
Jiachen Lei, Shuang Ma, Zhongjie Ba +3
In this report, we present our approach and empirical results of applying masked autoencoders in two egocentric video understanding tasks, namely, Object State Change Classificatio…
PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training
Rogerio Bonatti, Sai Vemprala, Shuang Ma +3
Robotics has long been a field riddled with complex systems architectures whose modules and connections, whether traditional or learning-based, require significant human expertise…
LATTE: LAnguage Trajectory TransformEr
Arthur Bucker, Luis Figueredo, Sami Haddadin +4
Natural language is one of the most intuitive ways to express human intent. However, translating instructions and commands towards robotic motion generation and deployment in the r…
Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers
Arthur Bucker, Luis Figueredo, Sami Haddadin +3
Natural language is the most intuitive medium for us to interact with other people when expressing commands and instructions. However, using language is seldom an easy task when hu…
COMPASS: Contrastive Multimodal Pretraining for Autonomous Systems
Shuang Ma, Sai Vemprala, Wenshan Wang +4
Learning representations that generalize across tasks and domains is challenging yet necessary for autonomous systems. Although task-driven approaches are appealing, designing mode…
A Neural Network-Based Linguistic Similarity Measure for Entrainment in Conversations
Mingzhi Yu, Diane Litman, Shuang Ma +1
Linguistic entrainment is a phenomenon where people tend to mimic each other in conversation. The core instrument to quantify entrainment is a linguistic similarity measure between…
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
Daniel McDuff, Yale Song, Jiyoung Lee +7
The ability to perform causal and counterfactual reasoning are central properties of human intelligence. Decision-making systems that can perform these types of reasoning have the…
Contrastive Learning of Global-Local Video Representations
Shuang Ma, Zhaoyang Zeng, Daniel McDuff +1
Contrastive learning has delivered impressive results for various tasks in the self-supervised regime. However, existing approaches optimize for learning representations specific t…
Active Contrastive Learning of Audio-Visual Video Representations
Shuang Ma, Zhaoyang Zeng, Daniel McDuff +1
Contrastive learning has been shown to produce generalizable representations of audio and visual data by maximizing the lower bound on the mutual information (MI) between different…
Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency
Matt Whitehill, Shuang Ma, Daniel McDuff +1
Current multi-reference style transfer models for Text-to-Speech (TTS) perform sub-optimally on disjoints datasets, where one dataset contains only a single style class for one of…
Unpaired Image-to-Speech Synthesis with Multimodal Information Bottleneck
Shuang Ma, Daniel McDuff, Yale Song
Deep generative models have led to significant advances in cross-modal generation such as text-to-image synthesis. Training these models typically requires paired data with direct…
M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention
Shuang Ma, Daniel McDuff, Yale Song
Generative adversarial networks have led to significant advances in cross-modal/domain translation. However, typically these networks are designed for a specific task (e.g., dialog…
Characterizing Bias in Classifiers using Generative Models
Daniel McDuff, Shuang Ma, Yale Song +1
Models that are learned from real-world data are often biased because the data used to train them is biased. This can propagate systemic human biases that exist and ultimately lead…
DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
Shuang Ma, Jianlong Fu, Chang Wen Chen +1
Unsupervised image translation, which aims in translating two independent sets of images, is challenging in discovering the correct correspondences without paired data. Existing wo…
D-Sempre: Learning Deep Semantic-Preserving Embeddings for User interests-Social Contents Modeling
Shuang Ma, Chang Wen Chen
Exponential growth of social media consumption demands effective user interests-social contents modeling for more personalized recommendation and social media summarization. Howeve…
A-Lamp: Adaptive Layout-Aware Multi-Patch Deep Convolutional Neural Network for Photo Aesthetic Assessment
Shuang Ma, Jing Liu, Chang Wen Chen
Deep convolutional neural networks (CNN) have recently been shown to generate promising results for aesthetics assessment. However, the performance of these deep CNN methods is oft…