94 citations · 600 across the 26 of their papers we have counts for
26 papers
FIND: A Function Description Benchmark for Evaluating Interpretability Methods
Sarah Schwettmann, Tamar Rott Shaham, Joanna Materzynska +5
Labeling neural network submodules with human-legible descriptions is useful for many downstream tasks: such descriptions can surface failures, guide interventions, and perhaps eve…
Discovering Intrinsic Spatial-Temporal Logic Rules to Explain Human Actions
Chengzhi Cao, Chao Yang, Shuang Li
We propose a logic-informed knowledge-driven modeling framework for human movements by analyzing their trajectories. Our approach is inspired by the fact that human actions are usu…
Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models
Nan Liu, Yilun Du, Shuang Li +2
Text-to-image generative models have enabled high-resolution image synthesis across different domains, but require users to specify the content they wish to generate. In this paper…
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Yilun Du, Shuang Li, Antonio Torralba +2
Large language models (LLMs) have demonstrated remarkable capabilities in language generation, understanding, and few-shot learning in recent years. An extensive body of work has e…
ConceptFusion: Open-set Multimodal 3D Mapping
Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu +14
Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts wit…
Composing Ensembles of Pre-trained Models via Iterative Consensus
Shuang Li, Yilun Du, Joshua B. Tenenbaum +2
Large pre-trained models exhibit distinct and complementary capabilities dependent on the data they are trained on. Language models such as GPT-3 are capable of textual reasoning b…
Common Corruption Robustness of Point Cloud Detectors: Benchmark and Enhancement
Shuangzhi Li, Zhijie Wang, Felix Juefei-Xu +3
Object detection through LiDAR-based point cloud has recently been important in autonomous driving. Although achieving high accuracy on public benchmarks, the state-of-the-art dete…
Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification
Xinyu Lin, Jinxing Li, Zeyu Ma +5
Thanks for the cross-modal retrieval techniques, visible-infrared (RGB-IR) person re-identification (Re-ID) is achieved by projecting them into a common space, allowing person Re-I…
Learning Iterative Reasoning through Energy Minimization
Yilun Du, Shuang Li, Joshua B. Tenenbaum +1
Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition. However, it struggles with tasks requiring nontrivial reasoning…
Feature Learning and Ensemble Pre-Tasks Based Self-Supervised Speech Denoising and Dereverberation
Yi Li, ShuangLin Li, Yang Sun +1
Self-supervised learning (SSL) achieves great success in monaural speech enhancement, while the accuracy of the target speech estimation, particularly for unseen speakers, remains…
Compositional Visual Generation with Composable Diffusion Models
Nan Liu, Shuang Li, Yilun Du +2
Large text-guided diffusion models, such as DALLE-2, are able to generate stunning photorealistic images given natural language descriptions. While such models are highly flexible,…
Pre-Trained Language Models for Interactive Decision-Making
Shuang Li, Xavier Puig, Chris Paxton +11
Language model (LM) pre-training is useful in many language processing tasks. But can pre-trained LMs be further leveraged for more general machine learning problems? We propose an…
Learning to Compose Visual Relations
Nan Liu, Shuang Li, Yilun Du +2
The visual world around us can be described as a structured set of objects and their associated relations. An image of a room may be conjured given only the description of the unde…
Unsupervised Learning of Compositional Energy Concepts
Yilun Du, Shuang Li, Yash Sharma +2
Humans are able to rapidly understand scenes by utilizing concepts extracted from prior experience. Such concepts are diverse, and include global scene descriptors, such as the wea…
Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions
Shuang Li, Yilun Du, Antonio Torralba +2
We introduce the task of weakly supervised learning for detecting human and object interactions in videos. Our task poses unique challenges as a system does not know what types of…
3D Neural Scene Representations for Visuomotor Control
Yunzhu Li, Shuang Li, Vincent Sitzmann +2
Humans have a strong intuitive understanding of the 3D environment around us. The mental model of the physics in our brain applies to objects of different materials and enables us…
Improved Contrastive Divergence Training of Energy Based Models
Yilun Du, Shuang Li, Joshua Tenenbaum +1
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability. We propose an adaptation to improve contrasti…
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
Xavier Puig, Tianmin Shu, Shuang Li +5
In this paper, we introduce Watch-And-Help (WAH), a challenge for testing social intelligence in agents. In WAH, an AI agent needs to help a human-like agent perform a complex hous…
Compositional Visual Generation and Inference with Energy Based Models
Yilun Du, Shuang Li, Igor Mordatch
A vital aspect of human intelligence is the ability to compose increasingly complex concepts out of simpler ideas, enabling both rapid learning and adaptation of knowledge. In this…
Question-Guided Hybrid Convolution for Visual Question Answering
Peng Gao, Pan Lu, Hongsheng Li +4
In this paper, we propose a novel Question-Guided Hybrid Convolution (QGHC) network for Visual Question Answering (VQA). Most state-of-the-art VQA methods fuse the high-level textu…
Diversity Regularized Spatiotemporal Attention for Video-based Person Re-identification
Shuang Li, Slawomir Bak, Peter Carr +1
Video-based person re-identification matches video clips of people across non-overlapping cameras. Most existing methods tackle this problem by encoding each video frame in its ent…
Can Image Retrieval help Visual Saliency Detection?
Shuang Li, Peter Mathews
We propose a novel image retrieval framework for visual saliency detection using information about salient objects contained within bounding box annotations for similar images. For…
Identity-Aware Textual-Visual Matching with Latent Co-attention
Shuang Li, Tong Xiao, Hongsheng Li +2
Textual-visual matching aims at measuring similarities between sentence descriptions and images. Most existing methods tackle this problem without effectively utilizing identity-le…
Learning Feature Pyramids for Human Pose Estimation
Wei Yang, Shuang Li, Wanli Ouyang +2
Articulated human pose estimation is a fundamental yet challenging task in computer vision. The difficulty is particularly pronounced in scale variations of human body parts when c…