1.9k citations · 2.1k across the 21 of their papers we have counts for
21 papers
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
NVIDIA, :, Yan Wang +41
End-to-end architectures trained via imitation learning have advanced autonomous driving by scaling model size and data, yet performance remains brittle in safety-critical long-tai…
VideoSAM: Open-World Video Segmentation
Pinxue Guo, Zixu Zhao, Jianxiong Gao +5
Video segmentation is essential for advancing robotics and autonomous driving, particularly in open-world settings where continuous perception and object association across video f…
Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation
Jiaxin Cheng, Zixu Zhao, Tong He +3
Recent advancements in generative models have significantly enhanced their capacity for image generation, enabling a wide range of applications such as image editing, completion an…
Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach
Zechen Bai, Tianjun Xiao, Tong He +4
As online video content rapidly grows, the task of text-video retrieval (TVR) becomes increasingly important. A key challenge in TVR is the information asymmetry between video and…
Adaptive Slot Attention: Object Discovery with Dynamic Slot Number
Ke Fan, Zechen Bai, Tianjun Xiao +5
Object-centric learning (OCL) extracts the representation of objects with slots, offering an exceptional blend of flexibility and interpretability for abstracting low-level percept…
Hallucination of Multimodal Large Language Models: A Survey
Zechen Bai, Pichao Wang, Tianjun Xiao +4
This survey presents a comprehensive analysis of the phenomenon of hallucination in multimodal large language models (MLLMs), also known as Large Vision-Language Models (LVLMs), wh…
Consistent Video-to-Video Transfer Using Synthetic Dataset
Jiaxin Cheng, Tianjun Xiao, Tong He
We introduce a novel and efficient approach for text-based video-to-video editing that eliminates the need for resource-intensive per-video-per-model finetuning. At the core of our…
Rethinking Amodal Video Segmentation from Learning Supervised Signals with Object-centric Representation
Ke Fan, Jingshi Lei, Xuelin Qian +5
Video amodal segmentation is a particularly challenging task in computer vision, which requires to deduce the full shape of an object from the visible parts of it. Recently, some s…
Object-Centric Multiple Object Tracking
Zixu Zhao, Jiaze Wang, Max Horn +13
Unsupervised object-centric learning methods allow the partitioning of scenes into entities without additional localization information and are excellent candidates for reducing th…
Unsupervised Open-Vocabulary Object Localization in Videos
Ke Fan, Zechen Bai, Tianjun Xiao +11
In this paper, we show that recent advances in video representation learning and pre-trained vision-language models allow for substantial improvements in self-supervised video obje…
Coarse-to-Fine Amodal Segmentation with Shape Prior
Jianxiong Gao, Xuelin Qian, Yikai Wang +4
Amodal object segmentation is a challenging task that involves segmenting both visible and occluded parts of an object. In this paper, we propose a novel approach, called Coarse-to…
OpenVIS: Open-vocabulary Video Instance Segmentation
Pinxue Guo, Tony Huang, Peiyang He +4
Open-vocabulary Video Instance Segmentation (OpenVIS) can simultaneously detect, segment, and track arbitrary object categories in a video, without being constrained to categories…
Learning for Transductive Threshold Calibration in Open-World Recognition
Qin Zhang, Dongsheng An, Tianjun Xiao +6
In deep metric learning for visual recognition, the calibration of distance thresholds is crucial for achieving desired model performance in the true positive rates (TPR) or true n…
LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation
Jiaxin Cheng, Xiao Liang, Xingjian Shi +3
Layout-to-image generation refers to the task of synthesizing photo-realistic images based on semantic layouts. In this paper, we propose LayoutDiffuse that adapts a foundational d…
Self-supervised Amodal Video Object Segmentation
Jian Yao, Yuxin Hong, Chiyu Wang +6
Amodal perception requires inferring the full shape of an object that is partially occluded. This task is particularly challenging on two levels: (1) it requires more information t…
Bridging the Gap to Real-World Object-Centric Learning
Maximilian Seitzer, Max Horn, Andrii Zadaianchuk +8
Humans naturally decompose their environment into entities at the appropriate level of abstraction to act in the world. Allowing machine learning algorithms to derive this decompos…
Learning Hierarchical Graph Neural Networks for Image Clustering
Yifan Xing, Tong He, Tianjun Xiao +6
We propose a hierarchical graph neural network (GNN) model that learns how to cluster a set of images into an unknown number of identities using a training set of images annotated…
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Tianqi Chen, Mu Li, Yutian Li +7
MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends decl…
The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification
Tianjun Xiao, Yichong Xu, Kuiyuan Yang +3
Fine-grained classification is challenging because categories can only be discriminated by subtle and local differences. Variances in the pose, scale or rotation usually make the p…
Scale-Invariant Convolutional Neural Networks
Yichong Xu, Tianjun Xiao, Jiaxing Zhang +2
Even though convolutional neural networks (CNN) has achieved near-human performance in various computer vision tasks, its ability to tolerate scale variations is limited. The popul…