Publications (346)
Meticulous Object Segmentation
Chenglin Yang, Yilin Wang, Jianming Zhang +3
Compared with common image segmentation tasks targeted at low-resolution images, higher resolution detailed image segmentation receives much less attention. In this paper, we propo…
GENFIG1: Visual Summaries of Scholarly Work as a Challenge for Vision-Language Models
Yaohan Guan, Pristina Wang, Najim Dehak +3
In many science papers, "Figure 1" serves as the primary visual summary of the core research idea. These figures are visually simple yet conceptually rich, often requiring signific…
Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation
Shuhao Fu, Yongyi Lu, Yan Wang +4
In this paper, we present a novel unsupervised domain adaptation (UDA) method, named Domain Adaptive Relational Reasoning (DARR), to generalize 3D multi-organ segmentation models t…
The Universal Weight Subspace Hypothesis
Prakhar Kaushik, Shravan Chaudhari, Ankit Vaidya +2
We show that deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces. We provide the first large-scale empirical evidence…
DINeMo: Learning Neural Mesh Models with no 3D Annotations
Weijie Guo, Guofeng Zhang, Wufei Ma +1
Category-level 3D/6D pose estimation is a crucial step towards comprehensive 3D scene understanding, which would enable a broad range of applications in robotics and embodied AI. R…
LychSim: A Controllable and Interactive Simulation Framework for Vision Research
Wufei Ma, Chloe Wang, Siyi Chen +3
While self-supervised pretraining has reduced vision systems' reliance on synthetic data, simulation remains an indispensable tool for closed-loop optimization and rigorous out-of-…
Play to Generalize: Learning to Reason Through Game Play
Yunfei Xie, Yinsong Ma, Shiyi Lan +3
Developing reasoning capabilities in multimodal large language models (MLLMs) remains challenging. Motivated by literature suggesting that gameplay promotes transferable reasoning…
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge
Jiyang Qi, Yan Gao, Yao Hu +7
Although deep learning methods have achieved advanced video object recognition performance in recent years, perceiving heavily occluded objects in a video is still a very challengi…
From Pixel to Cancer: Cellular Automata in Computed Tomography
Yuxiang Lai, Xiaoxi Chen, Angtian Wang +2
AI for cancer detection encounters the bottleneck of data scarcity, annotation difficulty, and low prevalence of early tumors. Tumor synthesis seeks to create artificial tumors in…
Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering
Xingrui Wang, Wufei Ma, Angtian Wang +3
For vision-language models (VLMs), understanding the dynamic properties of objects and their interactions in 3D scenes from videos is crucial for effective reasoning about high-lev…
AsyInst: Asymmetric Affinity with DepthGrad and Color for Box-Supervised Instance Segmentation
Siwei Yang, Longlong Jing, Junfei Xiao +3
The weakly supervised instance segmentation is a challenging task. The existing methods typically use bounding boxes as supervision and optimize the network with a regularization l…
Robust Face Detection via Learning Small Faces on Hard Images
Zhishuai Zhang, Wei Shen, Siyuan Qiao +3
Recent anchor-based deep face detectors have achieved promising performance, but they are still struggling to detect hard faces, such as small, blurred and partially occluded faces…
3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation
Yi Zhang, Pengliang Ji, Angtian Wang +3
Regression-based methods for 3D human pose estimation directly predict the 3D pose parameters from a 2D image using deep networks. While achieving state-of-the-art performance on s…
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
Chenxi Liu, Liang-Chieh Chen, Florian Schroff +4
Recently, Neural Architecture Search (NAS) has successfully identified neural network architectures that exceed human designed ones on large-scale image classification. In this pap…
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Jiacong Xu, Yi Zhang, Jiawei Peng +17
Accurately estimating the 3D pose and shape is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife c…
OriNet: A Fully Convolutional Network for 3D Human Pose Estimation
Chenxu Luo, Xiao Chu, Alan Yuille
In this paper, we propose a fully convolutional network for 3D human pose estimation from monocular images. We use limb orientations as a new way to represent 3D poses and bind the…
Gaussian Scenes: Pose-Free Sparse-View Scene Reconstruction using Depth-Enhanced Diffusion Priors
Soumava Paul, Prakhar Kaushik, Alan Yuille
In this work, we introduce a generative approach for pose-free (without camera parameters) reconstruction of 360 scenes from a sparse set of 2D images. Pose-free scene reconstructi…
ReVision: Refining Video Diffusion with Explicit 3D Motion Modeling
Qihao Liu, Ju He, Qihang Yu +2
In recent years, video generation has seen significant advancements. However, challenges still persist in generating complex motions and interactions. To address these challenges,…
ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning
Yuxuan Wang, Alan Yuille, Zhuowan Li +1
Compositional visual reasoning methods, which translate a complex query into a structured composition of feasible visual tasks, have exhibited a strong potential in complicated mul…
ARVideo: Autoregressive Pretraining for Self-Supervised Video Representation Learning
Sucheng Ren, Hongru Zhu, Chen Wei +3
This paper presents a new self-supervised video representation learning framework, ARVideo, which autoregressively predicts the next video token in a tailored sequence order. Two k…
Semantic Part Detection via Matching: Learning to Generalize to Novel Viewpoints from Limited Training Data
Yutong Bai, Qing Liu, Lingxi Xie +3
Detecting semantic parts of an object is a challenging task in computer vision, particularly because it is hard to construct large annotated datasets due to the difficulty of annot…
NOVUM: Neural Object Volumes for Robust Object Classification
Artur Jesslen, Guofeng Zhang, Angtian Wang +3
Discriminative models for object classification typically learn image-based representations that do not capture the compositional and 3D nature of objects. In this work, we show th…
Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy
Zhuotun Zhu, Dakai Jin, Ke Yan +7
Finding, identifying and segmenting suspicious cancer metastasized lymph nodes from 3D multi-modality imaging is a clinical task of paramount importance. In radiotherapy, they are…
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
Haoyu Chen, Kaichen Zhou, Hang Hua +11
The paper introduces MemoBench, a benchmark that tests video generation models' ability to remember and correctly update objects that disappear and later reappear in dynamically ch…
VTok: A Unified Video Tokenizer with Decoupled Spatial-Temporal Latents
Feng Wang, Yichun Shi, Ceyuan Yang +4
This work presents VTok, a unified video tokenization framework that can be used for both generation and understanding tasks. Unlike the leading vision-language systems that tokeni…
PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter
Junfei Xiao, Zheng Xu, Alan Yuille +2
This paper demonstrates that a progressively aligned language model can effectively bridge frozen vision encoders and large language models (LLMs). While the fundamental architectu…
Unsupervised Domain Adaptation through Shape Modeling for Medical Image Segmentation
Yuan Yao, Fengze Liu, Zongwei Zhou +4
Shape information is a strong and valuable prior in segmenting organs in medical images. However, most current deep learning based segmentation algorithms have not taken shape info…
Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search
Dazhou Guo, Dakai Jin, Zhuotun Zhu +7
OAR segmentation is a critical step in radiotherapy of head and neck (H&N) cancer, where inconsistencies across radiation oncologists and prohibitive labor costs motivate automated…
Occluded Video Instance Segmentation: A Benchmark
Jiyang Qi, Yan Gao, Yao Hu +7
Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset called OVIS for occluded vi…
Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data
Qi Chen, Xinze Zhou, Chen Liu +11
AI for tumor segmentation is limited by the lack of large, voxel-wise annotated datasets, which are hard to create and require medical experts. In our proprietary JHH dataset of 3,…
RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition
Yi Zhang, Xinyue Wei, Weichao Qiu +3
Despite the rapid growth in datasets for video activity, stable robust activity recognition with neural networks remains challenging. This is in large part due to the explosion of…
CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
Haoran Li, Sucheng Ren, Alan Yuille +1
Rotary Positional Embedding (RoPE) is a key component of context scaling in Large Language Models (LLMs). While various methods have been proposed to adapt RoPE to longer contexts,…
Deep Co-Training for Semi-Supervised Image Recognition
Siyuan Qiao, Wei Shen, Zhishuai Zhang +2
In this paper, we study the problem of semi-supervised image recognition, which is to learn classifiers using both labeled and unlabeled images. We present Deep Co-Training, a deep…
Context-Enhanced Stereo Transformer
Weiyu Guo, Zhaoshuo Li, Yongkui Yang +5
Stereo depth estimation is of great interest for computer vision research. However, existing methods struggles to generalize and predict reliably in hazardous regions, such as larg…
A Very Big Video Reasoning Suite
Maijunxian Wang, Ruisi Wang, Juyi Lin +53
Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grounds intelligence in spatiotemporally c…
Analyzing Tumors by Synthesis
Qi Chen, Yuxiang Lai, Xiaoxi Chen +3
Computer-aided tumor detection has shown great potential in enhancing the interpretation of over 80 million CT scans performed annually in the United States. However, challenges ar…
CO2: Consistent Contrast for Unsupervised Visual Representation Learning
Chen Wei, Huiyu Wang, Wei Shen +1
Contrastive learning has been adopted as a core method for unsupervised visual representation learning. Without human annotation, the common practice is to perform an instance disc…
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
Shiyu Tang, Ruihao Gong, Yan Wang +9
Deep neural networks (DNNs) are vulnerable to adversarial noises, which motivates the benchmark of model robustness. Existing benchmarks mainly focus on evaluating defenses, but th…
Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu +3
Detecting objects becomes difficult when we need to deal with large shape deformation, occlusion and low resolution. We propose a novel approach to i) handle large deformations and…
Exploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images
Tiange Xiang, Yixiao Zhang, Yongyi Lu +4
Radiography imaging protocols focus on particular body regions, therefore producing images of great similarity and yielding recurrent anatomical structures across patients. Exploit…
Shared LoRA Subspaces for almost Strict Continual Learning
Prakhar Kaushik, Ankit Vaidya, Shravan Chaudhari +2
Adapting large pretrained models to new tasks efficiently and continually is crucial for real-world deployment but remains challenging due to catastrophic forgetting and the high c…
Name That Part: 3D Part Segmentation and Naming
Soumava Paul, Prakhar Kaushik, Ankit Vaidya +2
We address semantic 3D part segmentation: decomposing objects into parts with meaningful names. While datasets exist with part annotations, their definitions are inconsistent acros…
Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images
Junhua Mao, Jiajing Xu, Yushi Jing +1
In this paper, we focus on training and evaluating effective word embeddings with both text and visual information. More specifically, we introduce a large-scale dataset with 300 m…
Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
Ju He, Adam Kortylewski, Shaokang Yang +4
Semi-Supervised Learning (SSL) has shown its strong ability in utilizing unlabeled data when labeled data is scarce. However, most SSL algorithms work under the assumption that the…
ImageNet3D: Towards General-Purpose Object-Level 3D Understanding
Wufei Ma, Guanning Zeng, Guofeng Zhang +5
A vision model with general-purpose object-level 3D understanding should be capable of inferring both 2D (e.g., class name and bounding box) and 3D information (e.g., 3D location a…
FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning
Peiran Xu, Zeyu Wang, Jieru Mei +4
Federated learning (FL) is an emerging paradigm in machine learning, where a shared model is collaboratively learned using data from multiple devices to mitigate the risk of data l…
Efficient Large Multi-modal Models via Visual Context Compression
Jieneng Chen, Luoxin Ye, Ju He +3
While significant advancements have been made in compressed representations for text embeddings in large language models (LLMs), the compression of visual tokens in multi-modal LLM…
World-in-World: World Models in a Closed-Loop World
Jiahan Zhang, Muqing Jiang, Nanru Dai +14
Generative world models (WMs) can now simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive pe…
Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More
Feng Wang, Yaodong Yu, Guoyizhe Wei +4
Since the introduction of Vision Transformer (ViT), patchification has long been regarded as a de facto image tokenization approach for plain visual architectures. By compressing t…
Synthetic Data as Validation
Qixin Hu, Alan Yuille, Zongwei Zhou
This study leverages synthetic data as a validation set to reduce overfitting and ease the selection of the best model in AI development. While synthetic data have been used for au…
Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation
Yuyin Zhou, Yingwei Li, Zhishuai Zhang +5
Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers with an overall five-year survival rate of 8%. Due to subtle texture changes of PDAC, pancreatic dual-phas…
CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
Runtao Liu, Chenxi Liu, Yutong Bai +1
Referring object detection and referring image segmentation are important tasks that require joint understanding of visual information and natural language. Yet there has been evid…
Dictionary-based Framework for Interpretable and Consistent Object Parsing
Tiezheng Zhang, Qihang Yu, Alan Yuille +1
In this work, we present CoCal, an interpretable and consistent object parsing framework based on dictionary-based mask transformer. Designed around Contrastive Components and Logi…
Rethinking Normalization and Elimination Singularity in Neural Networks
Siyuan Qiao, Huiyu Wang, Chenxi Liu +2
In this paper, we study normalization methods for neural networks from the perspective of elimination singularity. Elimination singularities correspond to the points on the trainin…
Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles
Chenxu Luo, Lin Sun, Dariush Dabiri +1
Trajectory prediction is crucial for autonomous vehicles. The planning system not only needs to know the current state of the surrounding objects but also their possible states in…
Few-shot Learning by Exploiting Visual Concepts within CNNs
Boyang Deng, Qing Liu, Siyuan Qiao +1
Convolutional neural networks (CNNs) are one of the driving forces for the advancement of computer vision. Despite their promising performances on many tasks, CNNs still face major…
PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation
Yuanhao Cai, Kunpeng Li, Menglin Jia +11
Recent advances in text-to-video (T2V) generation have achieved good visual quality, yet synthesizing videos that faithfully follow physical laws remains an open challenge. Existin…
RATS! Patches Talk Through Registers: Emergent Parts in Register Attention Transformers
Timing Yang, Predrag Neskovic, Jansen Seheult +4
When humans see a bird, they recognize far more than just "bird" -- they see a head, wings, and talons, a structured assembly of reusable parts that can be identified across every…
Resisting Large Data Variations via Introspective Transformation Network
Yunhan Zhao, Ye Tian, Charless Fowlkes +2
Training deep networks that generalize to a wide range of variations in test data is essential to building accurate and robust image classifiers. One standard strategy is to apply…
WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark
Wang Lin, Feng Wang, Majun Zhang +7
Recent advances in image editing models have demonstrated remarkable capabilities in executing explicit instructions, such as attribute manipulation, style transfer, and pose synth…
Understanding Pan-Sharpening via Generalized Inverse
Shiqi Liu, Yihua Tan, Yutong Bai +1
Pan-sharpening algorithms utilize a panchromatic image and a multispectral image to generate a high spatial and high spectral image. However, the optimizations of the algorithms ar…
CORL: Compositional Representation Learning for Few-Shot Classification
Ju He, Adam Kortylewski, Alan Yuille
Few-shot image classification consists of two consecutive learning processes: 1) In the meta-learning stage, the model acquires a knowledge base from a set of training classes. 2)…
Robust Object Detection under Occlusion with Context-Aware CompositionalNets
Angtian Wang, Yihong Sun, Adam Kortylewski +1
Detecting partially occluded objects is a difficult task. Our experimental results show that deep learning approaches, such as Faster R-CNN, are not robust at object detection unde…
A Simple Data Mixing Prior for Improving Self-Supervised Learning
Sucheng Ren, Huiyu Wang, Zhengqi Gao +4
Data mixing (e.g., Mixup, Cutmix, ResizeMix) is an essential component for advancing recognition models. In this paper, we focus on studying its effectiveness in the self-supervise…
Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features
Wufei Ma, Angtian Wang, Alan Yuille +1
We consider the problem of category-level 6D pose estimation from a single RGB image. Our approach represents an object category as a cuboid mesh and learns a generative model of t…
Universal Physical Camouflage Attacks on Object Detectors
Lifeng Huang, Chengying Gao, Yuyin Zhou +4
In this paper, we study physical adversarial attacks on object detectors in the wild. Previous works mostly craft instance-dependent perturbations only for rigid or planar objects.…
FusionNet: Incorporating Shape and Texture for Abnormality Detection in 3D Abdominal CT Scans
Fengze Liu, Yuyin Zhou, Elliot Fishman +1
Automatic abnormality detection in abdominal CT scans can help doctors improve the accuracy and efficiency in diagnosis. In this paper we aim at detecting pancreatic ductal adenoca…
Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis
Angtian Wang, Wufei Ma, Alan Yuille +1
Human vision demonstrates higher robustness than current AI algorithms under out-of-distribution scenarios. It has been conjectured such robustness benefits from performing analysi…
A Light-weight Interpretable Compositional Model for Nuclei Detection and Weakly-Supervised Segmentation
Yixiao Zhang, Adam Kortylewski, Qing Liu +9
The field of computational pathology has witnessed great advancements since deep neural networks have been widely applied. These networks usually require large numbers of annotated…
Identity Preserve Transform: Understand What Activity Classification Models Have Learnt
Jialing Lyu, Weichao Qiu, Xinyue Wei +3
Activity classification has observed great success recently. The performance on small dataset is almost saturated and people are moving towards larger datasets. What leads to the p…
Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis
Yuanhao Cai, Yixun Liang, Jiahao Wang +5
X-ray is widely applied for transmission imaging due to its stronger penetration than natural light. When rendering novel view X-ray projections, existing methods mainly based on N…
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
Junhua Mao, Wei Xu, Yi Yang +3
In this paper, we present a multimodal Recurrent Neural Network (m-RNN) model for generating novel image captions. It directly models the probability distribution of generating a w…
Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
Yingda Xia, Dong Yang, Zhiding Yu +7
Although having achieved great success in medical image segmentation, deep learning-based approaches usually require large amounts of well-annotated data, which can be extremely ex…
Ground-truth dataset and baseline evaluations for image base-detail separation algorithms
Xuan Dong, Boyan Bonev, Weixin Li +3
Base-detail separation is a fundamental computer vision problem consisting of modeling a smooth base layer with the coarse structures, and a detail layer containing the texture-lik…
Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Ruxiao Duan, Jieneng Chen, Adam Kortylewski +2
Replay-based methods in class-incremental learning (CIL) have attained remarkable success. Despite their effectiveness, the inherent memory restriction results in saving a limited…
HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction
Angtian Wang, Yuanlu Xu, Nikolaos Sarafianos +4
Neural reconstruction and rendering strategies have demonstrated state-of-the-art performances due, in part, to their ability to preserve high level shape details. Existing approac…
IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers
Chenglin Yang, Siyuan Qiao, Yuan Cao +4
Generative training has been demonstrated to be powerful for building visual-language models. However, on zero-shot discriminative benchmarks, there is still a performance gap betw…
DASZL: Dynamic Action Signatures for Zero-shot Learning
Tae Soo Kim, Jonathan D. Jones, Michael Peven +6
There are many realistic applications of activity recognition where the set of potential activity descriptions is combinatorially large. This makes end-to-end supervised training o…
FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matching
Sucheng Ren, Qihang Yu, Ju He +3
Autoregressive (AR) modeling has achieved remarkable success in natural language processing by enabling models to generate text with coherence and contextual understanding through…
Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistency
Qing Liu, Vignesh Ramanathan, Dhruv Mahajan +2
Weakly supervised instance segmentation reduces the cost of annotations required to train models. However, existing approaches which rely only on image-level class labels predomina…
Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
Yingda Xia, Yi Zhang, Fengze Liu +2
The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of…
PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation
Qihao Liu, Adam Kortylewski, Alan Yuille
Human pose and shape (HPS) estimation methods achieve remarkable results. However, current HPS benchmarks are mostly designed to test models in scenarios that are similar to the tr…
Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinate
Soumava Paul, Prakhar Kaushik, Alan Yuille
Multiview 3D evaluation assumes that the images being scored are observations of one static 3D scene. This assumption can fail in NVS and sparse-view reconstruction: inputs or gene…
Localization vs. Semantics: Visual Representations in Unimodal and Multimodal Models
Zhuowan Li, Cihang Xie, Benjamin Van Durme +1
Despite the impressive advancements achieved through vision-and-language pretraining, it remains unclear whether this joint learning paradigm can help understand each individual mo…
SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation
Yiqing Wang, Zihan Li, Jieru Mei +7
Recent advancements in large-scale Vision Transformers have made significant strides in improving pre-trained models for medical image segmentation. However, these methods face a n…
Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
Chenglin Yang, Lingxi Xie, Siyuan Qiao +1
We focus on the problem of training a deep neural network in generations. The flowchart is that, in order to optimize the target network (student), another network (teacher) with t…
TriDiff-4D: Fast 4D Generation through Diffusion-based Triplane Re-posing
Eddie Pokming Sheung, Qihao Liu, Wufei Ma +3
With the increasing demand for 3D animation, generating high-fidelity, controllable 4D avatars from textual descriptions remains a significant challenge. Despite notable efforts in…
Adventurer: Optimizing Vision Mamba Architecture Designs for Efficiency
Feng Wang, Timing Yang, Yaodong Yu +7
In this work, we introduce the Adventurer series models where we treat images as sequences of patch tokens and employ uni-directional language models to learn visual representation…
SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images
Sky Cen, Wufei Ma, Guofeng Zhang +2
Learning deformable 3D object models from single-view in-the-wild images has enabled impressive 3D shape reconstruction without supervision. However, it remains unclear whether the…
Lite Vision Transformer with Enhanced Self-Attention
Chenglin Yang, Yilin Wang, Jianming Zhang +4
Despite the impressive representation capacity of vision transformer models, current light-weight vision transformer models still suffer from inconsistent and incorrect dense predi…
Scene Graph Parsing as Dependency Parsing
Yu-Siang Wang, Chenxi Liu, Xiaohui Zeng +1
In this paper, we study the problem of parsing structured knowledge graphs from textual descriptions. In particular, we consider the scene graph representation that considers objec…
Batch Normalization with Enhanced Linear Transformation
Yuhui Xu, Lingxi Xie, Cihang Xie +5
Batch normalization (BN) is a fundamental unit in modern deep networks, in which a linear transformation module was designed for improving BN's flexibility of fitting complex data…
When Radiology Report Generation Meets Knowledge Graph
Yixiao Zhang, Xiaosong Wang, Ziyue Xu +3
Automatic radiology report generation has been an attracting research problem towards computer-aided diagnosis to alleviate the workload of doctors in recent years. Deep learning t…
Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data
Yu-Cheng Chou, Bowen Li, Deng-Ping Fan +2
Creating large-scale and well-annotated datasets to train AI algorithms is crucial for automated tumor detection and localization. However, with limited resources, it is challengin…
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition under Occlusion
Adam Kortylewski, Qing Liu, Angtian Wang +2
Computer vision systems in real-world applications need to be robust to partial occlusion while also being explainable. In this work, we show that black-box deep convolutional neur…
Medical Vision Generalist: Unifying Medical Imaging Tasks in Context
Sucheng Ren, Xiaoke Huang, Xianhang Li +5
This study presents Medical Vision Generalist (MVG), the first foundation model capable of handling various medical imaging tasks -- such as cross-modal synthesis, image segmentati…
Recurrent Multimodal Interaction for Referring Image Segmentation
Chenxi Liu, Zhe Lin, Xiaohui Shen +3
In this paper we are interested in the problem of image segmentation given natural language descriptions, i.e. referring expressions. Existing works tackle this problem by first mo…
Towards Generalizable Tumor Synthesis
Qi Chen, Xiaoxi Chen, Haorui Song +4
Tumor synthesis enables the creation of artificial tumors in medical images, facilitating the training of AI models for tumor detection and segmentation. However, success in tumor…
TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion
Mingqing Xiao, Adam Kortylewski, Ruihai Wu +3
Despite deep convolutional neural networks' great success in object classification, it suffers from severe generalization performance drop under occlusion due to the inconsistency…
XModBench: Benchmarking Cross-Modal Capabilities and Consistency in Omni-Language Models
Xingrui Wang, Jiang Liu, Chao Huang +7
Omni-modal large language models (OLLMs) aim to unify audio, vision, and text understanding within a single framework. While existing benchmarks primarily evaluate general cross-mo…