134 citations · 701 across the 33 of their papers we have counts for
33 papers
LOVM: Language-Only Vision Model Selection
Orr Zohar, Shih-Cheng Huang, Kuan-Chieh Wang +1
Pre-trained multi-modal vision-language models (VLMs) are becoming increasingly popular due to their exceptional performance on downstream vision applications, particularly in the…
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou +4
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data. In this work, we introduce NeQA,…
Hyperbolic Deep Learning in Computer Vision: A Survey
Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressel +2
Deep representation learning is a ubiquitous part of modern computer vision. While Euclidean space has been the de facto standard manifold for learning visual representations, hype…
Video Pretraining Advances 3D Deep Learning on Chest CT Tasks
Alexander Ke, Shih-Cheng Huang, Chloe P O'Connell +3
Pretraining on large natural image classification datasets such as ImageNet has aided model development on data-scarce 2D medical tasks. 3D medical tasks often have much less data…
Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation
Yuhui Zhang, Shih-Cheng Huang, Zhengping Zhou +2
Given the prevalence of 3D medical imaging technologies such as MRI and CT that are widely used in diagnosing and treating diverse diseases, 3D segmentation is one of the fundament…
Diagnosing and Rectifying Vision Models using Language
Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang +3
Recent multi-modal contrastive learning models have demonstrated the ability to learn an embedding space suitable for building strong vision classifiers, by leveraging the rich inf…
NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action
Kuan-Chieh Wang, Zhenzhen Weng, Maria Xenochristou +4
The task of reconstructing 3D human motion has wideranging applications. The gold standard Motion capture (MoCap) systems are accurate but inaccessible to the general public due to…
PROB: Probabilistic Objectness for Open World Object Detection
Orr Zohar, Kuan-Chieh Wang, Serena Yeung
Open World Object Detection (OWOD) is a new and challenging computer vision task that bridges the gap between classic object detection (OD) benchmarks and object detection in the r…
Adaptation of Surgical Activity Recognition Models Across Operating Rooms
Ali Mottaghi, Aidean Sharghi, Serena Yeung +1
Automatic surgical activity recognition enables more intelligent surgical devices and a more efficient workflow. Integration of such technology in new operating rooms has the poten…
DataPerf: Benchmarks for Data-Centric AI Development
Mark Mazumder, Colby Banbury, Xiaozhe Yao +42
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithf…
Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery
Zhenzhen Weng, Kuan-Chieh Wang, Angjoo Kanazawa +1
The ability to perceive 3D human bodies from a single image has a multitude of applications ranging from entertainment and robotics to neuroscience and healthcare. A fundamental ch…
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
Weixin Liang, Yuhui Zhang, Yongchan Kwon +2
We present modality gap, an intriguing geometric phenomenon of the representation space of multi-modal models. Specifically, we show that different data modalities (e.g. images and…
A real-time spatiotemporal AI model analyzes skill in open surgical videos
Emmett D. Goodman, Krishna K. Patel, Yilun Zhang +10
Open procedures represent the dominant form of surgery worldwide. Artificial intelligence (AI) has the potential to optimize surgical practice and improve patient outcomes, but eff…
FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation
Julia Gong, F. Christopher Holsinger, Serena Yeung
We consider the task of semi-supervised video object segmentation (VOS). Our approach mitigates shortcomings in previous VOS work by addressing detail preservation and temporal con…
Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning
Jeffrey Gu, Serena Yeung
Creating representations of shapes that are invari-ant to isometric or almost-isometric transforma-tions has long been an area of interest in shape anal-ysis, since enforcing invar…
DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images
Joy Hsu, Wah Chiu, Serena Yeung
In the biomedical domain, there is an abundance of dense, complex data where objects of interest may be challenging to detect or constrained by limits of human knowledge. Labelled…
Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
Zhenzhen Weng, Mehmet Giray Ogut, Shai Limonchik +1
Instance segmentation is an active topic in computer vision that is usually solved by using supervised learning approaches over very large datasets composed of object level masks.…
Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery
Michael Zhang, Xiaotian Cheng, Daniel Copeland +4
Open, or non-laparoscopic surgery, represents the vast majority of all operating room procedures, but few tools exist to objectively evaluate these techniques at scale. Current eff…
Personalized Federated Learning with First Order Model Optimization
Michael Zhang, Karan Sapra, Sanja Fidler +2
While federated learning traditionally aims to train a single global model across decentralized local datasets, one model may not always be ideal for all participating clients. Her…
Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
Joy Hsu, Jeffrey Gu, Gong-Her Wu +2
We consider the task of representation learning for unsupervised segmentation of 3D voxel-grid biomedical images. We show that models that capture implicit hierarchical relationshi…
Holistic 3D Human and Scene Mesh Estimation from Single View Images
Zhenzhen Weng, Serena Yeung
The 3D world limits the human body pose and the human body pose conveys information about the surrounding objects. Indeed, from a single image of a person placed in an indoor scene…
Medical symptom recognition from patient text: An active learning approach for long-tailed multilabel distributions
Ali Mottaghi, Prathusha K Sarma, Xavier Amatriain +2
We study the problem of medical symptoms recognition from patient text, for the purposes of gathering pertinent information from the patient (known as history-taking). A typical pa…
Rapidly Personalizing Mobile Health Treatment Policies with Limited Data
Sabina Tomkins, Peng Liao, Predrag Klasnja +2
In mobile health (mHealth), reinforcement learning algorithms that adapt to one's context without learning personalized policies might fail to distinguish between the needs of indi…
Adversarial Representation Active Learning
Ali Mottaghi, Serena Yeung
Active learning aims to develop label-efficient algorithms by querying the most informative samples to be labeled by an oracle. The design of efficient training methods that requir…