7 citations · 10 across the 5 of their papers we have counts for
7 papers
Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation
Damiano Marsili, Raphi Kang, Aditya Mehta +2
Mixture-of-Experts (MoE) architectures scale model capacity through sparse computation, routing each token through only a small subset of experts. In this work, we explore whether…
Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models
Raphi Kang, Hongqiao Chen, Georgia Gkioxari +1
Spatio-temporal reasoning is a remarkable capability of Vision Language Models (VLMs), but the underlying mechanisms of such abilities remain largely opaque. We postulate that visu…
NitroGen: An Open Foundation Model for Generalist Gaming Agents
Loïc Magne, Anas Awadalla, Guanzhi Wang +11
We introduce NitroGen, a vision-action foundation model for generalist gaming agents that is trained on 40,000 hours of gameplay videos across more than 1,000 games. We incorporate…
Feedforward 3D Editing via Text-Steerable Image-to-3D
Ziqi Ma, Hongqiao Chen, Yisong Yue +1
Recent progress in image-to-3D has opened up immense possibilities for design, AR/VR, and robotics. However, to use AI-generated 3D assets in real applications, a critical requirem…
Is CLIP ideal? No. Can we fix it? Yes!
Raphi Kang, Yue Song, Georgia Gkioxari +1
Contrastive Language-Image Pre-Training (CLIP) is a popular method for learning multimodal latent spaces with well-organized semantics. Despite its wide range of applications, CLIP…
A Foundation Model for Cell Segmentation
Uriah Israel, Markus Marks, Rohit Dilip +11
Cells are the fundamental unit of biological organization, and identifying them in imaging data - cell segmentation - is a critical task for various cellular imaging experiments. W…
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
Jennifer J. Sun, Lili Karashchuk, Amil Dravid +9
Quantifying motion in 3D is important for studying the behavior of humans and other animals, but manual pose annotations are expensive and time-consuming to obtain. Self-supervised…