23 citations · 35 across the 11 of their papers we have counts for
12 papers
Open-Ended CT Volume Segmentation with Weak Supervision from Language
Sanjay Subramanian, Junwei Yu, Zirui Wang +5
We introduce a method for training a text-conditioned segmentation model for CT scans, which combines voxel-level supervision with coarse but scalable slice-level supervision from…
Stateful Visual Encoders for Vision-Language Models
Zirui Wang, Junwei Yu, Adam Yala +3
Vision-language models (VLMs) are increasingly used in multi-image, multi-turn agentic settings where decisions depend on visual changes. However, in existing open-weight VLMs, vis…
PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation
Yichuan Wang, Zhifei Li, Zirui Wang +5
Augmenting large language models (LLMs) with retrieved web text has become a dominant paradigm, yet the web is not natively textual: existing systems depend on complex parsing pipe…
VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents
Zirui Wang, Junyi Zhang, Jiaxin Ge +9
Modern Vision-Language Models (VLMs) remain poorly characterized in multi-step visual interactions, particularly in how they integrate perception, memory, and action over long hori…
YOLO-Count: Differentiable Object Counting for Text-to-Image Generation
Guanning Zeng, Xiang Zhang, Zirui Wang +4
We propose YOLO-Count, a differentiable open-vocabulary object counting model that tackles both general counting challenges and enables precise quantity control for text-to-image (…
CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
Zirui Wang, Mengzhou Xia, Luxi He +10
Chart understanding plays a pivotal role when applying Multimodal Large Language Models (MLLMs) to real-world tasks such as analyzing scientific papers or financial reports. Howeve…
OmniControlNet: Dual-stage Integration for Conditional Image Generation
Yilin Wang, Haiyang Xu, Xiang Zhang +4
We provide a two-way integration for the widely adopted ControlNet by integrating external condition generation algorithms into a single dense prediction method and incorporating i…
Improving Language Understanding from Screenshots
Tianyu Gao, Zirui Wang, Adithya Bhaskar +1
An emerging family of language models (LMs), capable of processing both text and images within a single visual view, has the promise to unlock complex tasks such as chart understan…
Language Models as Science Tutors
Alexis Chevalier, Jiayi Geng, Alexander Wettig +19
NLP has recently made exciting progress toward training language models (LMs) with strong scientific problem-solving skills. However, model development has not focused on real-life…
TokenCompose: Text-to-Image Diffusion with Token-level Supervision
Zirui Wang, Zhizhou Sha, Zheng Ding +2
We present TokenCompose, a Latent Diffusion Model for text-to-image generation that achieves enhanced consistency between user-specified text prompts and model-generated images. De…
Language Models Meet World Models: Embodied Experiences Enhance Language Models
Jiannan Xiang, Tianhua Tao, Yi Gu +4
While large language models (LMs) have shown remarkable capabilities across numerous tasks, they often struggle with simple reasoning and planning in physical environments, such as…
On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning
Yifan Xu, Nicklas Hansen, Zirui Wang +3
Reinforcement Learning (RL) algorithms can solve challenging control problems directly from image observations, but they often require millions of environment interactions to do so…