479 citations · 1.9k across the 94 of their papers we have counts for
94 papers
TouchStone: Evaluating Vision-Language Models by Language Models
Shuai Bai, Shusheng Yang, Jinze Bai +6
Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information b…
SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentation
Fan Bai, Ke Yan, Xiaoyu Bai +6
Medical image analysis using deep learning is often challenged by limited labeled data and high annotation costs. Fine-tuning the entire network in label-limited scenarios can lead…
Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation
Dawei Gao, Haibin Wang, Yaliang Li +4
Large language models (LLMs) have emerged as a new paradigm for Text-to-SQL task. However, the absence of a systematical benchmark inhibits the development of designing effective,…
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Jinze Bai, Shuai Bai, Shusheng Yang +6
In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images. Starting from the Qwen…
Parse and Recall: Towards Accurate Lung Nodule Malignancy Prediction like Radiologists
Jianpeng Zhang, Xianghua Ye, Jianfeng Zhang +8
Lung cancer is a leading cause of death worldwide and early screening is critical for improving survival outcomes. In clinical practice, the contextual structure of nodules and the…
CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility
Guohai Xu, Jiayi Liu, Ming Yan +11
With the rapid evolution of large language models (LLMs), there is a growing concern that they may pose risks or have negative social impacts. Therefore, evaluation of human values…
Matching in the Wild: Learning Anatomical Embeddings for Multi-Modality Images
Xiaoyu Bai, Fan Bai, Xiaofei Huo +10
Radiotherapists require accurate registration of MR/CT images to effectively use information from both modalities. In a typical registration pipeline, rigid or affine transformatio…
SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid
Zi Li, Lin Tian, Tony C. W. Mok +8
Estimating displacement vector field via a cost volume computed in the feature space has shown great success in image registration, but it suffers excessive computation burdens. Mo…
Liver Tumor Screening and Diagnosis in CT with Pixel-Lesion-Patient Network
Ke Yan, Xiaoli Yin, Yingda Xia +10
Liver tumor segmentation and classification are important tasks in computer aided diagnosis. We aim to address three problems: liver tumor screening and preliminary diagnosis in no…
VideoComposer: Compositional Video Synthesis with Motion Controllability
Xiang Wang, Hangjie Yuan, Shiwei Zhang +6
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in customizable image synthesis. However, achieving controllable vide…
On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions
Leijie Wu, Song Guo, Junxiao Wang +3
As Federated Learning (FL) has gained increasing attention, it has become widely acknowledged that straightforwardly applying stochastic gradient descent (SGD) on the overall frame…
Lipschitz Singularities in Diffusion Models
Zhantao Yang, Ruili Feng, Han Zhang +8
Diffusion models, which employ stochastic differential equations to sample images through integrals, have emerged as a dominant class of generative models. However, the rationality…
Cones 2: Customizable Image Synthesis with Multiple Subjects
Zhiheng Liu, Yifei Zhang, Yujun Shen +7
Synthesizing images with user-specified subjects has received growing attention due to its practical applications. Despite the recent success in single subject customization, exist…
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
Peng Wang, Shijie Wang, Junyang Lin +5
In this work, we explore a scalable way for building a general representation model toward unlimited modalities. We release ONE-PEACE, a highly extensible model with 4B parameters…
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human
Junfeng Tian, Hehong Chen, Guohai Xu +14
In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified i…
Layph: Making Change Propagation Constraint in Incremental Graph Processing by Layering Graph
Song Yu, Shufeng Gong, Yanfeng Zhang +7
Real-world graphs are constantly evolving, which demands updates of the previous analysis results to accommodate graph changes. By using the memoized previous computation state, in…
FS-Real: Towards Real-World Cross-Device Federated Learning
Daoyuan Chen, Dawei Gao, Yuexiang Xie +5
Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data, which attracts increasing attention in bot…
ViM: Vision Middleware for Unified Downstream Transferring
Yutong Feng, Biao Gong, Jianwen Jiang +4
Foundation models are pre-trained on massive data and transferred to downstream tasks via fine-tuning. This work presents Vision Middleware (ViM), a new learning paradigm that targ…
Cones: Concept Neurons in Diffusion Models for Customized Generation
Zhiheng Liu, Ruili Feng, Kai Zhu +6
Human brains respond to semantic features of presented stimuli with different neurons. It is then curious whether modern deep neural networks admit a similar behavior pattern. Spec…
Meta-information-aware Dual-path Transformer for Differential Diagnosis of Multi-type Pancreatic Lesions in Multi-phase CT
Bo Zhou, Yingda Xia, Jiawen Yao +5
Pancreatic cancer is one of the leading causes of cancer-related death. Accurate detection, segmentation, and differential diagnosis of the full taxonomy of pancreatic lesions, i.e…
Rethinking Efficient Tuning Methods from a Unified Perspective
Zeyinzi Jiang, Chaojie Mao, Ziyuan Huang +3
Parameter-efficient transfer learning (PETL) based on large-scale pre-trained foundation models has achieved great success in various downstream applications. Existing tuning metho…
Composer: Creative and Controllable Image Synthesis with Composable Conditions
Lianghua Huang, Di Chen, Yu Liu +3
Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability. This work offers a new generation pa…
Lero: A Learning-to-Rank Query Optimizer
Rong Zhu, Wei Chen, Bolin Ding +4
A recent line of works apply machine learning techniques to assist or rebuild cost-based query optimizers in DBMS. While exhibiting superiority in some benchmarks, their deficienci…
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
Haiyang Xu, Qinghao Ye, Ming Yan +12
Recent years have witnessed a big convergence of language, vision, and multi-modal pretraining. In this work, we present mPLUG-2, a new unified paradigm with modularized design for…