35 citations · 171 across the 21 of their papers we have counts for
21 papers
RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
Yuhao Du, Zhuo Li, Pengyu Cheng +4
Reinforcement Learning from Human Feedback (RLHF) is crucial for aligning Large Language Models (LLMs) with human values. However, RLHF has been continuously challenged by its high…
Large Multimodal Agents: A Survey
Junlin Xie, Zhihong Chen, Ruifei Zhang +2
Large language models (LLMs) have achieved superior performance in powering text-based AI agents, endowing them with decision-making and reasoning abilities akin to humans. Concurr…
ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models
Guiming Hardy Chen, Shunian Chen, Ruifei Zhang +7
Large vision-language models (LVLMs) have shown premise in a broad range of vision-language tasks with their strong reasoning and generalization capabilities. However, they require…
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
Wentao Ge, Shunian Chen, Guiming Hardy Chen +14
Multimodal large language models (MLLMs) have broadened the scope of AI applications. Existing automatic evaluation methodologies for MLLMs are mainly limited in evaluating queries…
AceGPT, Localizing Large Language Models in Arabic
Huang Huang, Fei Yu, Jianqing Zhu +17
This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addr…
CMB: A Comprehensive Medical Benchmark in Chinese
Xidong Wang, Guiming Hardy Chen, Dingjie Song +8
Large Language Models (LLMs) provide a possibility to make a great breakthrough in medicine. The establishment of a standardized medical benchmark becomes a fundamental cornerstone…
Advancing Visual Grounding with Scene Knowledge: Benchmark and Method
Zhihong Chen, Ruifei Zhang, Yibing Song +2
Visual grounding (VG) aims to establish fine-grained alignment between vision and language. Ideally, it can be a testbed for vision-and-language models to evaluate their understand…
Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation
Zunnan Xu, Zhihong Chen, Yong Zhang +3
Parameter Efficient Tuning (PET) has gained attention for reducing the number of parameters while maintaining performance and providing better hardware resource savings, but few st…
On the Difference of BERT-style and CLIP-style Text Encoders
Zhihong Chen, Guiming Hardy Chen, Shizhe Diao +2
Masked language modeling (MLM) has been one of the most popular pretraining recipes in natural language processing, e.g., BERT, one of the representative models. Recently, contrast…
HuatuoGPT, towards Taming Language Model to Be a Doctor
Hongbo Zhang, Junying Chen, Feng Jiang +10
In this paper, we present HuatuoGPT, a large language model (LLM) for medical consultation. The core recipe of HuatuoGPT is to leverage both \textit{distilled data from ChatGPT} an…
Phoenix: Democratizing ChatGPT across Languages
Zhihong Chen, Feng Jiang, Junying Chen +11
This paper presents our efforts to democratize ChatGPT across language. We release a large language model "Phoenix", achieving competitive performance among open-source English and…
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts
Zhihong Chen, Shizhe Diao, Benyou Wang +2
Medical vision-and-language pre-training (Med-VLP) has shown promising improvements on many downstream medical tasks owing to its applicability to extracting generic representation…
GIPA: A General Information Propagation Algorithm for Graph Learning
Houyi Li, Zhihong Chen, Zhao Li +3
Graph neural networks (GNNs) have been widely used in graph-structured data computation, showing promising performance in various applications such as node classification, link pre…
Improving Radiology Summarization with Radiograph and Anatomy Prompts
Jinpeng Hu, Zhihong Chen, Yang Liu +2
The impression is crucial for the referring physicians to grasp key information since it is concluded from the findings and reasoning of radiologists. To alleviate the workload of…
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge
Zhihong Chen, Guanbin Li, Xiang Wan
Medical vision-and-language pre-training (Med-VLP) has received considerable attention owing to its applicability to extracting generic vision-and-language representations from med…
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training
Zhihong Chen, Yuhao Du, Jinpeng Hu +4
Medical vision-and-language pre-training provides a feasible solution to extract effective vision-and-language representations from medical images and texts. However, few studies h…
Cross-modal Memory Networks for Radiology Report Generation
Zhihong Chen, Yaling Shen, Yan Song +1
Medical imaging plays a significant role in clinical practice of medical diagnosis, where the text reports of the images are essential in understanding them and facilitating later…
Graph Enhanced Contrastive Learning for Radiology Findings Summarization
Jinpeng Hu, Zhuo Li, Zhihong Chen +3
The impression section of a radiology report summarizes the most prominent observation from the findings section and is the most important section for radiologists to communicate t…
Word Graph Guided Summarization for Radiology Findings
Jinpeng Hu, Jianling Li, Zhihong Chen +4
Radiology reports play a critical role in communicating medical findings to physicians. In each report, the impression section summarizes essential radiology findings. In clinical…
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Benyou Wang, Qianqian Xie, Jiahuan Pei +4
Pre-trained language models (PLMs) have been the de facto paradigm for most natural language processing (NLP) tasks. This also benefits biomedical domain: researchers from informat…
Generating Radiology Reports via Memory-driven Transformer
Zhihong Chen, Yan Song, Tsung-Hui Chang +1
Medical imaging is frequently used in clinical practice and trials for diagnosis and treatment. Writing imaging reports is time-consuming and can be error-prone for inexperienced r…