110 citations · 189 across the 19 of their papers we have counts for
19 papers
IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment
Zichen Zhu, Yuheng Sun, Mingxuan Zhu +11
Current image editing software often hinges on fixed filters or expert tuning, leaving a gap between amateur users' intent and outcomes. Creations by generative models may contain…
CharTool: Tool-Integrated Visual Reasoning for Chart Understanding
Situo Zhang, Yifan Zhang, Zichen Zhu +6
Charts are ubiquitous in scientific and financial literature for presenting structured data. However, chart reasoning remains challenging for multimodal large language models (MLLM…
A Scientific Reasoning Model for Organic Synthesis Procedure Generation
Guoqing Liu, Junren Li, Zihan Zhao +7
Solving computer-aided synthesis planning is essential for enabling fully automated, robot-assisted synthesis workflows and improving the efficiency of drug discovery. A key challe…
ChemDFM-R: A Chemical Reasoning LLM Enhanced with Atomized Chemical Knowledge
Zihan Zhao, Ziping Wan, Lu Chen +12
Atomized chemical knowledge, such as functional group information of molecules and reactions, plays a pivotal intermediate role in the reasoning process that connects molecular str…
RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning
Situo Zhang, Hanqi Li, Lu Chen +7
Retrosynthetic planning is a cornerstone of organic synthesis and drug discovery. Yet existing AI methods often rely on pattern matching rather than transferable chemical reasoning…
ProgRM: Build Better GUI Agents with Progress Rewards
Danyang Zhang, Situo Zhang, Ziyue Yang +5
LLM-based (Large Language Model) GUI (Graphical User Interface) agents can potentially reshape our daily lives significantly. However, current LLM-based GUI agents suffer from the…
SciDFM: A Large Language Model with Mixture-of-Experts for Science
Liangtai Sun, Danyu Luo, Da Ma +7
Recently, there has been a significant upsurge of interest in leveraging large language models (LLMs) to assist scientific discovery. However, most LLMs only focus on general scien…
ChemDFM-X: Towards Large Multimodal Model for Chemistry
Zihan Zhao, Bo Chen, Jingpiao Li +10
Rapid developments of AI tools are expected to offer unprecedented assistance to the research of natural science including chemistry. However, neither existing unimodal task-specif…
Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding
Hongshen Xu, Lu Chen, Zihan Zhao +4
The growing prevalence of visually rich documents, such as webpages and scanned/digital-born documents (images, PDFs, etc.), has led to increased interest in automatic document und…
MULTI: Multimodal Understanding Leaderboard with Text and Images
Zichen Zhu, Yang Xu, Lu Chen +11
The rapid development of multimodal large language models (MLLMs) raises the question of how they compare to human performance. While existing datasets often feature synthetic or o…
Developing ChemDFM as a large language foundation model for chemistry
Zihan Zhao, Da Ma, Lu Chen +11
Artificial intelligence (AI) has played an increasingly important role in chemical research. However, most models currently used in chemistry are specialist models that require tra…
SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research
Liangtai Sun, Yang Han, Zihan Zhao +5
Recently, there has been growing interest in using Large Language Models (LLMs) for scientific research. Numerous benchmarks have been proposed to evaluate the ability of LLMs for…
Large Language Models Are Semi-Parametric Reinforcement Learning Agents
Danyang Zhang, Lu Chen, Situo Zhang +3
Inspired by the insights in cognitive science with respect to human memory and reasoning mechanism, a novel evolvable LLM-based (Large Language Model) agent framework is proposed a…
Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction
Danyang Zhang, Zhennan Shen, Rui Xie +8
The Graphical User Interface (GUI) is pivotal for human interaction with the digital world, enabling efficient device control and the completion of complex tasks. Recent progress i…
TIE: Topological Information Enhanced Structural Reading Comprehension on Web Pages
Zihan Zhao, Lu Chen, Ruisheng Cao +3
Recently, the structural reading comprehension (SRC) task on web pages has attracted increasing research interests. Although previous SRC work has leveraged extra information such…
WebSRC: A Dataset for Web-Based Structural Reading Comprehension
Xingyu Chen, Zihan Zhao, Lu Chen +5
Web search is an essential way for humans to obtain information, but it's still a great challenge for machines to understand the contents of web pages. In this paper, we introduce…
An Investigation on Different Underlying Quantization Schemes for Pre-trained Language Models
Zihan Zhao, Yuncong Liu, Lu Chen +3
Recently, pre-trained language models like BERT have shown promising performance on multiple natural language processing tasks. However, the application of these models has been li…
From Pixel to Patch: Synthesize Context-aware Features for Zero-shot Semantic Segmentation
Zhangxuan Gu, Siyuan Zhou, Li Niu +2
Zero-shot learning has been actively studied for image classification task to relieve the burden of annotating image labels. Interestingly, semantic segmentation task requires more…
Context-aware Feature Generation for Zero-shot Semantic Segmentation
Zhangxuan Gu, Siyuan Zhou, Li Niu +2
Existing semantic segmentation models heavily rely on dense pixel-wise annotations. To reduce the annotation pressure, we focus on a challenging task named zero-shot semantic segme…