20 citations · 30 across the 22 of their papers we have counts for
22 papers
MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction
Zitian Tang, Xu Zhang, Jianbo Yuan +4
Multimodal Large Language Models (MLLMs) have recently demonstrated promising capabilities in multimodal coding tasks such as chart-to-code generation. However, existing methods pr…
Learning Compact Video Representations for Efficient Long-form Video Understanding in Large Multimodal Models
Yuxiao Chen, Jue Wang, Zhikang Zhang +8
With recent advancements in video backbone architectures, combined with the remarkable achievements of large language models (LLMs), the analysis of long-form videos spanning tens…
Model Merging Scaling Laws in Large Language Models
Yuanyi Wang, Yanggan Gu, Yiming Zhang +6
We study empirical scaling laws for language model merging measured by cross-entropy. Despite its wide practical use, merging lacks a quantitative rule that predicts returns as we…
InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization
Yuhang Liu, Zeyu Liu, Shuanghe Zhu +10
The emergence of Multimodal Large Language Models (MLLMs) has propelled the development of autonomous agents that operate on Graphical User Interfaces (GUIs) using pure visual inpu…
Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models
Zeyu Liu, Yuhang Liu, Guanghao Zhu +9
Recent advancements in large language models (LLMs) have demonstrated substantial progress in reasoning capabilities, such as DeepSeek-R1, which leverages rule-based reinforcement…
InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
Congkai Xie, Shuo Cai, Wenjun Wang +17
Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have made significant advancements in reasoning capabilities. However, they still face challenges such as…
Unconstrained Model Merging for Enhanced LLM Reasoning
Yiming Zhang, Baoyi He, Shengyu Zhang +12
Recent advancements in building domain-specific large language models (LLMs) have shown remarkable success, especially in tasks requiring reasoning abilities like logical inference…
BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data
Xuwu Wang, Qiwen Cui, Yunzhe Tao +16
Large language models (LLMs) have become increasingly pivotal across various domains, especially in handling complex data types. This includes structured data processing, as exempl…
Law of Vision Representation in MLLMs
Shijia Yang, Bohan Zhai, Quanzeng You +3
We present the "Law of Vision Representation" in multimodal large language models (MLLMs). It reveals a strong correlation between the combination of cross-modal alignment, corresp…
An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
Ziwei Chai, Guoyin Wang, Jing Su +8
We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs. Our framework represents expert LLMs as special exper…
How Can LLM Guide RL? A Value-Based Approach
Shenao Zhang, Sirui Zheng, Shuqi Ke +6
Reinforcement learning (RL) has become the de facto standard practice for sequential decision-making problems by improving future acting policies with feedback. However, RL algorit…
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
Yiqi Wang, Wentao Chen, Xiaotian Han +7
Strong Artificial Intelligence (Strong AI) or Artificial General Intelligence (AGI) with abstract reasoning ability is the goal of next-generation AI. Recent advancements in Large…
InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks
Xueyu Hu, Ziyu Zhao, Shuang Wei +14
In this paper, we introduce InfiAgent-DABench, the first benchmark specifically designed to evaluate LLM-based agents on data analysis tasks. These tasks require agents to end-to-e…
InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
Xiaotian Han, Quanzeng You, Yongfei Liu +9
Multi-modal Large Language Models (MLLMs) are increasingly prominent in the field of artificial intelligence. These models not only excel in traditional vision-language tasks but a…
Improving In-Context Learning in Diffusion Models with Visual Context-Modulated Prompts
Tianqi Chen, Yongfei Liu, Zhendong Wang +4
In light of the remarkable success of in-context learning in large language models, its potential extension to the vision domain, particularly with visual foundation models like St…
Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis
Xiaohui Chen, Yongfei Liu, Yingxiang Yang +4
Recent advancements in text-to-image (T2I) generative models have shown remarkable capabilities in producing diverse and imaginative visuals based on text prompts. Despite the adva…
Self-Infilling Code Generation
Lin Zheng, Jianbo Yuan, Zhi Zhang +2
This work introduces self-infilling code generation, a general framework that incorporates infilling operations into auto-regressive decoding. Our approach capitalizes on the obser…
Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
Qianli Ma, Haotian Zhou, Tingkai Liu +4
Recent years have seen considerable advancements in multi-step reasoning with Large Language Models (LLMs). The previous studies have elucidated the merits of integrating feedback…
LEMON: Lossless model expansion
Yite Wang, Jiahao Su, Hanlin Lu +6
Scaling of deep neural networks, especially Transformers, is pivotal for their surging performance and has further led to the emergence of sophisticated reasoning capabilities in f…
DavIR: Data Selection via Implicit Reward for Large Language Models
Haotian Zhou, Tingkai Liu, Qianli Ma +5
We introduce DavIR, a model-based data selection method for post-training Large Language Models. DavIR generalizes Reducible Holdout Loss to core-set selection problem of causal la…
Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling
Huangjie Zheng, Zhendong Wang, Jianbo Yuan +5
Diffusion models excel at generating photo-realistic images but come with significant computational costs in both training and sampling. While various techniques address these comp…
Let Models Speak Ciphers: Multiagent Debate through Embeddings
Chau Pham, Boyi Liu, Yingxiang Yang +6
Discussion and debate among Large Language Models (LLMs) have gained considerable attention due to their potential to enhance the reasoning ability of LLMs. Although natural langua…