9 citations · 71 across the 58 of their papers we have counts for
27 papers · 1 filter
HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks
Ting Zhou, Daoyuan Chen, Qirui Jiao +3
Evaluating the nuanced human-centric video understanding capabilities of Multimodal Large Language Models (MLLMs) remains a great challenge, as existing benchmarks often overlook t…
Provable Scaling Laws for the Test-Time Compute of Large Language Models
Yanxi Chen, Xuchen Pan, Yaliang Li +2
We propose two simple, principled and practical algorithms that enjoy provable scaling laws for the test-time compute of large language models (LLMs). The first one is a two-stage…
LLM-Based Multi-Agent Systems are Scalable Graph Generative Models
Jiarui Ji, Runlin Lei, Jialing Bi +6
The structural properties of naturally arising social graphs are extensively studied to understand their evolution. Prior approaches for modeling network dynamics typically rely on…
Agent-Oriented Planning in Multi-Agent Systems
Ao Li, Yuexiang Xie, Songze Li +3
Through the collaboration of multiple LLM-empowered agents possessing diverse expertise and tools, multi-agent systems achieve impressive progress in solving real-world problems. G…
Understanding Byzantine Robustness in Federated Learning with A Black-box Server
Fangyuan Zhao, Yuexiang Xie, Xuebin Ren +3
Federated learning (FL) becomes vulnerable to Byzantine attacks where some of participators tend to damage the utility or discourage the convergence of the learned model via sendin…
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective
Zhen Qin, Daoyuan Chen, Wenhao Zhang +5
The rapid development of large language models (LLMs) has been witnessed in recent years. Based on the powerful LLMs, multi-modal LLMs (MLLMs) extend the modality from text to a br…
Exploring Selective Layer Fine-Tuning in Federated Learning
Yuchang Sun, Yuexiang Xie, Bolin Ding +2
Federated learning (FL) has emerged as a promising paradigm for fine-tuning foundation models using distributed data in a privacy-preserving manner. Under limited computational res…
Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Models
Qirui Jiao, Daoyuan Chen, Yilun Huang +3
High-performance Multimodal Large Language Models (MLLMs) are heavily dependent on data quality. To advance fine-grained image recognition within MLLMs, we introduce a novel data s…
Very Large-Scale Multi-Agent Simulation in AgentScope
Xuchen Pan, Dawei Gao, Yuexiang Xie +6
Recent advances in large language models (LLMs) have opened new avenues for applying multi-agent systems in very large-scale simulations. However, there remain several challenges w…
Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
Yanxi Chen, Yaliang Li, Bolin Ding +1
This work presents an analytical framework for the design and analysis of LLM-based algorithms, i.e., algorithms that contain one or multiple calls of large language models (LLMs)…
Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development
Daoyuan Chen, Haibin Wang, Yilun Huang +4
The emergence of multimodal large models has advanced artificial intelligence, introducing unprecedented levels of performance and functionality. However, optimizing these models r…
VertiMRF: Differentially Private Vertical Federated Data Synthesis
Fangyuan Zhao, Zitao Li, Xuebin Ren +3
Data synthesis is a promising solution to share data for various downstream analytic tasks without exposing raw data. However, without a theoretical privacy guarantee, a synthetic…
FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model
Feijie Wu, Zitao Li, Yaliang Li +2
Large language models (LLMs) show amazing performance on many domain-specific tasks after fine-tuning with some appropriate data. However, many domain-specific data are privately d…
AspirinSum: an Aspect-based utility-preserved de-identification Summarization framework
Ya-Lun Li
Due to the rapid advancement of Large Language Model (LLM), the whole community eagerly consumes any available text data in order to train the LLM. Currently, large portion of the…
BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
Ce Ge, Zhijian Ma, Daoyuan Chen +2
Large language models have demonstrated remarkable capabilities across various tasks, primarily attributed to the utilization of diversely sourced data. However, the impact of pret…
ChartThinker: A Contextual Chain-of-Thought Approach to Optimized Chart Summarization
Mengsha Liu, Daoyuan Chen, Yaliang Li +2
Data visualization serves as a critical means for presenting data and mining its valuable insights. The task of chart summarization, through natural language processing techniques,…
Review of Data-centric Time Series Analysis from Sample, Feature, and Period
Chenxi Sun, Hongyan Li, Yaliang Li +1
Data is essential to performing time series analysis utilizing machine learning approaches, whether for classic models or today's large language models. A good time-series dataset…
Dynamic Demonstration Retrieval and Cognitive Understanding for Emotional Support Conversation
Zhe Xu, Daoyuan Chen, Jiayi Kuang +3
Emotional Support Conversation (ESC) systems are pivotal in providing empathetic interactions, aiding users through negative emotional states by understanding and addressing their…
Improving LoRA in Privacy-preserving Federated Learning
Youbang Sun, Zitao Li, Yaliang Li +1
Low-rank adaptation (LoRA) is one of the most popular task-specific parameter-efficient fine-tuning (PEFT) methods on pre-trained language models for its good performance and compu…
A Bargaining-based Approach for Feature Trading in Vertical Federated Learning
Yue Cui, Liuyi Yao, Zitao Li +3
Vertical Federated Learning (VFL) has emerged as a popular machine learning paradigm, enabling model training across the data and the task parties with different features about the…
An Auction-based Marketplace for Model Trading in Federated Learning
Yue Cui, Liuyi Yao, Yaliang Li +3
Federated learning (FL) is increasingly recognized for its efficacy in training models using locally distributed data. However, the proper valuation of shared data in this collabor…
Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
Shen Li, Liuyi Yao, Jinyang Gao +2
To support various applications, a prevalent and efficient approach for business owners is leveraging their valuable datasets to fine-tune a pre-trained LLM through the API provide…
AgentScope: A Flexible yet Robust Multi-Agent Platform
Dawei Gao, Zitao Li, Xuchen Pan +16
With the rapid advancement of Large Language Models (LLMs), significant progress has been made in multi-agent applications. However, the complexities in coordinating agents' cooper…
Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources
Jiamu Bai, Daoyuan Chen, Bingchen Qian +2
Federated Learning (FL) has recently been applied to the parameter-efficient fine-tuning of Large Language Models (LLMs). While promising, it raises significant challenges due to t…