9 citations · 41 across the 49 of their papers we have counts for
22 papers · 1 filter
HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks
Ting Zhou, Daoyuan Chen, Qirui Jiao +3
Evaluating the nuanced human-centric video understanding capabilities of Multimodal Large Language Models (MLLMs) remains a great challenge, as existing benchmarks often overlook t…
Provable Scaling Laws for the Test-Time Compute of Large Language Models
Yanxi Chen, Xuchen Pan, Yaliang Li +2
We propose two simple, principled and practical algorithms that enjoy provable scaling laws for the test-time compute of large language models (LLMs). The first one is a two-stage…
Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning
Pengfei He, Zitao Li, Yue Xing +3
Zero-shot reasoning methods with Large Language Models (LLMs) offer significant advantages including great generalization to novel tasks and reduced dependency on human-crafted exa…
LLM-Based Multi-Agent Systems are Scalable Graph Generative Models
Jiarui Ji, Runlin Lei, Jialing Bi +6
The structural properties of naturally arising social graphs are extensively studied to understand their evolution. Prior approaches for modeling network dynamics typically rely on…
GenSim: A General Social Simulation Platform with Large Language Model based Agents
Jiakai Tang, Heyang Gao, Xuchen Pan +11
With the rapid advancement of large language models (LLMs), recent years have witnessed many promising studies on leveraging LLM-based agents to simulate human social behavior. Whi…
Agent-Oriented Planning in Multi-Agent Systems
Ao Li, Yuexiang Xie, Songze Li +3
Through the collaboration of multiple LLM-empowered agents possessing diverse expertise and tools, multi-agent systems achieve impressive progress in solving real-world problems. G…
Understanding Byzantine Robustness in Federated Learning with A Black-box Server
Fangyuan Zhao, Yuexiang Xie, Xuebin Ren +3
Federated learning (FL) becomes vulnerable to Byzantine attacks where some of participators tend to damage the utility or discourage the convergence of the learned model via sendin…
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective
Zhen Qin, Daoyuan Chen, Wenhao Zhang +5
The rapid development of large language models (LLMs) has been witnessed in recent years. Based on the powerful LLMs, multi-modal LLMs (MLLMs) extend the modality from text to a br…
Exploring Selective Layer Fine-Tuning in Federated Learning
Yuchang Sun, Yuexiang Xie, Bolin Ding +2
Federated learning (FL) has emerged as a promising paradigm for fine-tuning foundation models using distributed data in a privacy-preserving manner. Under limited computational res…
Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Models
Qirui Jiao, Daoyuan Chen, Yilun Huang +3
High-performance Multimodal Large Language Models (MLLMs) are heavily dependent on data quality. To advance fine-grained image recognition within MLLMs, we introduce a novel data s…
Very Large-Scale Multi-Agent Simulation in AgentScope
Xuchen Pan, Dawei Gao, Yuexiang Xie +6
Recent advances in large language models (LLMs) have opened new avenues for applying multi-agent systems in very large-scale simulations. However, there remain several challenges w…
Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
Yanxi Chen, Yaliang Li, Bolin Ding +1
This work presents an analytical framework for the design and analysis of LLM-based algorithms, i.e., algorithms that contain one or multiple calls of large language models (LLMs)…
Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development
Daoyuan Chen, Haibin Wang, Yilun Huang +4
The emergence of multimodal large models has advanced artificial intelligence, introducing unprecedented levels of performance and functionality. However, optimizing these models r…
VertiMRF: Differentially Private Vertical Federated Data Synthesis
Fangyuan Zhao, Zitao Li, Xuebin Ren +3
Data synthesis is a promising solution to share data for various downstream analytic tasks without exposing raw data. However, without a theoretical privacy guarantee, a synthetic…
FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model
Feijie Wu, Zitao Li, Yaliang Li +2
Large language models (LLMs) show amazing performance on many domain-specific tasks after fine-tuning with some appropriate data. However, many domain-specific data are privately d…
BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
Ce Ge, Zhijian Ma, Daoyuan Chen +2
Large language models have demonstrated remarkable capabilities across various tasks, primarily attributed to the utilization of diversely sourced data. However, the impact of pret…
AAA: an Adaptive Mechanism for Locally Differential Private Mean Estimation
Fei Wei, Ergute Bao, Xiaokui Xiao +2
Local differential privacy (LDP) is a strong privacy standard that has been adopted by popular software systems. The main idea is that each individual perturbs their own data local…
Improving LoRA in Privacy-preserving Federated Learning
Youbang Sun, Zitao Li, Yaliang Li +1
Low-rank adaptation (LoRA) is one of the most popular task-specific parameter-efficient fine-tuning (PEFT) methods on pre-trained language models for its good performance and compu…
A Bargaining-based Approach for Feature Trading in Vertical Federated Learning
Yue Cui, Liuyi Yao, Zitao Li +3
Vertical Federated Learning (VFL) has emerged as a popular machine learning paradigm, enabling model training across the data and the task parties with different features about the…
An Auction-based Marketplace for Model Trading in Federated Learning
Yue Cui, Liuyi Yao, Yaliang Li +3
Federated learning (FL) is increasingly recognized for its efficacy in training models using locally distributed data. However, the proper valuation of shared data in this collabor…
AgentScope: A Flexible yet Robust Multi-Agent Platform
Dawei Gao, Zitao Li, Xuchen Pan +16
With the rapid advancement of Large Language Models (LLMs), significant progress has been made in multi-agent applications. However, the complexities in coordinating agents' cooper…
EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
Xuchen Pan, Yanxi Chen, Yaliang Li +2
This work introduces EE-Tuning, a lightweight and economical solution to training/tuning early-exit large language models (LLMs). In contrast to the common approach of full-paramet…