68 citations · 102 across the 22 of their papers we have counts for
22 papers
A Multi-Language Object-Oriented Programming Benchmark for Large Language Models
Shuai Wang, Liang Ding, Li Shen +4
Establishing fair and robust benchmarks is essential for evaluating intelligent code generation by large language models (LLMs). Our survey of 35 existing benchmarks uncovers three…
CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference
Guanyu Xu, Zhiwei Hao, Li Shen +5
The impressive performance of transformer models has sparked the deployment of intelligent applications on resource-constrained edge devices. However, ensuring high-quality service…
Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learning
Ziyi Zhang, Li Shen, Deheng Ye +5
Text-to-multiview (T2MV) diffusion models have shown great promise in generating multiple views of a scene from a single text prompt. While few-step backbones enable real-time T2MV…
Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG
Wenbin Wang, Yongcheng Jing, Liang Ding +5
High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs). To overcome the limitations of existing methods, this paper shifts away f…
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning
Zhiwei Hao, Jianyuan Guo, Li Shen +3
Recent advancements in multimodal fusion have witnessed the remarkable success of vision-language (VL) models, which excel in various multimodal applications such as image captioni…
: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding
Shuai Wang, Liang Ding, Li Shen +4
Large language models (LLMs) have shown remarkable capabilities in code generation. However, the effects of hallucinations (e.g., output noise) make it particularly challenging for…
SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
Anke Tang, Li Shen, Yong Luo +5
Deep model training on extensive datasets is increasingly becoming cost-prohibitive, prompting the widespread adoption of deep model fusion techniques to leverage knowledge from pr…
Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
Anke Tang, Li Shen, Yong Luo +4
Solving multi-objective optimization problems for large deep neural networks is a challenging task due to the complexity of the loss landscape and the expensive computational cost…
FusionBench: A Unified Library and Comprehensive Benchmark for Deep Model Fusion
Anke Tang, Li Shen, Yong Luo +5
Deep model fusion is an emerging technique that unifies the predictions or parameters of several deep neural networks into a single better-performing model in a cost-effective and…
Federated Learning with Only Positive Labels by Exploring Label Correlations
Xuming An, Dui Wang, Li Shen +5
Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification pro…
Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
Changtong Zan, Liang Ding, Li Shen +3
Translation-tailored Large language models (LLMs) exhibit remarkable translation capabilities, even competing with supervised-trained commercial translation systems. However, off-t…
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
Shuai Wang, Liang Ding, Li Shen +3
Advancing automated programming necessitates robust and comprehensive code generation benchmarks, yet current evaluation frameworks largely neglect object-oriented programming (OOP…
WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
Wenbin Wang, Liang Ding, Li Shen +3
Sentiment analysis is rapidly advancing by utilizing various data modalities (e.g., text, image). However, most previous works relied on superficial information, neglecting the inc…
Revisiting Knowledge Distillation for Autoregressive Language Models
Qihuang Zhong, Liang Ding, Li Shen +3
Knowledge distillation (KD) is a common approach to compress a teacher model to reduce its inference cost and memory footprint, by training a smaller student model. However, in the…
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts
Anke Tang, Li Shen, Yong Luo +3
Merging various task-specific Transformer-based models trained on different tasks into a single unified model can execute all the tasks concurrently. Previous methods, exemplified…
FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models
Feihong He, Gang Li, Fuhui Sun +4
The rapid development of generative diffusion models has significantly advanced the field of style transfer. However, most current style transfer methods based on diffusion models…
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Anke Tang, Xianglin Luo, Li Shen +5
Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct…
Federated Learning with Manifold Regularization and Normalized Update Reaggregation
Xuming An, Li Shen, Han Hu +1
Federated Learning (FL) is an emerging collaborative machine learning framework where multiple clients train the global model without sharing their own datasets. In FL, the model i…
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models
Miaoxi Zhu, Qihuang Zhong, Li Shen +4
Quantization is a promising approach for reducing memory overhead and accelerating inference, especially in large pre-trained language model (PLM) scenarios. While having no access…
Learning from models beyond fine-tuning
Hongling Zheng, Li Shen, Anke Tang +5
Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primaril…
Parameter Efficient Multi-task Model Fusion with Partial Linearization
Anke Tang, Li Shen, Yong Luo +5
Large pre-trained models have enabled significant advances in machine learning and served as foundation components. Model fusion methods, such as task arithmetic, have been proven…
Deep Model Fusion: A Survey
Weishi Li, Yong Peng, Miao Zhang +3
Deep model fusion/merging is an emerging technique that merges the parameters or predictions of multiple deep learning models into a single one. It combines the abilities of differ…