5 citations · 11 across the 17 of their papers we have counts for
11 papers · 1 filter
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models
Xiaolin Hu, Xiang Cheng, Peiyu Liu +4
Low-rank adaptation (LoRA) reduces the computational and memory demands of fine-tuning large language models (LLMs) by approximating updates with low-rank matrices. However, low-ra…
MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding
Qinzhuo Wu, Weikai Xu, Wei Liu +6
Recently, mobile AI agents based on VLMs have been gaining increasing attention. These works typically utilize VLM as a foundation, fine-tuning it with instruction-based mobile dat…
HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
Yuhan Chen, Ang Lv, Jian Luan +2
Many positional encodings (PEs) are designed to exhibit long-term decay, based on an entrenched and long-standing inductive opinion: tokens farther away from the current position c…
PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
Qibin Wang, Xiaolin Hu, Weikai Xu +3
Low-rank adaptation (LoRA) and its variants have recently gained much interest due to their ability to avoid excessive inference costs. However, LoRA still encounters the following…
Mixture of Diverse Size Experts
Manxi Sun, Wei Liu, Jian Luan +2
The Sparsely-Activated Mixture-of-Experts (MoE) has gained increasing popularity for scaling up large language models (LLMs) without exploding computational costs. Despite its succ…
ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback
Qinzhuo Wu, Wei Liu, Jian Luan +1
Recently, tool-augmented LLMs have gained increasing attention. Given an instruction, tool-augmented LLMs can interact with various external tools in multiple rounds and provide a…
Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations
Bowen Shen, Zheng Lin, Daren Zha +4
Structured pruning fundamentally reduces computational and memory overheads of large language models (LLMs) and offers a feasible solution for end-side LLM deployment. Structurally…
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
Shihan Deng, Weikai Xu, Hongda Sun +8
With the remarkable advancements of large language models (LLMs), LLM-based agents have become a research hotspot in human-computer interaction. However, there is a scarcity of ben…
SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM
Quandong Wang, Yuxuan Yuan, Xiaoyu Yang +6
While Large Language Models (LLMs) have achieved remarkable success in various fields, the efficiency of training and inference remains a major challenge. To address this issue, we…
ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval
Yuanhang Zheng, Peng Li, Wei Liu +3
Tool learning aims to extend the capabilities of large language models (LLMs) with external tools. A major challenge in tool learning is how to support a large number of tools, inc…
A Comprehensive Evaluation of Quantization Strategies for Large Language Models
Renren Jin, Jiangcun Du, Wuwei Huang +4
Increasing the number of parameters in large language models (LLMs) usually improves performance in downstream tasks but raises compute and memory costs, making deployment difficul…